The detailed and chilling descriptions of physical violence in many slave narratives often overshadow the fact that slaveholders in the American South also relied on an intricate system of surveillance to control and exploit their slaves. In this essay, I argue that Frederick Douglass’s first autobiography Narrative of the Life of Frederick Douglass, an American Slave pictures surveillance, especially its production of space, as a central tool of slavery. The resulting spatial boundaries are invested with metaphorical meaning and serve as an expression of Douglass’s emancipation. The first part of the paper considers the plantation architecture and outlines how overseers, slave patrols and panopticism create seemingly impermeable boundaries for Douglass, which are both of physical and psychological nature. I further demonstrate how the architecture of Baltimore’s city space leads to a loosening of surveillance and allows Douglass to become literate. Finally, I draw on Jurij Lotmann’s theory of aesthetic space in order to analyze how spatial boundaries are crossed and metaphorical boundaries between whiteness and blackness are rendered contingent in the Narrative.