Born to Mary Brady and Maurice S. Julian, he was raised in Chapel Hill, North Carolina, a childhood playmate of James Taylor. His father, Maurice S. Julian (1915–1993), opened a cycling shop in Chapel Hill, Julian's Cyclery, later becoming a clothier and opening Julian's in 1942. Julian pursued a degree in English at the University of North Carolina, Chapel Hill while working at his father's clothing store. Julian pleaded with his father to open a store of his own, and while his family was away for a month in the summer of 1969, Julian released one of his father's tenants from a lease, dropping out of school and at age 19 opening his first store, Alexander's Ambition. He subsequently bought out his father's interest, leading to a period where he and his father were in direct competition. In 1973 Julian campaigned for the Chapel Hill Board of Aldermen (town council), finishing fifth.