The street was laid out and titled after William Warren Baldwin, who purchased much of the area, in the early nineteenth century.

The neighbourhood advanced in the early 20th century as the Jewish ghetto moved westward from its original location in The Ward towards what became the Kensington Market Jewish community, and the section became home to a number of Jewish stores and restaurants. Also close to the University of Toronto and encompassed by the area known as the "Student Ghetto" it also became a centre for the 1960s youth movements. Most notably it was the central location for draft dodgers from the Vietnam War in Toronto. The draft dodgers reformed a number of homes to shops catering to the new community.

In 1978 the first cafe on the block opened, and since then the area has develop into one of the best known restaurant districts in Toronto. Mandel's Creamery at 29 Baldwin Street, the last fragment of the block's Jewish heritage, closed in 1995 after 90 years of operation, though the Yiddish lettering on its window has been conserved by the John's Italian Caffe. The Ontario Jewish Archives is making an effort for keeping the window after the property was sold in 2015.

Baldwin Village is a hidden jewel nestled in the heart of downtown - a history-rich urban village that speaks for the culturally diverse heartbeat of Toronto.