By 1800 Handsworth had become a fashionable rural location for the wealthy of Birmingham and the Black Country. A number of notable people lived here, including:
- James Watt in Heathfield House
- William Murdock at Sycamore Hill
- William Bullock at Hawthorne House
By 1840 the southern part of Handsworth parish roughly north of Soho Road and Lozells Road had developed into an upper middle-class suburb, much on a par with Edgbaston.
Perry Barr station on the Grand Junction Railway (Birmingham to Liverpool) opened in 1837, encouraging development to the north, and the opening of Handsworth and Smethwick station in 1854 on the Birmingham, Wolverhampton and Dudley Railway continued to the northwards development. By the end of the century the suburban area had reached into Handsworth Wood, houses there being large and few to the acre.
South of the Holyhead/ Soho Road, a much greater density of working-class housing was beginning to be developed. With an increasing population Handsworth Urban Sanitary Authority was set up in 1874.