Small Heath was likely developed for grazing livestock. It lies on the route from Birmingham to Coventry and it is thought to have been used for transporting animals between markets at the cities. The Coventry Road, around which Small Heath developed, was first recorded in 1226, when Birmingham was a medieval market town and Coventry a city of national importance.
The first recording of Small Heath was noted in 1461, which term applied to a narrow heath between Green Lane and the Coventry Road, where the baths and library were built later. The 1799 opening of the Warwick and Birmingham Canal (now the Grand Union Canal), from Digbeth to Warwick, defined the southern edge of this scattered rural community. In 1852, this definition was enforced with the opening of the Isambard Kingdom Brunel designed Birmingham and Oxford Junction Railway, which follows the same route.