583 Sherbourne Street, Toronto is one of the few remaining 19th-century residential properties in North St. James Town. Much of the neighbourhood's 19th century built environment was demolished in the mid-20th century during the St. James Town residential high-rise tower projects, which have since become synonymous with the name St. James Town.

583 Sherbourne Street, Toronto is presently surrounded on its north, south, and east sides by St. James Town Park West which gives the property the appearance of being in the middle of the park.  

The first occupant of 583 Sherbourne Street was Susan Jane Ellis (née Quirk) - who was the widow of William Henry Ellis.

From the Archives of Ontario's Philip William Ellis fonds: 

William Henry Ellis (b. ca. 1835), born into a family of jewellers, was an engineer and civil servant in Ontario during the nineteenth century, active in the construction of the Grand Trunk Railway.

William Henry Ellis, son of jeweller James E. Ellis, was born around 1835 in England. He immigrated to Canada with his family in the mid-1840s. He was an engineer and civil servant, constructing part of the Grand Trunk Railway and the Toronto Esplanade. William Henry Ellis and his brother Richard Yates Ellis were also members of the Hungerford Marble Co, and its activities were connected to William Henry Ellis' activities in railroad construction, providing transportation routes for the company.

Ellis was married in Toronto in 1855 to Susannah or Susan Jane Quirk, also born in England, who had immigrated to Canada in 1848. Their children included twins Phillip William Ellis and Matthew Cain Ellis, Charles Shuttleworth Ellis, Thomas Danvers Ellis, and William Gordon Ellis. William Henry Ellis died before the end of the nineteenth century.

Ground floor commercial modifications have occured at 583 Sherbourne Street, Toronto. Recent commercial tenants include Downtown Kabob Grill House.