Robert Nugent Watts inherited the majority of Frederick George Heriot’s fortune, while previously acquiring Comfort Hall in December 1842 after he and his family had been living with Heriot for some time. Following Heriot’s death, Watts began to build a new residence on the property. The large estate was spread over several dozen acres, bounded on the south by Ferry Road (today’s Rue Saint-Georges), on the north by the Prevost River (known today as the St. Germain River), on the east by the St. Francis River, and finally on the west by Royal Road (today’s Chemin du Golf). As a recently elected member of Parliament with a quickly growing young family, Watts began the construction of an imposing stone manor right at the heart of the estate, with walls two to three feet thick. The house was located on what is today the intersection of Boulevard Gall and Des Lilas Avenue. Unlike other houses of the day, the manor featured large French-style windows, which brightened the rooms considerably by allowing in an abundance of natural light.
The new home was located right beside the old Comfort Hall, and boasted an impressive view of the St. Francis River and of Drummondville, which was only a small hamlet at the time. The residence was constructed according to a plan similar to that of Woodfield Villa in Sillery, which belonged to Watts’ father-in-law, William Sheppard. Sheppard experienced a heavy financial setback in 1847 due to the collapse of his lumber business. After this, he and his wife, Harriet Campbell, decided to sell their villa and move to Drummondville to be closer to their son-in-law and their daughter Charlotte’s family. They built a house in Wendover on the other side of the St. Francis River, calling their new home Fairymead. The Grantham Hall estate soon abounded in many lovely flower beds and landscaped areas, which was a mark of Harriet and William’s shared passion for botany and horticulture. The gardens added to the charm of an already impressive estate, including mills, various sheds and outbuildings, as well as stands of maple, elm, hemlock and pine trees where an abundance of wildlife could be found.
In 1907, the imposing slate-roofed manor house measured 40 feet by 75 feet. It boasted a large, 20-foot-wide balcony on three sides, and was surrounded by a landscaped lawn. The basement comprised eight rooms including a boiler room, with the others used for storage of such things as food, ice, and coal. A smoking room, a living room, a library, a kitchen and a dining room were located on the ground floor. There were four bedrooms on the second floor, as well as a billiards room and a bathroom. The attic was divided into servants’ quarters and additional storage space.