The Elms, named for the stately trees that graced the property, was built before the Civil War, about 1840, by Thomas S. Gray. a partner of Colonel Burhans in the conduct of a store and tannery. He had the house designed by an architect that he sent to Monticello, Thomas Jefferson's home, to garner ideas for his home. Colonel Burhans purchased the property and made extensive alterations and additions to the place, and upon his death willed it to his daughter, Sarah H. Burhans. She, in turn, left the property to her sister, Julia Isham, giving a life lease to her nieces, sisters Miss Clara R. Richards and Mrs. Mary R. Kellogg who also made additions to the property. A fire occurred in the 1940s, at which time it was sold to Philip and Ethel Roberts, who added cabins and a swimming pool, renaming it the Colony House. It has since been renamed, The Pillars. The gracious elm trees, for which it was originally named, all succumbed to Dutch elm disease in the 1940s. (See Colony House, also The Pillars.)
The Elms (84 Main Street)
Just beyond the Church of the Holy Cross, on the left, is the Pillars, formerly "The Elms", the home of Thomas Gray at the time he was partner with Colonel Burhans in the tannery business.
Colonel Burhans purchased the property and moved his family there from the dwelling which eventually became the Warren Inn. He made extensive alternations and additions and, upon his death, will The Elms to his daughter, Sarah H. Burhans, who passed it on to her sister, Julia Isham, with life lease going to Miss Clara Richard and Mrs. Mary Kellogg. The two sisters, Miss Richards and Mrs. Kellogg, donated the Richards Library to the town.
After World War II the property passed out of the Isham family and has been a motel with dining room and antique ship since that time.