Built in 1908 for the "feeble minded and epileptics," the Eastern New York State Custodial Asylum was eventually renamed to Letchworth Village Home for the Feeble Minded and Epileptics after William Pryor Letchworth, a philanthropist. The facility was meant to be a departure from normal hospitals of the time and instead chose to symbolize a "village" structure. One story cottages dotted the landscape instead of a few massive hulking structures, due to the idea that a homelike institution and human contact would provide a better healing setting. These ideals caused much controversy at the time. Patients were separated by sex, and then separated again into three groups, never to mix contact: young and improvable, middle aged and industrious, and infirm and helpless. Education, training and vocational instruction were provided, and farm acreage was worked by the residents to raise food for the institution.

The institution was constantly plagued by overcrowding due to its revolutionary treatment ideals and popularity. Geraldo Rivera produced a segment on overcrowding of New York Institutions such as Willowbrook and Letchworth called "The Last Great Disgrace," which eventually grew into a full length documentary on ABC. Rivera won a Peabody Award in 1971 for his work. As the population of the institution dwindled, residents were placed into group homes until the facility completely closed in 1996.

The farmland which once grew food for its residents and staff now has been sold and turned into country clubs and golf courses. The rest of the Letchworth buildings remain vacant, with the exception of one group of six, which are used as a daycare facility, and a couple of other buildings which were converted for school use and town administrative purposes.

Encore Haverstraw is now looking to redevelop the property into 550 single-family townhouses, an inn and restaurant, and retail stores. Oone December 6, 2006, Stewart Hall suffered from a very serious fire. The building was going to be reused in the condo development. Demolition of the reminaing buildings is slated for sometime in 2007.

A development proposal for the property known as Encore Haverstraw envisions 550 single-family townhouses, an inn and restaurant, and retail space.