In 1844, Brentwood began as Thompson Station and Suffolk Station, two new stations on the expansion of the mainline of the Long Island Rail Road. On March 21, 1851, it became the socialist community named Modern Times. The colony was established on land by Josiah Warren and Stephen Pearl Andrews. In 1864, it was renamed Brentwood after the town of Brentwood, Essex, in England. The colony failed after the Civil War. During the early 20th Century, Brentwood was home to the Ross Health Resort Onehtah and Pilgrim State Hospital.
The Brentwood Public Library was founded in 1937 with the mission of serving the information needs of its community.