The Jervis Public Library Association was incorporated in 1894 by an act of the New York state legislature. The Association took its name from prominent American civil engineer John Bloomfield Jervis, who bequeathed funds, property, and his personal collection of books and papers for the purpose of founding a library facility for the citizens of his home town, Rome, NY. Although a downtown site for a new library building was debated, in the end it was decided that the Jervis homestead would make an ideal location for Rome's public library. Melvil Dewey, inventor of the famous Dewey Decimal System for classifying books, helped draft Jervis Library's bylaws.
Jervis Public Library is a member of the Central New York Library Resources Council.