According to the United States Census Bureau, the town has a total area of 124.1 km² (47.9 mi²). 124.0 km² (47.9 mi²) of it is land and 0.1 km² (0.04 mi²) of it is water. The total area is 0.08% water.
As of the census of 2000, there are 4,853 people, 1,769 households, and 1,144 families residing in the town. The population density is 39.2/km² (101.4/mi²). There are 1,905 housing units at an average density of 15.4/km² (39.8/mi²). The racial makeup of the town is 97.77% White, 0.21% African American, 0.14% Native American, 0.60% Asian, 0.08% Pacific Islander, 0.02% from other races, and 1.17% from two or more races. 0.54% of the population are Hispanic or Latino of any race.
There are 1,769 households out of which 31.5% have children under the age of 18 living with them, 50.9% are married couples living together, 9.6% have a female householder with no husband present, and 35.3% are non-families. 27.7% of all households are made up of individuals and 12.6% have someone living alone who is 65 years of age or older. The average household size is 2.45 and the average family size is 3.00.
In the town the population is spread out with 23.3% under the age of 18, 15.9% from 18 to 24, 23.8% from 25 to 44, 23.1% from 45 to 64, and 13.9% who are 65 years of age or older. The median age is 36 years. For every 100 females there are 103.1 males. For every 100 females age 18 and over, there are 105.4 males.
The median income for a household in the town is $41,283, and the median income for a family is $50,756. Males have a median income of $31,353 versus $25,160 for females. The per capita income for the town is $20,591. 7.9% of the population and 5.3% of families are below the poverty line. Out of the total people living in poverty, 7.7% are under the age of 18 and 7.5% are 65 or older.
In 1922, Randolph was also the setting for a silent movie called The Offenders