For autonomous vehicles (AVs) to receive general acceptance, society must have a positive perception about their safety impact on vulnerable road users. Using data from a statewide random-digit-dialing telephone survey of 1,001 adults, this paper examines how New Jersey residents perceive the safety impact of AVs on pedestrians, bicyclists, and people with ambulatory disability. It uses a combination of confirmatory factor analysis and ordered probit models. Confirmatory factor analysis is used to create latent variables on socioeconomic status and built environment. Three ordered probit models are used to examine people’s perception of AV safety impact on each of the three population groups. The models also examine how frequent walkers, bicyclists, and people with ambulatory disability perceive their own safety as well as the safety of the other two groups. All three models examine the effect of familiarity with AV, gender, age, income, education, race, ethnicity, number of vehicles in household, political party affiliation, as well as built environment and socioeconomic status of the municipalities where the survey respondents live. The analysis showed that men, people with familiarity with the AV concept, Democrats, bicyclists, and people with high household income generally have a positive perception about the safety impact of AVs. While frequent walkers are ambivalent about their own safety as pedestrians, bicyclists have a positive perception about their own safety and the safety of pedestrians, whereas people with ambulatory disability have a strong negative perception about their own safety. The models did not show statistically significant effects of socioeconomic status or built environment of municipalities on AV safety perception.