Persons with occupational exposure to swine might be at disproportionate risk for zoonotic swine influenza A virus. To evaluate human antibody responses, we tested serum or plasma from swine veterinarian, farm employee, and general population cohorts by hemagglutination inhibition assays against representative swine and human seasonal influenza vaccine strains. We analyzed hemagglutination inhibition data by antigenic cartography to assess strain relationships and reproduction number modeling to evaluate pandemic potential using age-stratified immunity profiles. Occupationally exposed groups had lower human seasonal vaccine uptake (45.5% vs. 70%) and lower odds of seropositivity to several H1 and H3 strains from swine than did general population cohorts. One swine strain exhibited significant antigenic drift (3.62 antigenic units) from its nearest vaccine strain. Multiple strains required lower reproduction number thresholds for pandemic spread (1.09–1.35) than recorded pandemic strains (1.46–1.80), demonstrating that population immunity gaps heighten zoonotic risk to circulating swine H1 and H3 strains.