Human-to-swine transmission of influenza A virus (IAV) repeatedly occurs, leading to sustained transmission and increased diversity in swine; human seasonal H3N2 introductions occurred in the 1990s and 2010s and were maintained in North American swine. Swine H3N2 were subsequently associated with zoonotic infections, highlighting the need to understand the risk of endemic swine IAV to humans. We quantified antigenic distances between swine H3N2 and human seasonal vaccine strains from 1973 to 2014 using a panel of monovalent antisera raised in pigs in hemagglutination inhibition (HI) assays. Swine H3N2 lineages retained closest antigenic similarity to human vaccine strains from the decade of incursion. Swine lineages from the 1990s were antigenically more similar to human vaccine strains of the mid-1990s but had substantial distance from recent human vaccine strains. In contrast, lineages from the 2010s were closer to human vaccine strains from 2011 and 2014 and most antigenically distant from human vaccine strains prior to 2007. HI assays using ferret antisera demonstrated that swine lineages from the 1990s and 2010s had significant fold-reduction compared with the homologous HI titer of the nearest pandemic preparedness candidate vaccine virus (CVV) or seasonal vaccine strain. The assessment of post-infection and post-vaccination human sera cohorts demonstrated limited cross-reactivity to swine H3N2 from the 1990s, especially in older adults born before 1970s. We identified swine strains to which humans are likely to lack population immunity or are not protected against by a current human seasonal vaccine or CVV to use in prioritizing future human CVV strain selection. IMPORTANCE Human H3N2 influenza A viruses spread to pigs in North America in the 1990s and more recently in the 2010s. These cross-species events led to sustained circulation and increased H3N2 diversity in pig populations. Evolution of H3N2 in swine led to a reduced similarity with human seasonal H3N2 and the vaccine strains used to protect human populations. We quantified the antigenic phenotypes and found that North American swine H3N2 lineages retained more antigenic similarity to historical human vaccine strains from the decade of incursion but had substantial difference compared with recent human vaccine strains. Additionally, pandemic preparedness vaccine strains demonstrated a loss in similarity with contemporary swine strains. Lastly, human sera revealed that although these adults had antibodies against human H3N2 strains, many had limited immunity to swine H3N2, especially older adults born before 1970. Antigenic assessment of swine H3N2 provides critical information for pandemic preparedness and candidate vaccine development.