The unexpected circulation of clade 2.3.4.4b H5N1 influenza viruses in dairy cattle and the transmission to diverse mammalian species poses a pandemic risk. We sought to explore cattle and human respiratory susceptibility to the 2.3.4.4b H5N1 virus. We establish long-term expandable cattle airway and mammary organoids. The 2.3.4.4b H5N1 virus exhibits high replicative fitness in cattle mammary organoids, recapitulating its remarkable mammary tropism. The virus also replicates robustly in cattle airway organoids, suggesting an underrecognized respiratory component in ongoing outbreaks. Interestingly, human airway and nasal organoids are highly susceptible to the 2.3.4.4b H5N1 virus. Yet, a novel organoid-based neutralization assay reveals that N1 antibodies in human sera had cross-neutralizing activity against the 2.3.4.4b H5N1 and ancestral H5N1-VN1194 viruses. The cross-neutralization, exclusively manifested in the organoid-based assay, is enhanced after seasonal influenza vaccination and diminished after depleting N1-specific antibodies. Therefore, cross-neutralizing N1 antibodies are likely limiting zoonotic infection by H5N1 viruses in humans.