Antigenic evolution of the influenza A virus (IAV) hemagglutinin (HA) gene limits efforts to effectively control the spread of the virus in the population. Efforts to understand the mechanisms governing HA antigenic evolution typically examine the HA gene in isolation. This can ignore the importance of balancing HA receptor binding activities with the receptor-destroying activities of the viral neuraminidase (NA) to maintain viral fitness. We hypothesize that the need to maintain functional balance with NA significantly constrains the evolutionary potential of the HA. We use deep mutational scanning and show that variation in NA activity significantly reshapes the HA fitness landscape by modulating the overall mutational robustness of HA. Consistent with this, we observe that different NA backgrounds support the emergence of distinct repertoires of HA escape variants under neutralizing antibody pressure. Our results reveal a critical role for intersegment epistasis in influencing the evolutionary potential of the HA gene.