Poor geographic access can persist even when affordable and well-functioning health systems are in place, limiting efforts for universal health coverage (UHC). It is unclear how to balance support for health facilities and community health workers in UHC national strategies. The goal of this study was to evaluate how a health system strengthening (HSS) intervention aimed towards UHC affected the geographic access to primary care in a rural district of Madagascar. For this, we collected the fokontany of residence (lowest administrative unit) from nearly 300 000 outpatient consultations occurring in facilities of Ifanadiana district in 2014-2017 and in the subset of community sites supported by the HSS intervention. Distance from patients to facilities was accurately estimated following a full mapping of the district's footpaths and residential areas. We modelled per capita utilization for each fokontany through interrupted time-series analyses with control groups, accounting for nonlinear relationships with distance and travel time among other factors, and we predicted facility utilization across the district under a scenario with and without HSS. Finally, we compared geographic trends in primary care when combining utilization at health facilities and community sites. We find that facility-based interventions similar to those in UHC strategies achieved high utilization rates of 1-3 consultations per person year only among populations living in close proximity to facilities. We predict that scaling only facility-based HSS programmes would result in large gaps in access, with over 75% of the population unable to reach one consultation per person year. Community health delivery, available only for children under 5 years, provided major improvements in service utilization regardless of their distance from facilities, contributing to 90% of primary care consultations in remote populations. Our results reveal the geographic limits of current UHC strategies and highlight the need to invest on professionalized community health programmes with larger scopes of service.