Modern precipitation anomalies in the Altiplano region of South America are closely linked to the strength of the South American Summer Monsoon (SASM) which is influenced by large-scales climate features sourced in the tropics such as latitudinal shifts of the Intertropical Convergence Zone (ITCZ) and El Niño-Southern Oscillation (ENSO). However, the timing, direction and spatial extent of precipitation changes prior to the instrumental period are still largely unknown, preventing a better understanding of the long-term drivers of the SASM and their effects over the Altiplano. Here we present a detailed pollen reconstruction from a sedimentary sequence covering the period between 4500-1000 cal yr BP in Lago Chungará (18° S; 4570 masl), a high elevation lake in the southwestern margin of the Altiplano where precipitation is delivered almost exclusively during the mature phase of the SASM in the austral summer. We distinguish three well-defined centennial-scale anomalies, with dry conditions between 4100-3300 and 1600-1000 cal yr BP, and a conspicuous humid interval between 2400-1600 cal yr BP; which resulted from weakening and strengthening of the SASM respectively. Comparisons with other climate reconstructions from the Altiplano, the Atacama Desert, the Tropical Andes and the southwestern Atlantic coast reveal that - unlike the modern climatological controls - past precipitation anomalies at Lago Chungará were largely decoupled from north-south shifts in the ITCZ and ENSO. A regionally coherent pattern of centennial-scale SASM variations and a significant latitudinal gradient in precipitation responses suggest the contribution of an extra-tropical moisture source for the SASM, with significant effects over precipitation variability in the Southern Altiplano.