African Pygmies are hunter-gatherer populations from the equatorial rainforest that present the lowest height averages among humans. The biological basis and the putative adaptive role of the short stature of Pygmy populations has been one of the most intriguing topics for human biologists in the last century, which still remains elusive. Worldwide convergent evolution of the Pygmy size suggests the presence of strong selective pressures on the phenotype. We developed a novel approach to survey the genetic architecture of phenotypes and applied it to study the genomic covariation between allele frequencies and height measurements among Pygmy and non-Pygmy populations. Among the regions that were most associated with the phenotype, we identified a significant excess of genes with pivotal roles in bone homeostasis, such as PPPT3B and the height associated SUPT3H-RUNX2. We hypothesize that skeletal remodeling could be a key biological process underlying the Pygmy phenotype. In addition, we showed that these regions have most likely evolved under positive selection. These results constitute the first genetic hint of adaptive evolution in the African Pygmy phenotype, which is consistent with the independent emergence of the Pygmy height in other continents with similar environments.