A flourishing line of evidence has highlighted the encoding of speech sounds in the subcortical auditory system as being shaped by acoustic, linguistic, and musical experience and training. And while the heritability of auditory speech as well as nonspeech processing has been suggested, the genetic determinants of subcortical speech processing have not yet been uncovered. Here, we postulated that the serotonin transporter-linked polymorphic region (5-HTTLPR), a common functional polymorphism located in the promoter region of the serotonin transporter gene (SLC6A4), is implicated in speech encoding in the human subcortical auditory pathway. Serotonin has been shown as essential for modulating the brain response to sound both cortically and subcortically, yet the genetic factors regulating this modulation regarding speech sounds have not been disclosed. We recorded the frequency following response, a biomarker of the neural tracking of speech sounds in the subcortical auditory pathway, and cortical evoked potentials in 58 participants elicited to the syllable /ba/, which was presented >2000 times. Participants with low serotonin transporter expression had higher signal-to-noise ratios as well as a higher pitch strength representation of the periodic part of the syllable than participants with medium to high expression, possibly by tuning synaptic activity to the stimulus features and hence a more efficient suppression of noise. These results imply the 5-HTTLPR in subcortical auditory speech encoding and add an important, genetically determined layer to the factors shaping the human subcortical response to speech sounds. SIGNIFICANCE STATEMENT: The accurate encoding of speech sounds in the subcortical auditory nervous system is of paramount relevance for human communication, and it has been shown to be altered in different disorders of speech and auditory processing. Importantly, this encoding is plastic and can therefore be enhanced by language and music experience. Whether genetic factors play a role in speech encoding at the subcortical level remains unresolved. Here we show that a common polymorphism in the serotonin transporter gene relates to an accurate and robust neural tracking of speech stimuli in the subcortical auditory pathway. This indicates that serotonin transporter expression, eventually in combination with other polymorphisms, delimits the extent to which lifetime experience shapes the subcortical encoding of speech.