The BDNF Val66Met gene polymorphism is a relevant factor explaining inter-individual differences to TMS responses in studies of the motor system. However, whether this variant also contributes to TMS-induced memory effects, as well as their underlying brain mechanisms, remains unexplored. In this investigation, we applied rTMS during encoding of a visual memory task either over the left frontal cortex (LFC; experimental condition) or the cranial vertex (control condition). Subsequently, individuals underwent a recognition memory phase during a functional MRI acquisition. We included 43 young volunteers and classified them as 19 Met allele carriers and 24 as Val/Val individuals. The results revealed that rTMS delivered over LFC compared to vertex stimulation resulted in reduced memory performance only amongst Val/Val allele carriers. This genetic group also exhibited greater fMRI brain activity during memory recognition, mainly over frontal regions, which was positively associated with cognitive performance. We concluded that BDNF Val66Met gene polymorphism, known to exert a significant effect on neuroplasticity, modulates the impact of rTMS both at the cognitive as well as at the associated brain networks expression levels. This data provides new insights on the brain mechanisms explaining cognitive inter-individual differences to TMS, and may inform future, more individually-tailored rTMS interventions.