We sometimes vividly remember things that did not happen, a phenomenon with general relevance, not only in the courtroom. It is unclear to what extent individual differences in false memories are driven by anatomical differences in memory-relevant brain regions. Here we show in humans that microstructural properties of different white matter tracts as quantified using diffusion tensor imaging are strongly correlated with true and false memory retrieval. To investigate these hypotheses, we tested a large group of participants in a version of the Deese-Roediger-McDermott paradigm (recall and recognition) and subsequently obtained diffusion tensor images. A voxel-based whole-brain level linear regression analysis was performedto relatefractional anisotropyto indices oftrue andfalse memory recall and recognition. True memory was correlated to diffusion anisotropy in the inferior longitudinal fascicle, the major connective pathway of the medial temporal lobe, whereas a greater proneness to retrieve false items was related to the superior longitudinal fascicle connecting frontoparietal structures. Our results show that individual differences in white matter microstructure underlie true and false memory performance.