OBJECTIVES: To identify distinct latent groups of baseline levels and age-related decline in verbal episodic memory in middle-aged and older adults, and to identify factors associated with these trajectories. DESIGN: Longitudinal study of six data collections over a period of 10 years. SETTING: Population-based cohort in England. PARTICIPANTS: 9,515 community-dwelling adults aged 50-79 years. MEASUREMENTS: Six repeated measurements of immediate and delayed recall of 10 words over 10-year follow-up. Group-based trajectory modeling was used to identify patterns of baseline levels and subsequent decline in memory in two age categories (50-64 and 65-79 years), and to investigate associations between trajectories and baseline predictors of group membership (gender, education, household wealth, marital status, smoking and physical activity) and time-varying covariates (depressive symptoms and number of chronic conditions). RESULTS: Four trajectories were identified and labelled according to baseline status and decline in memory: very low/decline (9.8%), low/stable (40.2%), average/stable (39.5%) and good/stable (10.5%) in the younger group, and very low/rapid decline (15.7%), low/decline (32.0%), average/stable (38.8%), and good/stable (13.5%) in older participants. In people with stable or declining trajectories, a higher number of depressive symptoms and the presence of cardiovascular diseases were associated with worse memory. Female sex, younger age, and higher education, wealth and physical activity were consistently associated with more favourable trajectories. CONCLUSIONS: We identified four memory trajectories. Factors known to be associated with cognitive reserve (such as education, wealth and physical activity) were associated with better memory function while depressive symptoms and cardiovascular disease were associated with poorer memory. This suggests that interventions to reduce depressive symptoms and better manage cardiovascular risk factors and disease in midlife may help prevent or delay future memory decline.