Visual impairment significantly alters the quality of life of people with Multiple Sclerosis (MS). The objective of this study was to identify predictors (independent variables) of visual outcomes, and to define their relationship with neurological disability and retinal atrophy when assessed by optical coherence tomography (OCT). We performed a cross-sectional analysis of 119 consecutive patients with MS, assessing vision using high contrast visual acuity (LogMar), 2.5% and 1.25% low contrast visual acuity (Sloan charts), and color vision (Hardy-Rand-Rittler plates). Quality of vision is a patient reported outcome based on an individual's unique perception of his or her vision and was assessed with the Visual Functioning Questionnaire-25 (VFQ-25) with the 10 neuro-ophthalmologic items. MS disability was assessed using the expanded disability status scale (EDSS), the MS functional composite (MSFC) and the brief repetitive battery-neuropsychology (BRB-N). Retinal atrophy was assessed using spectral domain OCT, measuring the thickness of the peripapillar retinal nerve fiber layer (pRNFL) and the volume of the ganglion cell plus inner plexiform layer (GCIPL). The vision of patients with MS was impaired, particularly in eyes with prior optic neuritis. Retinal atrophy (pRNFL and GCIPL) was closely associated with impaired low contrast vision and color vision, whereas the volume of the GCIPL showed a trend (p = 0.092) to be associated with quality of vision. Multiple regression analysis revealed that EDSS was an explanatory variable for high contrast vision after stepwise analysis, GCIPL volume for low contrast vision, and GCIPL volume and EDSS for color vision. The explanatory variables for quality of vision were high contrast vision and color vision. In summary, quality of vision in MS depends on the impairment of high contrast visual acuity and color vision due to the disease.