Designing and controlling particle self-assembly into robust and reliable high-performance smart materials often involves crystalline ordering in curved spaces. Examples include carbon allotropes like graphene, synthetic materials such as colloidosomes, or biological systems like lipid membranes, solid domains on vesicles, or viral capsids. Despite the relevance of these structures, the irreversible deformation and failure of curved crystals is still mostly unexplored. Here, we report simulation results of the mechanical deformation of colloidal crystalline shells that illustrate the subtle role played by geometrically necessary topological defects in controlling plastic yielding and failure. We observe plastic deformation attributable to the migration and reorientation of grain boundary scars, a collective process assisted by the intermittent proliferation of disclination pairs or abrupt structural failure induced by crack nucleating at defects. Our results provide general guiding principles to optimize the structural and mechanical stability of curved colloidal crystals.