This article documents a strong connection between unemployment and mental distress using data from the Spanish National Health Survey. We exploit the collapse of the construction sector to identify the causal effect of job losses in different segments of the Spanish labor market. Our results suggest that an increase of the unemployment rate by 10 percentage points due to the breakdown in construction raised reported poor health and mental disorders in the affected population by 3 percentage points, respectively. We argue that the size of this effect responds to the fact that the construction sector was at the center of the economic recession. As a result, workers exposed to the negative labor demand shock faced very low chances of re-entering employment. We show that this led to long unemployment spells, stress, hopelessness, and feelings of uselessness. These effects point towards a potential channel for unemployment hysteresis.