This paper proposes a Self-organized Spectrum Assignment strategy in the context of next generation multicell Orthogonal Frequency Division Multiple Access networks. The proposed strategy is able to dynamically find spectrum assignments per cell depending on the spatial distribution of the users over the scenario, opening new spectrum access opportunities for secondary spectrum usage. Reinforcement Learning methodology has been employed to implement the strategy, which compared with other fixed and dynamic spectrum assignment strategies shows the best tradeoff between spectral efficiency and Qualityof- Service while releases spectrum in large geographical areas.