This paper proposes reinforcement learning as a foundational stone of a framework for efficient spectrum usage in the context of nextgeneration mobile cellular networks. The objective of the framework is to efficiently use the spectrum in a cellular orthogonal frequency-division multiple access network while unnecessary spectrum is released for secondary spectrum usage within a private commons spectrum accessmodel. Numerical results show that the proposed framework obtains the best performance compared with other approaches for spectrum assignment. Moreover, the framework is relatively simple to implement in terms of computational requirements and signaling overhead.