The importance of binder performance on the mixture response to accelerated fatigue loading and thermal cracking was investigated. Binder’s fatigue performance was measured by means of the Linear Amplitude Sweep test (LAS) while fatigue properties of the mixtures were investigated by performing the EBADE test. The low temperature properties of the binders were investigated by measuring the glass transition temperature and fracture properties with the Single Edge Notched Beam (SENB) test. The mixtures fracture properties were investigated using the FENIX test. The experimental matrix for this study included unmodified and rubber-modified binders and limestone aggregates. A good correlation between binder and mixture fracture energy was observed at low temperatures. This good correlation indicates the importance of fracture response of the binder to the overall low temperature cracking performance of the mixture. Experimental results suggest that a significant part of the variation of the fracture energy of the mixture can be explained by the binder fracture properties. Good correlations were also obtained for the displacement at maximum load in the SENB and Fenix tests. Similar accelerated fatigue response for binders and mixtures were observed when the stresses and strains were normalized. Significant reduction in the stress happened at about the same normalized strain in the binder and mixture. It can be seen that the mixture has remaining strength after reaching peak stress probably due to the aggregate structure.