Integrating and manipulating the nano-optoelectronic properties of Van der Waals heterostructures can enable unprecedented platforms for photodetection and sensing. The main challenge of infrared photodetectors is to funnel the light into a small nanoscale active area and efficiently convert it into an electrical signal. Here, we overcome all of those challenges in one device, by efficient coupling of a plasmonic antenna to hyperbolic phonon-polaritons in hexagonal-BN to highly concentrate mid-infrared light into a graphene pn-junction. We balance the interplay of the absorption, electrical and thermal conductivity of graphene via the device geometry. This approach yields remarkable device performance featuring room temperature high sensitivity (NEP of 82 pW/Hz) and fast rise time of 17 nanoseconds (setup-limited), among others, hence achieving a combination currently not present in the state-of-the-art graphene and commercial mid-infrared detectors. We also develop a multiphysics model that shows very good quantitative agreement with our experimental results and reveals the different contributions to our photoresponse, thus paving the way for further improvement of these types of photodetectors even beyond mid-infrared range.