Knowledge of the structure of interfacial water molecules at electrified solid materials is the first step toward a better understanding of important processes at such surfaces, in, e.g., electrochemistry, atmospheric chemistry, and membrane biophysics. As graphene is an interesting material with multiple potential applications such as in transistors or sensors, we specifically investigate the graphene-water interface. We use sum-frequency generation spectroscopy to investigate the pH- and potential-dependence of the interfacial water structure in contact with a chemical vapor deposited (CVD) grown graphene surface. Our results show that the SFG signal from the interfacial water molecules at the graphene layer is dominated by the underlying substrate and that there are water molecules between the graphene and the (hydrophilic) supporting substrate.