A single three-level atom driven by a longitudinal mode of a high-Q cavity is used to implement two-qubit quantum phase gates for the intracavity field. The two qubits are associated with the zero- and one-photon Fock states of each of the two opposite circular polarization states of the field. The three-level atom mediates the conditional phase gate provided the two polarization states and the atom interact in a V-type configuration and the two-photon resonance condition is satisfied. Microwave and optical implementations are discussed with gate fidelities being evaluated against several decoherence mechanisms such as atomic velocity fluctuations or the presence of a weak magnetic field. The use of coherent states for both polarization states is investigated to assess the entanglement capability of the proposed quantum gates.