Human cytomegalovirus (HCMV) is an opportunistic pathogen causing a variety of severe viral infections, including irreversible congenital disabilities. Nowadays, HCMV infection is treated by inhibiting the viral DNA polymerase. However, DNA polymerase inhibitors have several drawbacks. An alternative strategy is to use compounds against the packaging machinery or terminase complex, which is essential for viral replication. Our discovery that raltegravir (1), a human immunodeficiency virus drug, inhibits the nuclease function of UL89, one of the protein subunits of the complex, prompted us to further develop terminase inhibitors. On the basis of the structure of 1, a library of diketoacid (α,γ-DKA and β,δ-DKA) derivatives were synthesized and tested for UL89-C nuclease activity. The mode of action of α,γ-DKA derivatives on the UL89 active site was elucidated by using X-ray crystallography, molecular docking, and in vitro experiments. Our studies identified α,γ-DKA derivative 14 able to inhibit UL89 in vitro in the low micromolar range, making 14 an optimal candidate for further development and virus-infected cell assay.