Three exceptional modular invariants of SU(4) exist at levels 4, 6 and 8. They can be obtained from appropriate conformal embeddings and the corresponding graphs have self-fusion. From these embeddings, or from their associated modular invariants, we determine the algebras of quantum symmetries, obtain their generators,and, as a by-product, recover the known graphs E4, E6 and E8 describing exceptional quantum subgroups of type SU(4). We also obtain characteristic numbers (quantum cardinalities, dimensions) for each of them and for their associated quantum groupoïds.