Boron (B) doping of silicon nanocrystals requires the incorporation of a B-atom on a lattice site of the quantum dot and its ionization at room temperature. In case of successful B-doping the majority carriers (holes) should quench the photoluminescence of Si nanocrystals via non-radiative Auger recombination. In addition, the holes should allow for a non-transient electrical current. However, on the bottom end of the nanoscale, both substitutional incorporation and ionization are subject to significant increase in their respective energies due to confinement and size effects. Nevertheless, successful B-doping of Si nanocrystals was reported for certain structural conditions. Here, we investigate B-doping for small, well-dispersed Si nanocrystals with low and moderate B-concentrations. While small amounts of B-atoms are incorporated into these nanocrystals, they hardly affect their optical or electrical properties. If the B-concentration exceeds ~1 at%, the luminescence quantum yield is significantly quenched, whereas electrical measurements do not reveal free carriers. This observation suggests a photoluminescence quenching mechanism based on B-induced defect states. By means of density functional theory calculations, we prove that B creates multiple states in the bandgap of Si and SiO2. We conclude that non-percolated ultra-small Si nanocrystals cannot be efficiently B-doped.