> The private keys can't actually be stored in the HSM though, because you need to be able to back them up.

Every actual HSM I've ever used allows some sort of encrypted export. But actual HSMs are expensive and PKCS#11 is a terrible API so they suck to use.