How can I implement continual (incremental) learning in a face-recognition model without retraining from scratch?

I am building a face-recognition system using Python and a deep-learning model (currently experimenting with FaceNet / ArcFace + PyTorch). The system works for recognizing identities that the model was trained on, but I want to add new users later without retraining the entire model from scratch.

My goal is to include some form of continual learning so the model can learn new faces over time, ideally without catastrophic forgetting.