Problems using an OpenPGP smartcard for SSH with gpg-agent

I have been using an OpenPGP smartcard for encryption, signing and authentication for over a year now and I’ve found it to be really useful as a root of trust. I have all my systems locked down to only allow public key authentication as a 2 factor security mechanism. While the Free Software Foundation Europe have a good guide about setting up a OpenPGP smartcard using subkeys and offline backups  its unfortunately still not very straight forward to get the card set up.

Recently the EEPROM on my first card died and I had to replace the card. However after setting up the new card with the respective subkeys I  consistently encountered an error from gpg-agent where it was looking for the previous card to be inserted during SSH authentication:

Please insert the card with serial number xxxxxxxxxxxxxxxxx

Please remove the current card and insert the one with serial number xxxxxxxxxxxxxxxxx

Via the magicsauce of strace I eventually determined that gpg-agent was attempting to load the key and card data from a file in ~/.gnupg/private-keys-v1.d/ which referenced the original smartcard. Resolving this issues was as simple as removing the key file in that directory, logging out and logging back into the user account, and finally running the following commands with the card inserted to reload the desired key into the agent.

gpg --card-status
gpgkey2ssh [KEYID]
ssh-add -l

At this stage  ssh-add -l should list your correct card serial number and you will ``again be able to authenticate over SSH with the card.