Git and Azure DevOPS: agent refused operation

I struggle getting my GIT repositories in Azure working with 1Password keeping my SSH keys. Beside that Azure does not support Ed25519 keys (sigh), so I had to create a RSA key for it, authentication with it fails as soon as I activate the two lines for "Host *" and "IdentityAgent" pointing to the 1Password socket:
--- 8< ---
sign_and_send_pubkey: signing failed for RSA "/Users/myuser/.ssh/id_rsa_azure.pub" from agent: agent refused operation
--- 8< ---
(it also fails without setting IdentityFile and IdentitiesOnly btw.)

Exporting the key and using old fashioned file based key authentication works though.

Working with other GIT servers works, so I think this is a "Azure-Thing".

---

1Password Version: 8.7.0 (80700012)
Extension Version: Not Provided
OS Version: 12.2.1 (21D62)