We need a way to disable password prompts for a period of time

I've got all of my terraform and deployment/CI/CD secrets configured in 1pass, to help secure against supply chain attacks, and to help gate LLMs from making changes to infrastructure without it requiring a physical prompt. However, quite often I want to let the LLM run for an hour or so without it constantly re-prompting to approve credential access. It would be great if there was a way to leave the authentication window open for a period of time, so that Windows Hello doesn't re-prompt every time op:// credentials are accessed within that time period we configure. Something like a "Leave unlocked for 2 hours" or something.

I know there's a "Unlock with device for: X hours" setting, but this does not seem to do anything for me. I've got it set to 2 hours, but every op:// access of a credential still causes a reprompt. It does this whether I have "Unlock using Windows Hello" enabled OR disabled.