Getting started with 1Password for your growing team, or refining your setup? Our Secured Success quickstart guide is for you.
Forum Discussion
Solved
Why does the 1P CLI not access the local 1P store?
👋
I have a shell script with lots of "secret references"...
export EXAMPLE1_API_KEY=$(op read "op://<VAULT_NAME>/<ENTRY_NAME>/<SECTION_NAME>/<FIELD_NAME>")
export EXAMPLE2_API_KEY=$(op read "op://<VAULT_NAME>/<ENTRY_NAME>/<SECTION_NAME>/<FIELD_NAME>")
export EXAMPLE3_API_KEY=$(op read "op://<VAULT_NAME>/<ENTRY_NAME>/<SECTION_NAME>/<FIELD_NAME>")Every time I load my terminal shell it takes an age for each secret to load.
I was then told by someone that the 1P CLI doesn't actually communicate with the local 1Password app, but instead it goes remotely to fetch the data (which in the case of my friend, they couldn't load their shell once when 1Password had remote access issues, hence they discovered what was happening).
But for me, I'm wondering, is this why it can be so slow to load a terminal shell when you have lots of secret references? each secret, is a new CLI invocation, and then that has to be individually making remote calls with authorization etc etc.
Is there any way to speed this up?
Is there a reason for not consulting the local 1Password app data? Or does that 1P app not actually store any data and any time you're logging into that you're actually just doing a bunch of remote calls to pull data down?
Thanks!
Thanks theo I didn't realise that `op inject` did a bulk fetch. That actually means I can completely drop my other work-around in favour of doing this...
In a `~/.localrc` I have:
export EXAMPLE_API_KEY={{ op://VaultName/SecretName/SectionName/TokenName }} export ANOTHER_API_KEY={{ op://VaultName/SecretName/SectionName/TokenName }}I can then source via `op inject` from my `~/.zshrc`:
# Configuration you don't want as part of your main .zshrc # For me, this is a template file that includes 1Password secret references. # Meaning, I need to source the file via `op inject` so I can interpolate my secrets. # if [ -f "$HOME/.localrc" ]; then if command -v op >/dev/null; then op signin --account my.1password.com source <(op inject -i $HOME/.localrc) fi fiThis loads much faster now, even when I open a new shell instance I just get a single OS popup asking me to authorize the shell to access 1Password (and I don't need my password as it has that already and so it just needs me to press a button, which I'm fine with -- see screenshot).
10 Replies
Definitely would be nice if the SDKs and CLI had a way to relying on the local app cache. Here is the related issue (https://github.com/1Password/onepassword-sdk-js/issues/111) I opened for the js sdk.
Short of that, at least fetching items in bulk, is a good first step. When we build the dmno 1password plugin (https://dmno.dev/docs/plugins/1password/). Because we want to rely on the CLI for auth during local dev, we used `op inject` (https://developer.1password.com/docs/cli/reference/commands/inject/) which will fetch items in bulk. I imagine you could use this to set many vars at once.
In our plugin we also have some caching logic, will process the items and only fetch those that are needed, and a way to store those items (encrypted). Your script does make sense, and the caching is nice, but one downside is your cache then becomes a potential vulnerability.
Thanks theo I didn't realise that `op inject` did a bulk fetch. That actually means I can completely drop my other work-around in favour of doing this...
In a `~/.localrc` I have:
export EXAMPLE_API_KEY={{ op://VaultName/SecretName/SectionName/TokenName }} export ANOTHER_API_KEY={{ op://VaultName/SecretName/SectionName/TokenName }}I can then source via `op inject` from my `~/.zshrc`:
# Configuration you don't want as part of your main .zshrc # For me, this is a template file that includes 1Password secret references. # Meaning, I need to source the file via `op inject` so I can interpolate my secrets. # if [ -f "$HOME/.localrc" ]; then if command -v op >/dev/null; then op signin --account my.1password.com source <(op inject -i $HOME/.localrc) fi fiThis loads much faster now, even when I open a new shell instance I just get a single OS popup asking me to authorize the shell to access 1Password (and I don't need my password as it has that already and so it just needs me to press a button, which I'm fine with -- see screenshot).
- Scott_1P
1Password Team
Hi Integralist!
Really cool to see someone leaning hard on secret references!
As you note, the CLI is on the slower side. Running `op item get $someItem` gives you a sense of how long that takes. In my experience most item-level operations can take about one second.
Every time I load my terminal shell it takes an age for each secret to load.
Based on this, it sounds like you're using these secret references in a script that is loaded by your shell configuration, hence why it's slowing down any time you open a new shell?
If my inference is correct, I might recommend not loading these as part of your shell configuration (partly because if you are unable to unlock 1Password then you can't instantiate a terminal session....)
Without knowing more about your use-case or whether this is part of your shell config or a script you run, here are a few things to explore that _may_ help depending on your use-case:
- Our SDKs fair bit faster than the CLI, and there are some methods (e.g., client.secret.resolve_all() or .resolve()) that you can use to resolve a secret reference to it's value, optionally in bulk. If you're able to write the script in Go, Python, or JS, this might be faster.
- Our Shell Plugins may be helpful if the secrets you're resolving are credentials for CLI tools you use. Using one of these plugins means you can avoid loading all those secrets when launching your shell, and obtain only the secret you actually need in the moment you need it.
If none of the above are really hitting the mark, maybe you can share more about the specifics of your use case?
Thanks. It sounds like the shell plugin might help. I'll look into it more later and report back. Sounds like I can just create a custom shell "plugin" without having to necessarily contribute it back upstream to the relevant GitHub repo.
- Scott_1P
Sounds like I can just create a custom shell "plugin" without having to necessarily contribute it back upstream to the relevant GitHub repo.
Oh, now that's an interesting idea! Hopefully that makes the experience a bit smoother for you! Report back with how things go!
As far as we've found building integrations with CLI/SDK/App at DMNO, the CLI will only use the local app for auth (like biometrics) and all the data fetching still happens remotely. We've had to work around this as well and would love if the CLI/SDKs could fetch from a local cache as well.
👋 phildmno I've put my workaround below. I would be interested to know what y'all used as a workaround to this.
For anyone interested, this is the approach I'm taking now...
⚠️ Be sure this is what you want to do. Understand the code and do not blindly copy/paste.
https://github.com/Integralist/dotfiles/blob/main/.config/zsh/lazy_op.zsh
So instead of:
export EXAMPLE_API_KEY=$(op read "op://<VAULT_NAME>/<ENTRY_NAME>/<SECTION_NAME>/<FIELD_NAME>")I use:
lazy_op EXAMPLE_API_KEY "op://<VAULT_NAME>/<ENTRY_NAME>/<SECTION_NAME>/<FIELD_NAME>"The secrets are loaded the first time the shell is loaded if they don't exist in a local file that looks like this:
FOO_EXAMPLE_API_KEY=some_secret BAR_EXAMPLE_API_KEY=some_other_secretThe file is stored in my temp directory as I don't care if the file gets cleaned up. This is just to improve the performance generally (as I work 8 hours a day in the terminal and am constantly spinning up new shell instances).
The script (lazy_op.zsh) automatically deletes the temp cache file every hour anyway as a precaution (as I don't want that file to exist forever, and I'm OK with once an hour having to go back to the remote 1P server to get the secrets).
- Scott_1P
integralist Clever! Glad you found something that worked well for you. Thanks for sharing! :)
