Forum Discussion

Former Member's avatar
Former Member
4 years ago

export specific secret value as an environment variable

Hello 1Password Community :)

I am trying to fetch and export a specific secret value as an environment variable:

what I tried: - Note that I replaced the actual gcp location with [gcp location]

export AIRFLOW_SERVICE_ACCOUNT=$(op item get --cache --format=json [gcp location] | jq -r '.fields[]|select(.id="secret")|.value')

Then I fetch the environment variable as follows:

airflow_service_account = os.environ["AIRFLOW_SERVICE_ACCOUNT"]
service_account_json = "service-account.json"
with open(service_account_json, "w") as f:
f.write(airflow_service_account)

Error:

raise JSONDecodeError("Extra data", s, end)
json.decoder.JSONDecodeError: Extra data: line 2 column 1 (char 5)

when I run the following in the terminal:
op item get --cache --format=json [gcp location] | jq -r '.fields[]|select(.id="secret")|.value'

I get the following result (note that I replaced private data with the word bla


null
{
"type": "service_account",
"project_id": "bla",
"private_key_id": "bla",
"private_key": "-----BEGIN PRIVATE KEY-----\nbla\n-----END PRIVATE KEY-----\n",
"client_email": "bla",
"client_id": "bla",
"auth_uri": "bla",
"token_uri": "bla",
"auth_provider_x509_cert_url": "bla",
"client_x509_cert_url": "bla"
}

I hope I was clear enough in my description.
Does anyone know what modifications I need to do to properly fetch this concerned service-account dict?

Thank you for your time :)


1Password Version: 7.9.4
Extension Version: Not Provided
OS Version: macOS 12.3.1

7 Replies

  • Former Member's avatar
    Former Member

    Glad you got it working!

    Please do let us know if you encounter any other hurdles, we're here to help :)

    Best,
    Horia

  • Former Member's avatar
    Former Member

    ok now I got to receive the isolated value

    old command that was almost getting me the value of the secret:

    export AIRFLOW_SERVICE_ACCOUNT=$(op item get --cache --format=json [gcp location] | jq -r '.fields[]|select(.id="secret")|.value')

    new command:

    export AIRFLOW_SERVICE_ACCOUNT=$(op item get --cache --format=json [gcp location] | jq -r '.fields[]|select(.id=="secret")|.value')

    only difference is the double "=" sign in the "select" field.

  • Former Member's avatar
    Former Member

    Hmm let me make sure I understand the issue correctly:

    When you run op item get --cache --format=json [gcp location], the [gcp location] part is, in fact, the name of your item, is this correct?

    The issue here may be that this name contains the "/" character, which is not valid in secret references.

    So, to reiterate, in order to create a valid secret reference, you would need:
    1. the vault name or ID (Engineering, from the json output I see you provided above)
    2. the item name or ID - if your item name contains unsupported characters, you can use the item ID, that you replaced above with "bla"
    3. the field name or ID - secret, in your case

    So I think a valid secret reference should be op://Engineering/<your-item-id>/secret.
    Can you please let us know what does op read, given the above as an argument, return?

    Thank you!

    Best,
    Horia

  • Former Member's avatar
    Former Member

    Hello @"Horia.Culea_1P" ,
    I do not think that the secret in question is hosted within a vault folder, because when I run
    op read op://gcp/bla/iam/jw-airflow-serviceaccount, I get the following error:

    could not read secret op://gcp/bla/iam/xm-airflow-serviceaccount: could not get item gcp/bla: "gcp" isn't a vault in this account. Specify the vault with its ID or name.

    and when I run op read op://gcp/bla/iam/xm-airflow-serviceaccount/secret

    (the only thing that is present in the concerned OP folder is secret), I get the following:

    too many '/': secret references should match op://<vault>/<item>[/<section>]/<field>

    I am checking this with the devops team as we speak.

    Thank you for your time.
    Respectfully,

    Maher.

  • Former Member's avatar
    Former Member

    Hey @Nadros, thank you for reaching out to us!
    Quick question, before diving deeper into troubleshooting this: Have you tried the op read command? By glancing through the thread, I think this may actually be a simpler way to achieve the same result. The syntax would be: op read op://<vault-name>/<item-name>/<field-name>.
    More documentation can be found at https://developer.1password.com/docs/cli/reference/commands/read/
    Let us know if this helps you!
    Looking forward to hearing from you.

    Best,
    Horia

  • Former Member's avatar
    Former Member

    Hello @"Justin.Yoon_1P" .

    Thank you for getting back to me.

    Small context about the secret: is it used to access an account on https://airflow.apache.org/, which is kind of a task orchestrator.

    The output of the echo $AIRFLOW_SERVICE_ACCOUNT command is the following:


    null
    {
    "type": "service_account",
    "project_id": "bla",
    "private_key_id": "bla",
    "private_key": "-----BEGIN PRIVATE KEY-----
    bla
    -----END PRIVATE KEY-----
    ",
    "client_email": "bla",
    "client_id": "bla",
    "auth_uri": "bla",
    "token_uri": "bla",
    "auth_provider_x509_cert_url": "bla",
    "client_x509_cert_url": "bla"
    }

    which I believe is the same as calling the op command as per my 1st message.

    when I run the following command op item get --cache --format=json [gcp location] this is the entire dict:


    {
    "id": "bla",
    "title": "bla",
    "version": 1,
    "vault": {
    "id": "bla",
    "name": "Engineering"
    },
    "category": "SECURE_NOTE",
    "last_edited_by": "bla",
    "created_at": "bla",
    "updated_at": "bla",
    "fields": [
    {
    "id": "notesPlain",
    "type": "STRING",
    "purpose": "NOTES",
    "label": "notesPlain"
    },
    {
    "id": "secret",
    "type": "CONCEALED",
    "label": "secret",
    "value": "{\n\"type\": \"service_account\",\n \"project_id\": \"bla\",\n \"private_key_id\": \"bla\",\n \"private_key\": \"-----BEGIN PRIVATE KEY-----\\nbla\\n-----END PRIVATE KEY-----\\n\",\n \"client_email\": \"bla\",\n \"client_id\": \"bla\",\n \"auth_uri\": \"bla\",\n \"token_uri\": \"bla\",\n \"auth_provider_x509_cert_url\": \"bla\",\n \"client_x509_cert_url\": \"bla\"\n}"
    }
    ]
    }

    and the part that I am trying to isolate is the entire value inside the key 'value' near the end.

  • Former Member's avatar
    Former Member

    Hey @Nadros ,

    It looks like you are saving a JSON object directly as a value in 1Password. At first glance, it seems like your code is having trouble parsing the JSON. Could you print what echo $AIRFLOW_SERVICE_ACCOUNT returns for you after you set that var (with the info redacted with "bla" of course)? I would like to see if there are any strings that we add for formatting that may break in your code.

    Additionally, I do think there may be an alternative - to set each key:value in the JSON as its own field in the item.

    If you take this approach, each of the values can be accessed by our https://developer.1password.com/docs/cli/secrets-environment-variables syntax, which in conjunction with our op run command to start your app, can load distinct env vars for use during runtime.