Storing & retrieving sensitive data for your CI/CD pipelines securely

On-premises edition. See below for a great article by Jeff Smith covering the other scenario where your database resides in Oracle Cloud.

Secret management is a crucial part of your IT security strategy. Careless handling of secrets can leak these, and create security risks that are entirely avoidable. Hard-coding secrets, even for lab environments, is strongly discouraged by multiple sources such as OWASP, the Open Worldwide Application Security Project. OWASP also provides a handy cheat-sheet covering many aspects of secret management and handling.

The downside to handling secrets securely is a loss of convenience: typically you spend more time designing how to handle credentials securely, and you also need glue code to access them. In CI/CD pipelines, this becomes especially apparent.

Let’s consider a typical example: your Continuous Integration (CI) pipeline deploys the latest release of your database software to a target database, just prior to running unit tests. To do so, the tool in your CI pipeline must somehow connect to the target database. This is typically done using the holy trinity of username, password, and network connection string. They truly mark the keys to the kingdom: combined, they allow anyone who possesses them to connect to the database.

The big question is: how can you store sensitive data securely?

The question is very open-ended, and the answer depends on many things. A common answer for on-premises deployments revolves around a secret management service aka vault. If you don’t want to bear the burden of hosting one of the most critical services for your business yourself, you can delegate the responsibility to a cloud provider of your choice. In the following example, Oracle Cloud Infrastructure (OCI)’s secret management service will be used, although the concepts remain largely the same for other offerings.

You can read about the on-premises solution in this article. Initially I planned to write another article covering the cloud, but found out that Jeff Smith has already done that. You can find all the details on his blog.

Once you have created a secret, its value can be retrieved via the command line interface (CLI). As with all cloud providers, you need to have an appropriate Identity and Access Management (IAM) policy in place allowing the CI pipeline to retrieve the secrets it needs (and only these!). Narrow it down as much as you can, the need-to-know-basis applies to system accounts on CI job runners, too. For OCI this means:

  • Consider creating a separate compartment for all secrets you use in the CI pipeline, it makes working with your IAM policy easier.
  • Make sure secrets can be read, but never changed. In OCI secrets are known as secret-bundles. They are bundles because they are versioned, with one of them in current state with the others retained unless they expire. This makes secret rotation much easier, and you definitely should rotate your secrets!
  • Limit the number of secrets that can be read to only those that are required in your IAM policy.
  • Since the CI server runs outside OCI, it cannot use an instance principal for authentication and authorisation. Instead, it requires a dedicated OCI IAM user and an API signing key. This IAM user account should be used exclusively by the CI system and granted only the minimum permissions required.
  • The OCI API signing key used by the CI system should be stored securely on the build server with restrictive filesystem permissions, accessible only to the CI service account.

A policy granting access to exactly three secrets could look like this:

Allow group ci-secret-readers
to read secret-bundles
in compartment CICDSecrets
where any {
target.secret.id = 'ocid1.vaultsecret.oc1..secretA',
target.secret.id = 'ocid1.vaultsecret.oc1..secretB',
target.secret.id = 'ocid1.vaultsecret.oc1..secretC'
}

What does that look like in the real world? Here’s an example how to use shell scripting to retrieve a secret value. You could easily rewrite this in Python, Typescript, or whichever language you prefer. OCI SDKs typically allow you to use your language of choice; bash is very useful since it’s a common format for CI job runners. The OCI CLI must already be configured with credentials for the dedicated CI service account.

#!/usr/bin/env bash
####################################################################################
#
# get_secret_from_vault()
#
# get_secret_from_vault takes an OCID representing the OCI secret to be
# retrieved. The secret must exist or else an error is thrown.
# Once the secret has been retrieved it is converted from base64 to a
# usable format
#
# input:
# - secret_id: the secret's OCID
#
# output:
# - the base64 decoded clear text of the secret's value
#
# required:
# - either base64 or openssl must be available in the PATH, or else the base64
# encoded secret can't be converted.
#
# usage:
# source ./oracle-cli/bin/activate
# source ./utils.sh
# DB_PASSWORD=$(get_secret_from_vault ocid1.vaultsecret.oc1....)
#
get_secret_from_vault() {
local secret_id="${1}"
if [[ -z "${secret_id}" ]]; then
echo "Usage: get_secret_from_vault <secret_ocid>" >&2
return 1
fi
if ! command -v base64 >/dev/null 2>&1 && ! command -v openssl >/dev/null 2>&1; then
echo "get_secret_from_vault requires either base64 or openssl in your path" >&2
return 1
fi
local encoded_secret
encoded_secret="$(
oci secrets secret-bundle get \
--secret-id "${secret_id}" \
--stage CURRENT \
--raw-output \
--query 'data."secret-bundle-content".content'
)" || return $?
if base64 --help 2>&1 | grep -q -- '--decode'; then
printf '%s' "${encoded_secret}" | base64 --decode
elif base64 -D </dev/null >/dev/null 2>&1; then
printf '%s' "${encoded_secret}" | base64 -D
else
printf '%s' "${encoded_secret}" | openssl base64 -d -A
fi
}

You source the script (let’s call it utils.sh) into your session, and call it to retrieve the password and connection string

DB_USERNAME=CI_USER
DB_PASSWORD=$(get_secret_from_vault ocid...)
DB_CONNECT_STRING=$(get_secret_from_vault ocid...)
sql /nolog <<EOF
connect ${DB_USERNAME}/${DB_PASSWORD}@${DB_CONNECT_STRING}
select user;
exit
EOF

If the IAM policy and secret values are configured correctly, the connection should succeed.. Just be careful and avoid printing secret values, enabling shell tracing (set -x), or logging environment variables after secrets have been retrieved. Because the pipeline always retrieves the current version of a secret, rotating credentials typically requires no changes to the pipeline itself. This is one of the major advantages of using a vault.

Further strengthening your security posture

Depending on your environment, you may be able to eliminate database passwords altogether by using SQLcl named connections or Oracle Secure External Password Store (password wallets), optionally combined with TLS or mutual TLS authentication.

It gets even easier if your database lives in the cloud

And for those of you enjoying the ease of use of cloud databases in OCI, there’s an even faster, and more secure way to connect to your cloud system: using Database Tools in OCI all you need is an OCID to connect to your database.

$ sql /nolog
SQLcl: Release 26.2 Production on Tue Jul 21 16:30:47 2026
Copyright (c) 1982, 2026, Oracle. All rights reserved.
SQL> conn ocid1.databasetoolsconnection.oc1....
Connected.
SQL> show connection
COMMAND_PROPERTIES:
type: OCI
CONNECTION:
MARTIN@jdbc:oracle:thin:@(description= (retry_count=20)(retry_delay=3)(address=(protocol=tcps)...
CONNECTION_IDENTIFIER:
nnnnnnnnnnn_BLOGPOST_low
CONNECTION_DB_VERSION:
Oracle AI Database 26ai Enterprise Edition Release 23.26.3.1.0 - Production
Version 23.26.3.1.0
NOLOG:
false
PRELIMAUTH:
false

Instead of an OCID you can also use a human readable & understandable name. Jeff Smith explains all the details in his recent blog article How to connect to your OCI Oracle Database via SQLcl.

That’s it! Happy scripting.