Posted inSecurity

Deploy HashiCorp Vault on Linux: Centralized Secrets and Automated Credential Rotation

Security tutorial - IT technology blog
Security tutorial - IT technology blog

The Problem: The Danger of Secret Sprawl

Early in my sysadmin career, configuration management seemed simple. I dropped a .env file into a project directory, pasted in the database passwords and Stripe API keys, and moved on. That workflow broke down fast. As our infrastructure expanded to 18 virtual machines across staging and production, credentials leaked everywhere—into bash history files, Docker build caches, Git commit logs, and private Slack channels.

Everything changed after our staging cluster suffered an automated credential-stuffing attack. That incident proved that firewall rules and SSH keys only guard the perimeter. When an attacker grabs shell access to a single web app container, plaintext configuration files expose your entire backend. Operational maintenance was just as painful. Rotating one compromised PostgreSQL password meant updating 24 separate config files, restarting six microservices, and scheduling maintenance windows.

Solving secret sprawl requires three core capabilities: centralizing storage behind strong encryption, enforcing strict access controls, and generating dynamic credentials that expire automatically.

Core Architecture: How Vault Operates

HashiCorp Vault manages secrets and protects sensitive data through identity-based access. Here is the operational model you will interact with on a Linux host:

  • Storage Backend: The encrypted persistence layer. Vault supports the local filesystem, Consul, integrated Raft storage, and databases. Vault encrypts all payloads in memory before writing a single byte to disk.
  • Unsealing and Shamir’s Secret Sharing: Vault starts in a sealed state. It cannot decrypt stored secrets until administrators unseal it using a split master key (typically 3 of 5 key shards) or an automated Cloud KMS key.
  • Secrets Engines: Modular plugins for different data types. The kv-v2 engine manages versioned static secrets, while dynamic engines generate temporary credentials for PostgreSQL, MySQL, or AWS on demand.
  • Policies: Path-based access rules written in HCL (HashiCorp Configuration Language). They grant explicit permissions like read, create, or update.
  • Leases and TTLs: Every dynamic secret and access token has a Time-to-Live. Once a lease expires, Vault automatically drops the credential from the upstream database or service.

Hands-on Setup: Deploying Vault on Ubuntu Linux

Let us walk through installing Vault on Ubuntu 24.04/22.04 LTS, setting up a systemd daemon, initializing storage, and issuing dynamic PostgreSQL credentials.

Step 1: Install Vault from the HashiCorp Repository

Update your system package index, install required dependencies, and register HashiCorp’s official GPG signing key:

# Install dependencies
sudo apt-get update -y
sudo apt-get install -y gpg coreutils curl jq

# Import the HashiCorp GPG key
wget -O- https://apt.releases.hashicorp.com/gpg | sudo gpg --dearmor -o /usr/share/keyrings/hashicorp-archive-keyring.gpg

# Add the official repository
echo "deb [signed-by=/usr/share/keyrings/hashicorp-archive-keyring.gpg] https://apt.releases.hashicorp.com $(lsb_release -cs) main" | sudo tee /etc/apt/sources.list.d/hashicorp.list

# Install the Vault binary
sudo apt-get update -y
sudo apt-get install -y vault

Step 2: Configure the Vault Daemon

Vault reads its primary configuration from /etc/vault.d/vault.hcl. We will create a dedicated data directory, assign ownership to the vault system user, and configure the local listener.

# Prepare data directory with restricted permissions
sudo mkdir -p /opt/vault/data
sudo chown -R vault:vault /opt/vault
sudo chmod 750 /opt/vault/data

Write the daemon settings to /etc/vault.d/vault.hcl:

sudo tee /etc/vault.d/vault.hcl << 'EOF'
storage "file" {
  path = "/opt/vault/data"
}

listener "tcp" {
  address       = "127.0.0.1:8200"
  tls_disable   = 1
}

api_addr = "http://127.0.0.1:8200"
cluster_addr = "http://127.0.0.1:8201"
ui = true
disable_mlock = true
EOF

Start the systemd unit and verify that the daemon is active:

sudo systemctl enable vault
sudo systemctl start vault
sudo systemctl status vault --no-pager

Step 3: Initialize and Unseal the Vault Instance

Set the client environment variable so the CLI targets your local listener:

export VAULT_ADDR='http://127.0.0.1:8200'
echo "export VAULT_ADDR='http://127.0.0.1:8200'" >> ~/.bashrc

Initialize the Vault storage backend. This one-time command splits the master key into 5 shards and sets a 3-shard reconstruction threshold:

vault operator init -key-shares=5 -key-threshold=3 > ~/vault-init-keys.txt
chmod 600 ~/vault-init-keys.txt

Open ~/vault-init-keys.txt. Back up these 5 unseal keys and the root token to an encrypted password vault immediately.

Provide 3 distinct unseal keys to bring Vault into an operational state:

vault operator unseal <UNSEAL_KEY_1>
vault operator unseal <UNSEAL_KEY_2>
vault operator unseal <UNSEAL_KEY_3>

Check the seal status. Look for Sealed: false in the CLI output:

vault status

Log in using the root token to start configuring engines:

vault login <INITIAL_ROOT_TOKEN>

Step 4: Store and Retrieve Static Secrets (KV-v2)

Mount the version 2 Key-Value secrets engine at the secret/ path:

vault secrets enable -path=secret kv-v2

Store production configurations for a payment microservice:

vault kv put secret/myapp/config \
    api_key="sk_live_9948571029384756" \
    payment_gateway_url="https://api.payment.internal" \
    max_retries="5"

Validating complex payloads or policy files before writing them to the API prevents runtime parsing errors. When formatting raw JSON outputs, the JSON Formatter & Validator on ToolCraft comes in handy. It executes entirely in your browser without sending tokens or configs to an external server.

Retrieve the stored data using the CLI and filter the fields with jq:

vault kv get -format=json secret/myapp/config | jq '.data.data'

Step 5: Automate Dynamic Database Credential Rotation

Static secrets still require manual invalidation if leaked. Dynamic secrets solve this issue by creating unique, temporary database roles on demand.

Enable the database secrets engine:

vault secrets enable database

Connect Vault to your PostgreSQL instance running on 127.0.0.1:5432:

vault write database/config/my-postgres-db \
    plugin_name=postgresql-database-plugin \
    allowed_roles="app-readwrite" \
    connection_url="postgresql://{{username}}:{{password}}@127.0.0.1:5432/appdb?sslmode=disable" \
    username="vault_admin" \
    password="VaultAdminSecurePassword123!"

Define an application role with a 1-hour default TTL and a 24-hour maximum lease limit:

vault write database/roles/app-readwrite \
    db_name=my-postgres-db \
    creation_statements="CREATE ROLE \"{{name}}\" WITH LOGIN PASSWORD '{{password}}' VALID UNTIL '{{expiration}}'; \
        GRANT SELECT, INSERT, UPDATE, DELETE ON ALL TABLES IN SCHEMA public TO \"{{name}}\";" \
    default_ttl="1h" \
    max_ttl="24h"

Generate a fresh credential pair by querying the role endpoint:

vault read database/creds/app-readwrite

Vault provisions a dedicated user (such as v-token-app-read-0a1b2c3d), assigns a random password, and registers a 3600-second lease. When that lease expires, Vault drops the database role automatically.

Step 6: Integrate with a Python Application

Applications can request and manage their own short-lived credentials at boot time using the official hvac Python package.

import hvac
import psycopg2

# Initialize the Vault client
client = hvac.Client(url='http://127.0.0.1:8200', token='s.yourAppAuthTokenHere')

if not client.is_authenticated():
    raise PermissionError("Failed to authenticate with the Vault daemon.")

# Request short-lived PostgreSQL credentials (default 1-hour lease)
creds = client.secrets.database.generate_credentials(name='app-readwrite')
db_user = creds['data']['username']
db_pass = creds['data']['password']
lease_id = creds['lease_id']

print(f"[+] Generated dynamic user: {db_user}")
print(f"[+] Active lease ID: {lease_id} ({creds['lease_duration']}s)")

# Connect to PostgreSQL using temporary credentials
conn = psycopg2.connect(
    host="127.0.0.1",
    port=5432,
    dbname="appdb",
    user=db_user,
    password=db_pass
)

with conn.cursor() as cur:
    cur.execute("SELECT current_user, session_user, now();")
    user_info = cur.fetchone()
    print(f"[+] Active session: {user_info}")

conn.close()

# Renewal pattern: background workers can extend the lease before TTL expiry
# client.sys.renew_lease(lease_id=lease_id, increment=3600)

Next Steps

Migrating from plaintext .env files to HashiCorp Vault replaces long-lived credentials with audited, short-lived tokens. Static passwords disappear from source code and server disks, while dynamic credentials make credential rotation automatic.

Start by migrating database connections and third-party API keys to Vault. Once your services handle token renewal cleanly, integrate Vault with your CI/CD pipelines, Kubernetes clusters, and cloud identity providers.

Share:
Tags: