Set up Hashicorp Vault

Introduction
I came across this hashcorp vault which is a vault I need for storing network equipment credentials. There is an enterprise version and free version, the free version can be downloaded here.

Hashicorp Vault has API for accessing the data stored in the vault, after the hashicorp vault is initialized 5 keys and 1 root token are generated. In order to unseal the vault, 3 keys are required, then the root token is used to login. If these keys and token are lost the vault will be sealed forever.

In this example I am using CentOS 7, hence I downloaded the binary for linux. By the way the vault is open source and is written in Golang.

[cyruslab@localhost Downloads]$ ls -lah vault_1.2.3_linux_amd64.zip 
-rw-rw-r--. 1 cyruslab cyruslab 44M Oct 16 10:13 vault_1.2.3_linux_amd64.zip

Steps to install

  1. Make the directory to store the vault, mkdir -p /opt/vault/{bin,data,log}
  2. Unpack the binary, unzip vault_1.2.3_linux_amd64.zip
  3. Move the vault to the /opt/vault/bin, mv ~/Downloads/vault /opt/vault/bin
  4. Make a director under etc for storing the config.hcl, mkdir -p /etc/vault, then create a config.hcl file, touch /etc/vault/config.hcl.
  5. Add these settings to the config.hcl,
    disable_cache = true
    disable_mlock = true
    ui = true
    listener "tcp" {
       address          = "0.0.0.0:8200"
       #tls_disable      = 1
       tls_cert_file = "/etc/cert/vault/vault.crt"
       tls_key_file = "/etc/cert/vault/vault.key"
    }
    storage "file" {
       path  = "/opt/vault/data"
     }
    api_addr         = "http://0.0.0.0:8200"
    max_lease_ttl         = "10h"
    default_lease_ttl    = "10h"
    cluster_name         = "vault"
    raw_storage_endpoint     = true
    disable_sealwrap     = true
    disable_printable_check = true
    

    I am going to start the vault with tls, I will be generating the key later.

  6. Generate certificate openssl req -x509 -nodes -days 731 -newkey rsa:4096 -sha256 -out /etc/cert/vault/vault.crt -keyout /etc/cert/vault/vault.key this will create certificate and a key that does not need passphrase.
  7. Create the vault.service to start the service, you can find this configuration from google, but this is the one I used:
    [Unit]
    Description="HashiCorp Vault - A tool for managing secrets"
    Documentation=https://www.vaultproject.io/docs/
    Requires=network-online.target
    After=network-online.target
    ConditionFileNotEmpty=/etc/vault/config.hcl
    
    [Service]
    User=vault
    Group=vault
    ProtectSystem=full
    ProtectHome=read-only
    PrivateTmp=yes
    PrivateDevices=yes
    SecureBits=keep-caps
    AmbientCapabilities=CAP_IPC_LOCK
    NoNewPrivileges=yes
    ExecStart=//opt/vault/bin/vault server -config=/etc/vault/config.hcl
    ExecReload=/bin/kill --signal HUP 
    KillMode=process
    KillSignal=SIGINT
    Restart=on-failure
    RestartSec=5
    TimeoutStopSec=30
    StartLimitBurst=3
    LimitNOFILE=65536
    
    [Install]
    WantedBy=multi-user.target
    

    Create this vault.service in /etc/systemd/system/.

  8. Then systemctl enable vault.service this will start the service whenever the server starts, start the service now by systemctl start vault
  9. Add user vault, and change owner of /opt/vault, /etc/vault, /etc/cert/vault to vault:vault, useradd -r vault, chown -R vault:vault /opt/vault and so on.

Set environment variable
export PATH=$PATH:/opt/vault/bin this is so that vault can be run without using absolute path, you can also write this export to ~/.bashrc so that the variable exists together with your user account and is permanent.
My lab server bashrc file looks like this:

export PATH=/usr/lib64/qt-3.3/bin:/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin:/bin:/sbin:/home/cyruslab/.local/bin:/home/cyruslab/bin:/opt/vault/bin

complete -C /opt/vault/bin/vault vault
export VAULT_ADDR='https://127.0.0.1:8200'

Initialize the vault
The initialization will generate 5 keys and 1 token and store these into /opt/vault/data, you will notice there are two sub directories which are /opt/vault/data/core and /opt/vault/data/sys.

The command line to initialize vault is vault operator init, it will generate something similar like this:

Unseal Key 1: IEVavcw7mn8IhrkGGzo8Y5i0b0WXTkkMTsvRhjOkfw4+
Unseal Key 2: BheaaU05hOlggQ/tOXtzPYQJPpDjF7y1Ct6dSvbk1kbL
Unseal Key 3: UmZGIH1FVJ4HEWYpeGVAW9nfU8e5CBODbNkqorp9j/Li
Unseal Key 4: RMWLrJ97+BrvCxAg0Ai6mQTTcT039wWbxmWGamVDVUNR
Unseal Key 5: Cu6/tTzNBKXT1E3DimigYslSBgFE+VrAWlWOgF6R/YAN

Initial Root Token: s.eVc51pONg2lrOvQ1MwVHlFSC

Vault initialized with 5 key shares and a key threshold of 3. Please securely
distribute the key shares printed above. When the Vault is re-sealed,
restarted, or stopped, you must supply at least 3 of these keys to unseal it
before it can start servicing requests.

Vault does not store the generated master key. Without at least 3 key to
reconstruct the master key, Vault will remain permanently sealed!

It is possible to generate new unseal keys, provided you have a quorum of
existing unseal keys shares. See "vault operator rekey" for more information.

This output is one time as each vault init can only be generated once, so store the keys and token somewhere. I wrote a simple python to do just this, which includes keys and token storage to an encrypted file.

init_vault.py
This script generates a symmetric key from fernet, then check if the vault is initialize or not if not initialize the vault and store the dictionary to the variable, then use the symmetric key to encrypt the variable and store into a file.

from hvac import Client
from os.path import exists
from cryptography.fernet import Fernet
import json
'''
hvac is the python module for Hashicorp vault api
Because after initialization the vault returns a dictionary
and dictionary does not have encode method hence i json is used
to convert the dictionary into a string by using the dumps method.
The string then needs to be encoded into utf-8 as bytes,
then encrypt the data in bytes.
'''

# Client class parameter verify=False because the ssl cert cannot
# validate the ip address, there is no hostname vault.cyruslab.local yet.
client = Client(url='https://127.0.0.1:8200', verify=False)

# if the vault has not been initialized, initialize now.
if not client.is_initialized():
    shares = 5 # generate 5 key shares
    threshold = 3 # require 3 keys to unseal the vault.
    result = client.sys.initialize(shares, threshold)

# if symmetric key is not available generate one.
# save the key as a file for future use.
if not exists('enc.key'):
    key = Fernet.generate_key()
    with open('enc.key', 'wb') as file:
        file.write(key)

# use the symmetric key
with open('enc.key', 'rb') as file:
    sym_key = file.read()

fernet = Fernet(sym_key)
# encrypt the result delivered by initialization.
cipher_result = fernet.encrypt(json.dumps(result).encode('utf-8'))
# save the result into a file.
with open('result.enc', 'wb') as file:
    file.write(cipher_result)

get_vault_result.py
this script is a test code to open the key and token file, decrypt it and use the keys in the file to unseal the vault, the token is for login to vault.

from cryptography.fernet import Fernet
from pprint import pprint

with open('result.enc', 'rb') as file:
    cipher_result = file.read()

with open('enc.key', 'rb') as file:
    key = file.read()

fernet = Fernet(key)
plaintext_result = fernet.decrypt(cipher_result).decode('utf-8')
pprint(plaintext_result)

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s