Encryption as a Service using Vault with Spring Boot

Database columns can be encrypted multiple ways. Most of the databases have built-in support to encrypt the values. For example, in Postgres we can use the function pgp_sym_encrypt and pgp_sym_decrypt. It has some disadvantages like every read/write operation will have some operation overhead and slow down the DB servers. Most of the database providers give an option to encrypt the values. Moreover, keys used for the encryption should be properly managed. And it is complicated to do within the realms of the database servers. In a distributed system, the computing costs should be kept minimal and databases have a very high i/o. And cryptographic functions use a big chunk of resources and it is a well-known fact. Most of the industries have regulatory requirements and protect sensitive data in an effective way. Finally, a common concern for engineers and security teams alike is to protect the data in transit and avoid eavesdropping.

Encryption as a Service (EaaS) solves this problem and Hashicorp's Vault has a transit engine which takes out the burden of encrypting the data in transit. Vault is already a default key management and secret management solution in most of the organizations and has been integration with popular cloud providers. We will be seeing in this how we can leverage Vault to encrypt/decrypt the data in transit, effective key rotation, and re-encrypt the data when there is a key rotation happens.


Vault Server

Vault can be downloaded as a binary and you may have to download it first if you haven't done already. It can be started using the below command and we will use configuration furnished below. Since this is a demo application, TLS is disabled and it should be enabled normally. Secondly, Vault is coming with a UI admin tool starting from version 0.10 and we have to set it to true to enable it.

// Start the vault dev server with the above configuration
$ vault server -dev

Once the Vault is downloaded and started. We can visit the URL http://localhost:8200/ui to launch the web administration portal. Previously we have to initialize the vault using the command line now we can do it from the UI. Copy the unseal and Root token which you need at the later point. Vault contains many types of engine and we will be using the transit engine for encrypting the data in transit. In order to do that execute below script which creates the policies, enables the transit engine and mounts it on the path /transit/keys/customer

#!/bin/bash

export VAULT_TOKEN=s.1wjnHEGOpmLFVaSC6WazGDqA
export VAULT_ADDR='http://127.0.0.1:8200'

echo 'path "secret/spring-vault-demo" {
capabilities = ["create", "read", "update", "delete", "list"]
}
path "secret/application" {
capabilities = ["create", "read", "update", "delete", "list"]
}
path "transit/decrypt/customer" {
capabilities = ["update"]
}
path "transit/encrypt/customer" {
capabilities = ["update"]
}
path "database/creds/customer" {
capabilities = ["read"]
}
path "sys/renew/*" {
capabilities = ["update"]
}' | vault policy write customer -

#Mount transit backend
vault secrets enable transit

#Create transit key
vault write -f transit/keys/customer

Spring boot client

Our client is a spring boot application and will be using spring cloud vault to interact with the Vault server. It is important to note that the configuration for the vault must be in bootstrap.yml. Since this needs to be loaded by the application at the very beginning of the spring container initialization. And the VAULT_TOKEN is already exported in the script above.

spring:
cloud:
vault:
host: localhost
port: 8200
scheme: http
authentication: TOKEN
token: ${VAULT_TOKEN:}

JPA's AttributeConverter

Let's create an example entity called Customer. The property creditCardNumber is annotated with the interface Convert and we have to supply a converter. We will implement the TransitConverter which implements javax.persistence.AttributeConverter

private String id, firstName, lastName, emailAddress;

@Convert(converter = TransitConverter.class)
private String creditCardNumber;

Our TransitConverter needs to implement two methods convertToDatabaseColumn and convertToEntityAttribute. VaultOperations bean shall be used to encrypt and decrypt.

@Override public String convertToDatabaseColumn(String creditCardNumber) {
VaultOperations vaultOperations = BeanUtil.getBean(VaultOperations.class);
Plaintext plaintext = Plaintext.of(creditCardNumber);
return vaultOperations.opsForTransit().encrypt("customer", plaintext)
.getCiphertext();
}

@Override public String convertToEntityAttribute(String creditCardNumber) {
VaultOperations vaultOperations = BeanUtil.getBean(VaultOperations.class);
Ciphertext ciphertext = Ciphertext.of(creditCardNumber);
return vaultOperations.opsForTransit().decrypt("customer",
ciphertext).asString();
}

Let's fire up some test data and test. I will be using JOOQ to insert some data. JPA annotation AttributeConverter is yet to be supported by JOOQ. So we will use the VaultOperations bean directly to encrypt the credit card number in our example. Alternatively, our example application has spring data jpa which can pick up the AttributeConverter. Create the test data either using JOOQ or via the rest endpoint.

curl -X POST \
    http://localhost:8080/customers \
    -H 'cache-control: no-cache' \
    -H 'content-type: application/json' \
    -H 'postman-token: 731a3754-8d49-920e-ba0f-5c33f72af4d5' \
    -d '{
   "id": "john@doe.com",    
   "firstName": "John",
   "lastName": "Doe",
   "emailAddress": "john@doe.com",
   "creditCardNumber": "9999-9999-9999-9999"
  }'

Faker faker = new Faker();
dsl.insertInto(CUSTOMER, CUSTOMER.ID, CUSTOMER.FIRST_NAME, CUSTOMER.LAST_NAME,
CUSTOMER.EMAIL_ADDRESS,
CUSTOMER.CREDIT_CARD_NUMBER)
.values(UUID.randomUUID().toString(),
faker.name().firstName(),
faker.name().lastName(),
faker.internet().emailAddress(),
vaultOperations.opsForTransit().encrypt("customer", Plaintext.of(faker.finance().creditCard()))
.getCiphertext())
| ID                                   | CREDIT_CARD_NUMBER                                                        | EMAIL_ADDRESS             | FIRST_NAME   | LAST_NAME   |
|--------------------------------------|---------------------------------------------------------------------------|---------------------------|--------------|-------------|
| d1d7a86a-6381-40dd-9250-07ee9a57d9e9 | vault:v1:iOZTH5Vsw+9jkC717oGR0NurUSoj/PLUFwvCaPvz3IhdHEl0lgmSzBBHRTkdBT0= | sven.weber@yahoo.com      | Holly        | Ratke       |
| e95c055f-3250-4175-893d-b4e07b743324 | vault:v1:916Ribt50yPvlFkZZyd9tM8ampJAGLSoBx9IrUUPUnfpFqyckz0rvGe0sBiNyao= | odessa.raynor@hotmail.com | Darryl       | Abbott      |

You must notice that the column CREDIT_CARD_NUMBER contains encrypted text and with some special prefix. It indicates that this column is encrypted using vault with the key version v1. Nothing should be stripped from that string since vault will look for the version it has to use. During the select operation from the API value will be decrypted and sent to the client. You can refer to how this works with the below diagram from the official vault documentation. The plaintext data will be sent to the vault for encryption before being persisted into the database and vice versa.

Courtesy: hashicorp.com

Key Rotation

Key rotation is a requirement in many companies and also we should have the capability to rotate the keys seamlessly. The encryption key can be rotated using the web UI or from the command line. While rotating to v2 for encrypting the new values and we can specify the minimum decryption version as well. We have to re-encrypt the old values and then we can update the minimum version for encryption and decryption v2. JOOQ's batch update will be used to insert the re-encrypted values. Notice the new records in the table using the v2. If you the see the old values it will still be using v1.

| ID           | CREDIT_CARD_NUMBER                                                        | EMAIL_ADDRESS   | FIRST_NAME   | LAST_NAME   |
|--------------|---------------------------------------------------------------------------|-----------------|--------------|-------------|
| john@doe.com | vault:v2:S1uVXa1PTRQlbwQw6jyt/UVyG6yVyVdo7NvI7KyAthrHQX9V3Wh2dhpVkhdss9I= | john@doe.com    | John         | Doe         |

And all the old records have been updated with key v2. We used the special function rewrap available with in the vault client to do this.

String cipherText = vaultOperations.opsForTransit().rewrap("customer", customer
.getCreditCardNumber());
queries.add(dsl.update(CUSTOMER)
.set(CUSTOMER.CREDIT_CARD_NUMBER, cipherText)
.where(CUSTOMER.ID.eq(customer.getId())));

// And invoke it from the UpdateController
$ curl http://localhost:8080/update\?version=vault:v2
| ID                                   | CREDIT_CARD_NUMBER                                                        | EMAIL_ADDRESS            | FIRST_NAME   | LAST_NAME   |
|--------------------------------------|---------------------------------------------------------------------------|--------------------------|--------------|-------------|
| 27f6df5d-2ae8-4052-955c-0114a654f45e | vault:v2:yqF/C3P7UZjlZE74BhlZLIDOU+EfIHP+7HrfuZt5Dqj0WpQevgQyg2MZAh0oS7s= | verla.weissnat@yahoo.com | Sibyl        | Goldner     |
| bc8fe9f5-fd35-419e-a18e-eba0183a0d47 | vault:v2:EJWJsqDdjQBbfkE2TnFxidQJmS4QwQ6468nXgXW5iCK3l+jAH74mXF9+PeLvdAo= | briana.sauer@hotmail.com | Urban        | Rodriguez   |
| cfc09d23-fa35-4dd9-9cbf-078806bb95c7 | vault:v2:tCdXX1qeU74Uk5Z+NtfH25NkozZJUz+k2cmHvh+RM+uWaVkLv+p6t0n2i17eepY= | ahmad.corkery@yahoo.com  | Myriam       | Spencer     |

Tracing the DB queries using Zipkin's Brave

We also have used the Zipkin Brave's p6spy to instrument our code to understand the latency of encryption. And you can find the configuration in the source code. So a typical trace for the inserting the record including the encryption looks like this. You may notice there is slight overhead for every write and read in this approach. But it is worth every bit. Because you're offloading the responsibility of encryption to vault provider. Hence, your application doesn't have to use its computing resources to do that. Next, better key management, rotation of keys and policies. Vault has first-class support to create multiple roles and policies. We can create specific policies for insertion and read and provide them to different services. When using vault cryptography key is not colocated. So even if your database is compromised and it is highly unlikely the sensitive information can be decrypted.

Next, let's look at the trace of the update. Here since we have used the batch update all the rows affected will be inserted in one single request. Easily this can be extended to implement a scalable solution by specifying the size in the loader API. Currently, you cannot specify the batch size with JOOQ at the moment. The trace for the batch insert is a result of auto instrumented spans from brave's p6spy. I hoping to experiment with datasource-proxy sometime later though.


Gotchas

  • Never write your own cryptographic algorithms, use the well-known ones which are being constantly tested and upgraded by industry from time to time
  • Do no co-locate the encryption key with your data and it is one of the known common mistakes. Example: Marriot Data Breach
  • Create proper access policies that make sense to your business and at the same time never compromise on the security
  • Finally always use TLS 1.2 or above

As always the source code can found in the Github.

References:

Share