How to Use Secrets in Kubernetes Applications

In this blog post, you will learn how to use Secrets in Kubernetes applications. In Kubernetes, managing configuration data and sensitive information securely is crucial for maintaining robust and scalable applications. Kubernetes provides two primary resources for this purpose: Secrets and ConfigMaps. Understanding how to effectively create, use, and manage these resources is essential for Kubernetes administrators and developers.

Introduction to Secrets and ConfigMaps in Kubernetes

Secrets and ConfigMaps in Kubernetes provides a way to manage and store configuration data and sensitive information such as passwords for Kubernetes applications. But what are they exactly?

What are Secrets in Kubernetes?

In Kubernetes, Secrets are objects that are used to store sensitive information such as passwords, OAuth tokens, keys, TLS certificates and any other sensitive data required by applications thus reducing the risk of accidental exposure. They are stored as Base64-encoded strings by default.

While Kubernetes Secrets provide a convenient way to manage sensitive information within your cluster, they are not fully secure against potential threats. First of all, they are not encrypted by default and thus, anyone with API access, or access to etcd cluster can access, view or modify the secrets. Best practices include limiting access to Secrets through RBAC, rotating them regularly, and using additional encryption or external secret management tools where needed for enhanced security.

What are ConfigMaps in Kubernetes?

On the other hand, ConfigMaps are Kubernetes API objects that are used store and manage non-sensitive configuration data in key-value pairs. ConfigMaps provides a way to decouple configuration artifacts from the application code thus making containerized applications portable.

ConfigMaps are often used to store configuration parameters such as:

1. Environment variables: You can inject ConfigMap data into a container as environment variables, which allows you to configure how the application behaves without changing its container image.
2. Configuration files: ConfigMaps can also be mounted as volumes into containers, allowing applications to read configuration files directly from the volume.
3. Command Line arguments: ConfigMaps can also store command-line arguments that can be passed to containers in Kubernetes. This is particularly useful when applications need to start with specific runtime options or flags.

ConfigMaps are useful when you want to:

• Centralize configuration: Instead of embedding configuration directly into application code, container images or Kubernetes manifests, ConfigMaps provide a centralized way to manage configuration data.
• Override configuration: You can update ConfigMaps independently of the application code, which makes it easier to change configurations without rebuilding or redeploying the containers.

Read more on Step-by-Step Guide to Using ConfigMaps in Kubernetes

The table below gives a summary of the difference between ConfigMaps and Secrets in Kubernetes;

Are you getting started with Kubernetes and you need a book that offers a beginner-friendly approach to mastering Kubernetes architecture and core concepts such as Pods, Deployments, Services, StatefulSets, Ingress, ConfigMaps, and more? Look no further, The Kubernetes Book: 2024 Edition by Nigel Poulton is the best bet.

Types of Kubernetes Secrets

There are different types of Kubernetes Secrets as outlined below;

Opaque Secrets

• Opaque secrets are generic secrets that store arbitrary data as key-value pairs. They are encoded in base64 within Kubernetes to maintain confidentiality.
• They are commonly used for storing passwords, API tokens, and other sensitive data that applications need to access securely.
• It is the default Secret type if you don’t explicitly specify a type in a Secret manifest or command line.

Service Account Token

• ServiceAccoun token secrets are used to store a token credential that identifies a ServiceAccount in Kubernetes.
• Pods use these tokens to authenticate and interact securely with the Kubernetes API server, enabling them to perform actions such as retrieving configuration information or accessing other resources.
• type: kubernetes.io/service-account-token

Docker Config Secrets

• These are secrets used to authenticate with private Docker registries.
• They store Docker registry credentials (username, password, or token) securely, allowing Kubernetes to pull private images from Docker registries during deployments.
• type: kubernetes.io/dockercfg

Basic Authentication Secrets

• These are secrets that are used for basic authentication, typically used for integrating with legacy systems or services that require simple username/password authentication.
• They store credentials in a manner similar to Opaque secrets but are specifically used for basic authentication purposes.
• type: kubernetes.io/basic-auth

SSH Authentication Secrets

• They are used to authenticate SSH connections.
• They store SSH keys and other related data required for secure SSH access to pods or nodes within the Kubernetes cluster.
• type: kubernetes.io/ssh-auth

TLS Secrets

• They are secrets used for storing TLS certificates and private keys.
• They secure communication within Kubernetes, such as enabling HTTPS endpoints for services or securing communication between pods.
• type: kubernetes.io/tls or –type=’tls’

Bootstrap Secrets

• These are secrets used during the initial bootstrapping of Kubernetes cluster nodes.
• They are used for securely configuring and initializing the Kubernetes cluster, including initial setup and configuration tasks.
• type: bootstrap.kubernetes.io/token

How to Use Secrets in Kubernetes

Creating Kubernetes Secrets

Kubernetes offers multiple ways to create secrets, imperatively using kubectl create secret or declaratively using the manifest file. You can specify the secret type using –type option in command line or type: in the manifest file.

Command Syntax;

kubectl create secret (docker-registry | generic | tls) [options]

Rest of the other Secret types if you are creating them via command line can take the generic type!

1. Creating Secrets from Literal Values (specify each key-value pair directly in the command):

Use kubectl create secret with the –from-literal flag to create secrets directly from command-line input. This method is suitable for small pieces of sensitive data.

You can specify the respective namespace for the secrets (-n|–namespace <namespace>)

For example, to create a secret called user-logins with a username (username) and password (mypassword);

kubectl create secret generic user-logins --from-literal=username=myuser --from-literal=password=mypassword

Replace the values accordingly.

This will create a secret names user-login in the default namespace.

2. Creating Secrets from Files:

Use kubectl create secret with the –from-file flag to create secrets from one or more files. This method is useful for storing larger pieces of sensitive data, such as TLS certificates.

kubectl create secret generic tls-secret --from-file=cert.pem=/path/to/cert.pem --from-file=key.pem=/path/to/key.pem

Replace /path/to/cert.pem and /path/to/key.pem with the actual paths to your certificate and key files.

3. Creating Secret from Environment File:

This allows you to store multiple environment variables in a single file, which simplifies management and reduces the chance of errors.

Create a text file (let’s say env-file.txt) that contains your environment variables in the format KEY=VALUE, each on a new line:

cat env-file.txt

DB_USERNAME=username
DB_PASSWORD=password

Then use kubectl create secret with the --from-env-file option followed by the path to your environment file:

kubectl create secret generic my-secret --from-env-file=env-file.txt

4. Creating a Secret from a Directory

When you have multiple files that you want to store as different keys in a secret, you can put them under a specific directory and create them right from the directory. When creating a secret in Kubernetes based on a directory, each file whose basename (filename without the path) corresponds to a valid key in the directory will be packaged into the secret.

Let’s say you have a directory named secrets with the following files:

• username.txt containing user
• password.txt containing password

tree secrets/

secrets/
├── password.txt
└── username.txt

1 directory, 2 files

To create a Kubernetes secret named mysecret from this directory, where each file becomes a key-value pair in the secret:

kubectl create secret generic mysecret --from-file=secrets/

Kubernetes will package each file (username.txt and password.txt) in the secrets directory into the secret mysecret, with the filenames as keys and their contents as values.

5. Creating Secrets Declaratively from YAML Manifest:

You can define secrets directly in a YAML manifest file. The secret values has to base64-encoded;

cat my-secret.yaml

apiVersion: v1
kind: Secret
metadata:
  name: my-secret
type: Opaque
data:
  username: dXNlcm5hbWU=   # base64 encoded 'username'; echo -n "value" | base64
  password: cGFzc3dvcmQ=   # base64 encoded 'password'; echo -n "value" | base64

Then apply;

kubectl apply -f my-secret.yaml

To specify the namespace, define it under metadata.

apiVersion: v1
kind: Secret
metadata:
  name: my-secret
  namespace: my-namespace
type: Opaque
data:
  username: dXNlcm5hbWU=   # base64 encoded 'username'; echo -n "value" | base64
  password: cGFzc3dvcmQ=   # base64 encoded 'password'; echo -n "value" | base64

Listing Available Secrets in Kubernetes Cluster

You can use kubectl get secrets command to list available secrets in Kubernetes.

To list the secrets in the default namespace;

kubectl get secrets

NAME          TYPE     DATA   AGE
user-logins   Opaque   2      23m

To list secrets in a specific namespace;

kubectl get secrets -n <namespace> 

To list secrets in all namespaces;

kubectl get secrets -A

Get Details of a Secret in Kubernetes

You can use kubectl describe secret or kubectl get secret commands to view secrets details;

kubectl describe secrets <name-of-secret> [-n|--namespace <namespace>]

Or;

kubectl get secrets <name-of-secret> -o yaml [-n|--namespace <namespace>]

You can also decode data field from secret using json output;

kubectl get secret <name-of-secret> -o jsonpath='{.data}' [-n|--namespace <namespace>]

Using Secrets in Deployment/Pods

Once you have created a secret, you can use it in your pods to securely access sensitive information.

There are two ways in which you can inject the secrets into the Pod;

• Via the Environment variables
• By mounting it as volumes in the Pod.

Using Secrets as Environment Variables

Inject secrets as environment variables in your Deployment/pod specifications. For example, below is a manifest file for creating a MariaDB pods with root user password stored in Kubernetes secret.

cat mariadb.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: mariadb-deployment
  namespace: apps
spec:
  replicas: 2
  selector:
    matchLabels:
      app: mariadb
  template:
    metadata:
      labels:
        app: mariadb
    spec:
      containers:
      - name: mariadb
        image: mariadb:latest
        ports:
        - containerPort: 3306
        env:
        - name: MARIADB_ROOT_PASSWORD
          valueFrom:
            secretKeyRef:
              name: mariadb-secret
              key: mariadb-root-password

As you can see, the MariaDB root user password is injected as an environment variable, MYSQL_ROOT_PASSWORD. The value of the environment variable is sourced securely from a Kubernetes Secret named mariadb-secret, using the key mariadb-root-password from that secret.

To create a secret, you can follow steps above.

We have already created one.

kubectl get secret mariadb-secret -n apps -o yaml

apiVersion: v1
data:
  mariadb-root-password: aEBja2VyMTIz
kind: Secret
metadata:
  creationTimestamp: "2024-07-06T19:35:49Z"
  name: mariadb-secret
  namespace: apps
  resourceVersion: "4324348"
  uid: 3316104a-7079-4bba-8751-554637d51b9f
type: Opaque

Then let’s create our deployment;

kubectl apply -f mariadb.yaml

Pods are already running;

kubectl get pods -n apps -l app=mariadb

NAME                                  READY   STATUS    RESTARTS   AGE
mariadb-deployment-7f46d75896-6t666   1/1     Running   0          77s
mariadb-deployment-7f46d75896-g7dss   1/1     Running   0          77s

Let’s try to login to our MariaDB Pod;

kubectl exec -it mariadb-deployment-7f46d75896-6t666 -n apps -- mariadb -u root -p

You will be prompted to enter the password that was defined/stored in the secret.

Upon providing the correct password, you should drop into MariaDB prompt;

Enter password: 
Welcome to the MariaDB monitor.  Commands end with ; or \g.
Your MariaDB connection id is 4
Server version: 11.4.2-MariaDB-ubu2404 mariadb.org binary distribution

Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

MariaDB [(none)]>

Mounting Secrets as Volumes

You can also mount secrets as files in your pod’s filesystem for applications to read.

Let’s create a simple nginx app with basic authentication.

Set the basic auth credentials;

htpasswd -c -B -C 10 ./auth webadmin

Enter the password and confirm.

Create a basic auth secret from basic auth file;

kubectl create secret generic nginx-auth --from-file=./auth -n apps

Create a ConfigMap for Nginx configuration that defines path to basic auth file (/etc/nginx/auth/auth) that will be mounted as a volume in the Pod.

cat nginx.conf

server {
	listen 80 default_server;

	root /usr/share/nginx/html;
	index index.html index.htm index.nginx-debian.html;

	server_name _;
    	location / {
        	try_files $uri $uri/ =404;
    	}	

	location /private {
		auth_basic "Restricted Content";
                auth_basic_user_file /etc/nginx/auth/auth;
		try_files $uri $uri/ /index.html;
	}
}

kubectl create configmap nginx-config --from-file=nginx.conf -n apps

Create the NGINX Deployment;

cat nginx.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment
  namespace: apps
spec:
  replicas: 1
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx:latest
        ports:
        - containerPort: 80
        volumeMounts:
        - name: auth-volume
          mountPath: /etc/nginx/auth
          readOnly: true
        - name: nginx-config
          mountPath: /etc/nginx/conf.d/default.conf
          subPath: nginx.conf
      volumes:
      - name: auth-volume
        secret:
          secretName: nginx-auth
      - name: nginx-config
        configMap:
          name: nginx-config

• The configuration mounts two volumes into the container: auth-volume and nginx-config.
• The auth-volume is sourced from a Kubernetes Secret named nginx-auth, providing Basic Authentication credentials stored securely in /etc/nginx/auth within the container.
• The nginx-config volume is sourced from a ConfigMap named nginx-config, specifically mounting the nginx.conf file from the ConfigMap at /etc/nginx/conf.d/default.conf using subPath to reference the specific file.
• This setup allows the NGINX server to authenticate users access /private page with Basic Authentication and use a custom NGINX configuration defined in the ConfigMap for its operation.

Apply the manifest file;

kubectl apply -f nginx.yaml

Check the Nginx Pods;

kubectl get pods -n apps -l app=nginx

NAME                                READY   STATUS    RESTARTS   AGE
nginx-deployment-6fc846f549-hwjc6   1/1     Running   0          36s

Let’s try to access the default page;

kubectl exec -it nginx-deployment-6fc846f549-hwjc6 -n apps -- curl localhost

Sample output;

<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
html { color-scheme: light dark; }
body { width: 35em; margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif; }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>

<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>

<p><em>Thank you for using nginx.</em></p>
</body>
</html>

Now, try to access restricted page without basic auth creds;

kubectl exec -it nginx-deployment-6fc846f549-hwjc6 -n apps -- curl localhost/private

<html>
<head><title>401 Authorization Required</title></head>
<body>
<center><h1>401 Authorization Required</h1></center>
<hr><center>nginx/1.27.0</center>
</body>
</html>

Authorization required!

Provide authentication credentials and check;

kubectl exec -it nginx-deployment-6fc846f549-hwjc6 -n apps -- curl localhost/private -u webadmin

When prompted for password, provide the password created with htpasswd above.

Enter host password for user 'webadmin':
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
html { color-scheme: light dark; }
body { width: 35em; margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif; }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>

<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>

<p><em>Thank you for using nginx.</em></p>
</body>
</html>

And there you go!

Updating Kubernetes Secrets

There are different ways in which you can update your Kubernetes Secrets;

Editing via kubectl edit

Edit the secret interactively using the kubectl edit command. This opens the secret in your default editor (which can either be vi, nano, etc.) and allows you to modify it directly.

kubectl edit secret <name-of-secret> [-n|--namespace <namespace>]

Using kubectl apply

You can update a secret by applying changes from a YAML file. Once you make the changes to the Secrets manifest file, then re-run the kubectl apply command.

kubectl apply -f updated-secret.yaml

Assuming that updated-secret.yaml contains the updated secret definition with the changes you want to make.

Recreate the Secret with New Value

You can recreate your secret with new value. Replace my-secret with your secret’s name and key=newvalue with the key and new value pair you want to update.

kubectl create secret generic my-secret --from-literal=key=newvalue

Patch Kubernetes Secret with kubectl patch

You can patch the secret to update specific fields using the kubectl patch command.

kubectl patch secret my-secret --type='json' -p='[{"op": "replace", "path": "/data/key", "value": "newvalue"}]'

• Operation (op): "op": "replace": Specifies the operation to perform. In this case, it indicates that you want to replace the value of a specific field.
• Path (path): "path": "/data/key": Defines the path to the field within the secret that you want to update. /data/key refers to the key within the data section of the secret.
• Value (value): "value": "newvalue": Specifies the new value that you want to set for the field identified by the path.

For example, to update my MariaDB root password with the following current details;

kubectl get secret mariadb-secret -n apps -o yaml

apiVersion: v1
data:
  mariadb-root-password: aEBja2VyMTIz
kind: Secret
metadata:
  creationTimestamp: "2024-07-06T19:35:49Z"
  name: mariadb-secret
  namespace: apps
  resourceVersion: "4324348"
  uid: 3316104a-7079-4bba-8751-554637d51b9f
type: Opaque

To patch this secret;

kubectl patch secret mariadb-secret -n apps --type='json' -p='[{"op": "replace", "path": "/data/mariadb-root-password", "value": "newbase64encodedpassword"}]'

Deleting Kubernetes Secrets

To delete Kubernetes secrets, you can use the kubectl delete command.

Deleting a secret is irreversible. Ensure that you have backed up any data or configurations stored in the secret before deleting it. Similarly, ensure that you have appropriate permissions (delete role) to delete secrets in the namespace.

Here are a few ways to delete secrets:

Delete a Specific Secret by Name:

kubectl delete secret <secret-name> -n <namespace>

Replace <secret-name> with the name of the secret you want to delete and <namespace> with the namespace where the secret resides. For example:

kubectl delete secret mariadb-secret -n apps

Delete All Secrets in a Namespace:

To delete all secrets in a specific namespace, you can use a similar command without specifying a label selector:

kubectl delete secret --all -n <namespace>

Replace <namespace> with the namespace where you want to delete all secrets. For example:

kubectl delete secret --all -n apps

Delete Secrets Using Manifest File:

If the secret was created using a manifest file (yaml or json), you can delete it by applying the same manifest file with the delete action:

kubectl delete -f secret-manifest.yaml

Ensure that secret-manifest.yaml contains the correct definition of the secret you want to delete.

Conclusion

And that conclude our guide on how to use Secrets in Kubernetes.

Read more on Kubernetes Secrets page.