The Volume Mechanism
Volumes are mounted filesystems available to pods and their containers. Volumes may be backed by a number of host-local or network attached storage endpoints. The simplest volume type is EmptyDir, which is a temporary directory on a single machine. Administrators may also allow to request and attach Persistent Volumes.
Various contexts list the following objects as «volumes»:
Volume Security
Volume Types
EmptyDir
An «emptyDir», also known as a «temporary pod volume», is created when the pod is assigned to a node, and exists as long as the pod is running on the node. It is initially empty. Containers in a pod can all read and write the same files in the «emptyDir» volume, though the volume can be mounted at the same or different paths in each container.
When the pod is removed from the node, the data is deleted. If the container crashes, that does not remove the pod from a node, so data in an empty dir is safe across container crashes.
The «emptyDir» volumes are stored on whatever medium is backing the node (disk, network storage). The mapping on the local file system backing the node can be discovered by identifying the container and then executing a docker inspect:
"Mounts": [ "Source": "/var/lib/origin/openshift.local.volumes/pods/1806c74f-0ad4-11e8-85a1-525400360e56/volumes/kubernetes.io~empty-dir/emptydirvol1", "Destination": "/something", "Mode": "Z", "RW": true, "Propagation": "rprivate" > .
When emptyDir is backed by a node directory, the location of this directory is /var/lib/origin/openshift.local.volumes/pods//volumes/kubernetes.io~empty- but it can be configured in the Kubernetes node configuration file. How?
«emptyDir» volume storage may be restricted by a quota based on the pod’s FSGroup, if the FSGroup parameter is enabled by the cluster administrator.
This is how an emptyDir is declared as part of a deployment configuration:
spec: template: spec: containers: - name: some-container volumeMounts: - name: blue mountPath: /blue . volumes: - name: blue emptyDir: medium: sizeLimit:
medium — what type of storage medium should back this directory. The default (unspecified) is to use the node’s default medium. The alternative is «Memory».
sizeLimit — the total amount of local storage required for this emptyDir volume. The size limit is also applicable for the memory medium; for this case the emptyDir would be the minimum value between the SizeLimit specified here and the sum of memory limits of all containers in the pod. The default is nil, which means the limit is undefined (requires Kubernetes 1.8).
EmptyDir Quota
By default, pods can allocate on emptyDirs as much space as the underlying filesystem has available. It is possible to set up a configuration that would restrict space allocation to a quota enforced with the XFS quota mechanism, provided that the backing filesystem used to allocate space for emptyDirs is XFS. For more details see:
EmptyDir Operations
hostPath
spec: template: spec: containers: - name: fluentd-elasticsearch . volumeMounts: - mountPath: /var/log name: varlog . volumes: - name: varlog hostPath: path: /var/log
local
Persistent Volume Claim
A persistent volume claim is a request for a persistence resource with specific attributes, such as storage size. The persistent volume claims are created by the developers who manage the pods. Persistent volume claims are project-specific objects.
Persistent volume claims are matched to available volumes and binds the pod to the volume. This process allows a claim to be used as a volume in a pod: OpenShift finds the volume backing the claim and mounts it into the pod. Multiple PVCs within the same project can bind to the same PV. Once a PVC binds to a PV, that PV cannot be bound by a claim outside the first claim’s project — the PV «belongs» to the project. If the underlying storage needs to be accessed by multiple projects, that can be implemented by declaring multiple PVs, where each project needs its own PV, which can then point to the same physical storage.
The pod can be disassociated from the persistent volume by deleting the persistent volume claim. The persistent volume transitions from a «Bound» to «Released» state. To make the persistent volume «Available» again, edit it and remove the persistent volume claim reference, as shown here. Transitioning the persistent volume from «Released» to «Available» state does not clear the storage content — this will have to be done manually.
All persistent volume claims for the current project can be listed with:
ConfigMap
A ConfigMap is a component that holds key/value pairs of configuration data, and that can be consumed by pods, or can be used to store configuration for OpenShift system components such as controllers. It is a mechanism to inject containers with configuration while keeping the containers agnostic of the OpenShift platform. Aside from fine-grained information like individual properties, ConfigMaps can also store coarse-grained information such as entire configuration files or JSON blobs. The ConfigMaps can populate environment variables, set command-line arguments in a container and populate configuration files in a volume.
A ConfigMap is similar to a secret, but designed to be more convenient when working with strings that do not contain sensitive information.
ConfigMap can be created from directories, files, literal values.
The ConfigMaps must be created before they are consumed in pods. They cannot be shared between projects. If the ConfigMap is updated, it must be redeployed in then pod for the pod to see the changes.
Downward API
The downward API allows containers to consume information about OpenShift objects. The field within a pod are selected using ‘fieldRef’ API type, which has two types: ‘fieldPath’, the path of field to select relative to the pod and ‘apiVersion’. The downward API exposes the following selectors:
- metadata.name: the pod name.
- metadata.namespace: pod namespace.
- metadata.labels
- metadata.annotations
- status.podIP
‘resourceFieldRef’/’resource’, which refers to the resource entries.
The information can be exposed to pods via environment variables and volumes.
Secret
projected
NFS
These are the persistent volume mounts:
support.ocp36.local:/nfs/pv2 80G 7.0G 73G 9% \ /var/lib/origin/openshift.local.volumes/pods/cbc6c6f4-fda5-11e7-a14c-525400360e56/volumes/kubernetes.io~nfs/pv2
Persistent Volume
Persistent volumes are created by the cluster administrators.
Persistent volumes can be listed with oc get pv.
Chapter 26. Managing Volumes
Containers are not persistent by default; on restart, their contents are cleared. Volumes are mounted file systems available to pods and their containers which may be backed by a number of host-local or network attached storage endpoints.
To ensure that the file system on the volume contains no errors and, if errors are present, to repair them when possible, OpenShift Container Platform invokes the fsck utility prior to the mount utility. This occurs when either adding a volume or updating an existing volume.
The simplest volume type is emptyDir , which is a temporary directory on a single machine. Administrators may also allow you to request a persistent volume that is automatically attached to your pods.
emptyDir volume storage may be restricted by a quota based on the pod’s FSGroup, if the FSGroup parameter is enabled by your cluster administrator.
You can use the CLI command oc set volume to add, update, or remove volumes and volume mounts for any object that has a pod template like replication controllers or deployment configurations. You can also list volumes in pods or any object that has a pod template.
26.2. General CLI Usage
The oc set volume command uses the following general syntax:
$ oc set volume
This topic uses the form / for in later examples. However, you can choose one of the following options:
Table 26.1. Object Selection
Selects of type .
Selects of type .
Selects resources of type that matched the given label selector.
Selects all resources of type .
File name, directory, or URL to file to use to edit the resource.
The can be one of —add , —remove , or —list .
Any or are specific to the selected operation and are discussed in later sections.
26.3. Adding Volumes
To add a volume, a volume mount, or both to pod templates:
$ oc set volume / --add [options]
Table 26.2. Supported Options for Adding Volumes
Name of the volume.
Automatically generated, if not specified.
Name of the volume source. Supported values: emptyDir , hostPath , secret , configmap , persistentVolumeClaim or projected .
Select containers by name. It can also take wildcard ‘*’ that matches any character.
Mount path inside the selected containers.
Host path. Mandatory parameter for —type=hostPath .
Name of the secret. Mandatory parameter for —type=secret .
Name of the configmap. Mandatory parameter for —type=configmap .
Name of the persistent volume claim. Mandatory parameter for —type=persistentVolumeClaim .
Details of volume source as a JSON string. Recommended if the desired volume source is not supported by —type .
Display the modified objects instead of updating them on the server. Supported values: json , yaml .
Output the modified objects with the given version.
Examples
Add a new volume source emptyDir to deployment configuration registry :
$ oc set volume dc/registry --add
Add volume v1 with secret secret1 for replication controller r1 and mount inside the containers at /data :
$ oc set volume rc/r1 --add --name=v1 --type=secret --secret-name='secret1' --mount-path=/data
Add existing persistent volume v1 with claim name pvc1 to deployment configuration dc.json on disk, mount the volume on container c1 at /data , and update the deployment configuration on the server:
$ oc set volume -f dc.json --add --name=v1 --type=persistentVolumeClaim \ --claim-name=pvc1 --mount-path=/data --containers=c1
26.4. Updating Volumes
Updating existing volumes or volume mounts is the same as adding volumes, but with the —overwrite option:
$ oc set volume / --add --overwrite [options]
Examples
Replace existing volume v1 for replication controller r1 with existing persistent volume claim pvc1 :
$ oc set volume rc/r1 --add --overwrite --name=v1 --type=persistentVolumeClaim --claim-name=pvc1
Change deployment configuration d1 mount point to /opt for volume v1 :
$ oc set volume dc/d1 --add --overwrite --name=v1 --mount-path=/opt
26.5. Removing Volumes
To remove a volume or volume mount from pod templates:
$ oc set volume / --remove [options]
Table 26.3. Supported Options for Removing Volumes
Name of the volume.
Select containers by name. It can also take wildcard ‘*’ that matches any character.
Indicate that you want to remove multiple volumes at once.
Display the modified objects instead of updating them on the server. Supported values: json , yaml .
Output the modified objects with the given version.
Examples
Remove a volume v1 from deployment configuration d1 :
$ oc set volume dc/d1 --remove --name=v1
Unmount volume v1 from container c1 for deployment configuration d1 and remove the volume v1 if it is not referenced by any containers on d1 :
$ oc set volume dc/d1 --remove --name=v1 --containers=c1
Remove all volumes for replication controller r1 :
$ oc set volume rc/r1 --remove --confirm
26.6. Listing Volumes
To list volumes or volume mounts for pods or pod templates:
$ oc set volume / --list [options]
List volume supported options:
Name of the volume.
Select containers by name. It can also take wildcard ‘*’ that matches any character.
Examples
List all volumes for pod p1 :
$ oc set volume pod/p1 --list
List volume v1 defined on all deployment configurations:
$ oc set volume dc --all --name=v1
26.7. Specifying a Sub-path
Use the volumeMounts.subPath property to specify a subPath inside a volume instead of the volume’s root. subPath allows you to share one volume for multiple uses in a single pod.
To view the list of files in the volume, run the oc rsh command:
$ oc rsh sh-4.2$ ls /path/to/volume/subpath/mount example_file1 example_file2 example_file3
Specify the subPath :
Example subPath Usage
apiVersion: v1 kind: Pod metadata: name: my-site spec: containers: - name: mysql image: mysql volumeMounts: - mountPath: /var/lib/mysql name: site-data subPath: mysql 1 - name: php image: php volumeMounts: - mountPath: /var/www/html name: site-data subPath: html 2 volumes: - name: site-data persistentVolumeClaim: claimName: my-site-data
Chapter 27. Managing Volumes
Containers are not persistent by default; on restart, their contents are cleared. Volumes are mounted file systems available to pods and their containers which may be backed by a number of host-local or network attached storage endpoints.
To ensure that the file system on the volume contains no errors and, if errors are present, to repair them when possible, OpenShift Container Platform invokes the fsck utility prior to the mount utility. This occurs when either adding a volume or updating an existing volume.
The simplest volume type is emptyDir , which is a temporary directory on a single machine. Administrators may also allow you to request a persistent volume that is automatically attached to your pods.
emptyDir volume storage may be restricted by a quota based on the pod’s FSGroup, if the FSGroup parameter is enabled by your cluster administrator.
You can use the CLI command oc volume to add, update, or remove volumes and volume mounts for any object that has a pod template like replication controllers or deployment configurations. You can also list volumes in pods or any object that has a pod template.
27.2. General CLI Usage
The oc volume command uses the following general syntax:
$ oc volume
This topic uses the form / for in later examples. However, you can choose one of the following options:
Table 27.1. Object Selection
Selects of type .
Selects of type .
Selects resources of type that matched the given label selector.
Selects all resources of type .
File name, directory, or URL to file to use to edit the resource.
The can be one of —add , —remove , or —list .
Any or are specific to the selected operation and are discussed in later sections.
27.3. Adding Volumes
To add a volume, a volume mount, or both to pod templates:
$ oc volume / --add [options]
Table 27.2. Supported Options for Adding Volumes
Name of the volume.
Automatically generated, if not specified.
Name of the volume source. Supported values: emptyDir , hostPath , secret , configmap , persistentVolumeClaim or projected .
Select containers by name. It can also take wildcard ‘*’ that matches any character.
Mount path inside the selected containers.
Host path. Mandatory parameter for —type=hostPath .
Name of the secret. Mandatory parameter for —type=secret .
Name of the configmap. Mandatory parameter for —type=configmap .
Name of the persistent volume claim. Mandatory parameter for —type=persistentVolumeClaim .
Details of volume source as a JSON string. Recommended if the desired volume source is not supported by —type .
Display the modified objects instead of updating them on the server. Supported values: json , yaml .
Output the modified objects with the given version.
Examples
Add a new volume source emptyDir to deployment configuration registry :
$ oc volume dc/registry --add
Add volume v1 with secret $ecret for replication controller r1 and mount inside the containers at /data :
$ oc volume rc/r1 --add --name=v1 --type=secret --secret-name='$ecret' --mount-path=/data
Add existing persistent volume v1 with claim name pvc1 to deployment configuration dc.json on disk, mount the volume on container c1 at /data , and update the deployment configuration on the server:
$ oc volume -f dc.json --add --name=v1 --type=persistentVolumeClaim \ --claim-name=pvc1 --mount-path=/data --containers=c1
Add volume v1 based on Git repository https://github.com/namespace1/project1 with revision 5125c45f9f563 for all replication controllers:
$ oc volume rc --all --add --name=v1 \ --source='>'
27.4. Updating Volumes
Updating existing volumes or volume mounts is the same as adding volumes, but with the —overwrite option:
$ oc volume / --add --overwrite [options]
Examples
Replace existing volume v1 for replication controller r1 with existing persistent volume claim pvc1 :
$ oc volume rc/r1 --add --overwrite --name=v1 --type=persistentVolumeClaim --claim-name=pvc1
Change deployment configuration d1 mount point to /opt for volume v1 :
$ oc volume dc/d1 --add --overwrite --name=v1 --mount-path=/opt
27.5. Removing Volumes
To remove a volume or volume mount from pod templates:
$ oc volume / --remove [options]
Table 27.3. Supported Options for Removing Volumes
Name of the volume.
Select containers by name. It can also take wildcard ‘*’ that matches any character.
Indicate that you want to remove multiple volumes at once.
Display the modified objects instead of updating them on the server. Supported values: json , yaml .
Output the modified objects with the given version.
Examples
Remove a volume v1 from deployment configuration d1 :
$ oc volume dc/d1 --remove --name=v1
Unmount volume v1 from container c1 for deployment configuration d1 and remove the volume v1 if it is not referenced by any containers on d1 :
$ oc volume dc/d1 --remove --name=v1 --containers=c1
Remove all volumes for replication controller r1 :
$ oc volume rc/r1 --remove --confirm
27.6. Listing Volumes
To list volumes or volume mounts for pods or pod templates:
$ oc volume / --list [options]
List volume supported options:
Name of the volume.
Select containers by name. It can also take wildcard ‘*’ that matches any character.
Examples
List all volumes for pod p1 :
$ oc volume pod/p1 --list
List volume v1 defined on all deployment configurations:
$ oc volume dc --all --name=v1
27.7. Specifying a Sub-path
Use the volumeMounts.subPath property to specify a subPath inside a volume instead of the volume’s root. subPath allows you to share one volume for multiple uses in a single pod.
To view the list of files in the volume, run the oc rsh command:
$ oc rsh sh-4.2$ ls /path/to/volume/subpath/mount example_file1 example_file2 example_file3
Specify the subPath :
Example subPath Usage
apiVersion: v1 kind: Pod metadata: name: my-site spec: containers: - name: mysql image: mysql volumeMounts: - mountPath: /var/lib/mysql name: site-data subPath: mysql 1 - name: php image: php volumeMounts: - mountPath: /var/www/html name: site-data subPath: html 2 volumes: - name: site-data persistentVolumeClaim: claimName: my-site-data
Mounting volumes on pods
Hi, during a lab I had a series of preconfigured projects, each running different pods. I was asked to mount volumes, set secrets and so son. However, usually the fundamental parameter for all of these commands is the deployment from which the pod originates (unless there is another way, I’m open to suggestions). In order to have a deployment to work on, I’ve run something like
oc get pod mypod -o yaml > mypod.yaml
Edited the mypod.yaml file and then, to have a new pod running:
oc create -f mypod.yaml
And then run the mounts, etc. that require a deployment.