In this blog I will provide the steps to install the NetApp Docker Volume Plugin (nDVP) on CoreOS manually and automatically with cloud-config.
First, some introductions:
- NetApp Docker Volume Plugin (nDVP):
- Provides direct integration with the Docker ecosystem for NetApp storage including ONTAP, E-Series, and SolidFire. The nDVP allows for Docker integrated storage management including provisioning/deprovisioning, attaching/detaching, and cloning of NetApp storage. The nDVP software (a single binary) must be installed on the container host and reads its configuration from a JSON formatted config file when it is started. Each config file describes the connection with a single storage array config; multiple arrays == multiple nDVP daemons, each with a unique config file.
- Provides direct integration with the Docker ecosystem for NetApp storage including ONTAP, E-Series, and SolidFire. The nDVP allows for Docker integrated storage management including provisioning/deprovisioning, attaching/detaching, and cloning of NetApp storage. The nDVP software (a single binary) must be installed on the container host and reads its configuration from a JSON formatted config file when it is started. Each config file describes the connection with a single storage array config; multiple arrays == multiple nDVP daemons, each with a unique config file.
- CoreOS:
- A barebones linux distribution designed specifically to run containers at scale. Some common linux features like the OS installer and a package manager are absent. On the other hand, container ecosystem software like docker, rkt, and etcd are pre-installed. CoreOS is also self-updating and does so for all components in an atomic manner using an A/B partition strategy which causes the
/usr
filesystem to be read-only. Given CoreOS is a bit unique in how it works installing software on it needs some special care.
- A barebones linux distribution designed specifically to run containers at scale. Some common linux features like the OS installer and a package manager are absent. On the other hand, container ecosystem software like docker, rkt, and etcd are pre-installed. CoreOS is also self-updating and does so for all components in an atomic manner using an A/B partition strategy which causes the
Install nDVP manually
To install manually follow these steps:
Become root:
sudo -i
If you will use iSCSI connected storage start the iSCSI service:
systemctl start iscsid
Create a nDVP configuration file for your storage array type.
- Copy a template from nDVP Github site for your platform and customize to match your site details.
- Validate the JSON using JSONlint to avoid a syntax error that will prevent the driver from starting
Save this file to your container host. There are no requirements about name or directory location but for standardization the following filenames (matching the driver name) are suggested:
Storage Driver Full Path to Config file SolidFire iSCSI /etc/netappdvp-solidfire-san.json
ONTAP NFS /etc/netappdvp-ontap-nas.json
ONTAP iSCSI /etc/netappdvp-ontap-san.json
E-Series iSCSI /etc/netappdvp-eseries-iscsi.json
- Copy a template from nDVP Github site for your platform and customize to match your site details.
Download, install, and start the plugin.
- Replace the FULL-PATH-TO-CONFIG-FILENAME with the name of your config file
Replace the version of netappdvp with your desired version
# Make a directory for the nDVP mkdir -p /opt/netappdvp # Download and install the software wget https://github.com/NetApp/netappdvp/releases/download/v1.3.3/netappdvp-1.3.3.tar.gz -O /tmp/netappdvp-1.3.3.tar.gz tar -zxvf /tmp/netappdvp-1.3.3.tar.gz -C /opt/netappdvp chown root:root /opt/netappdvp/netappdvp chmod 755 /opt/netappdvp/netappdvp # Start the nDVP passing the full path to your config file, and put it to the background /opt/netappdvp/netappdvp --config=CONFIG-FILENAME &
You should now be able to use
docker volume COMMAND -d netapp
commands to manage persistent storage.
Install nDVP with cloud-config
To install using cloud-config
you need to add a few new snippets. These instructions assume you will distribute using a Config Drive and a user_data
file:
Create a cloud-config file named
user_data
in the current working directory.Create a nDVP configuration file for your storage array type.
- Copy a template from nDVP Github site for your platform into a text editor and customize to match your site details. It will be added to the
cloud-config
in the next step. - Validate the JSON using JSONlint to avoid a syntax error that will prevent the driver from starting
- Copy a template from nDVP Github site for your platform into a text editor and customize to match your site details. It will be added to the
Add a
write_files
definition touser_data
so your nDVP config file will be saved on the container host:- There are no requirements about name or directory location but for standardization the following filenames (matching the driver name) are suggested:
Storage Driver Full Path to Config file SolidFire iSCSI /etc/netappdvp-solidfire-san.json
ONTAP NFS /etc/netappdvp-ontap-nas.json
ONTAP iSCSI /etc/netappdvp-ontap-san.json
E-Series iSCSI /etc/netappdvp-eseries-iscsi.json
Replace FULL-PATH-TO-CONFIG-FILENAME with the name of your nDVP config file including the full path (example:
/etc/netappdvp-solidfire-san.json
)# Write out nDVP config files write_files: - path: "FULL-PATH-TO-CONFIG-FILENAME" permissions: "0644" owner: "root" content: | # Paste your nDVP config JSON snippet here; indentation in YAML matters!
If you will use iSCSI connected storage start the iSCSI service by adding a new unit definition to your
user_data
:coreos: units: # Start iSCSI - name: iscsid.service command: start
Add a unit definition to download, install, and start the nDVP.
- Remove the
coreos:
andunits:
lines if you already have this header. - Remove the
After=iscsid.service
line if you are not using iSCSI - Replace FULL-PATH-TO-CONFIG-FILENAME with the name of your nDVP config file including the full path (example:
/etc/netappdvp-solidfire-san.json
) Replace the version of netappdvp with your desired version
coreos: units: # Install and start ndvp service - name: "ndvp.service" command: "start" content: | [Unit] Description=nDVP service After=iscsid.service [Service] Restart=always ExecStartPre=-/usr/bin/mkdir -p /opt/netappdvp ExecStartPre=-/usr/bin/wget https://github.com/NetApp/netappdvp/releases/download/v1.3.3/netappdvp-1.3.3.tar.gz -O /tmp/netappdvp-1.3.3.tar.gz ExecStartPre=-/usr/bin/tar -zxvf /tmp/netappdvp-1.3.3.tar.gz -C /opt/netappdvp ExecStart=/opt/netappdvp/netappdvp --config=FULL-PATH-TO-CONFIG-FILENAME
- Remove the
Deploy! You should be able to use
docker volume COMMAND -d netapp
commands to manage persistent storage.
Troubleshooting
The nDVP daemon must be running for it to manage NetApp storage. Check that it is running with:
ps -ef | grep netappdvp
Check the nDVP logfile for status or error messages:
tail /var/log/netappdvp/*
Enable debug logging, try the action again, and then check the log. Enable debug logging in one of two ways:
(a) Edit your nDVP config JSON to include and then restart the nDVP:
"debug": true,
or (b) [Re]start the netappdvp with the
-debug
flag:/opt/netappdvp/netappdvp --config=FULL-PATH-TO-CONFIG-FILENAME -debug
As always, comments are welcome!