Preparing a CentOS or RHEL 7 system for running containers

https://platform9.com/support/preparing-centos-7-system-running-containers/

https://docs.docker.com/engine/userguide/storagedriver/device-mapper-driver/#configure-direct-lvm-mode-for-production

Docker Container Deployment

Step 1: read general requirements checklist

Step 2: install and patch operating system

  • Install the latest version of the operating system using a “base” or “minimal” package set.
  • Update all packages to receive the latest security and bug fixes:
    yum update

Step 3: disable incompatible services

  • In general, most network applications should be uninstalled or disabled because they can interfere with Docker and Kubernetes networking services. In particular, httpd must not be running, since its use of ports 80 and 443 will conflict with the Kubernetes reverse proxy. If httpd is present and cannot be uninstalled, run these two commands to permanently disable it:
    systemctl stop httpd
    system disable httpd
  • There is a known incompatibility between firewalld and Docker’s use of iptables, and it is documented at https://github.com/docker/docker/issues/16137. To disable firewalld:
    systemctl stop firewalld
    systemctl disable firewalld

Step 4: verify general system requirements

  • The /var/lib directory should be on a file system with at least 30 GB of free disk space, and more if DeviceMapper Direct mode is not used (see Step 6 below).
  • The machine must have at least one physical (or VLAN-mapped) NIC with an IP address.
  • The machine must have direct access to the Internet. If access through a proxy is required, contact your Platform9 representative for additional configuration instructions. During Platform9 configuration of container-related software, the following types of data sources will be accessed:
    • CentOS yum repository
    • Docker yum repository
    • Public Docker registries from Docker, Inc. and Google (Kubernetes project).

Step 5: configure firewall settings

Each host must allow incoming connections on the following ports

  • TCP (required): 443, 8080, 4001
  • TCP (recommended): 80
  • UDP (required): 8285

Step 6: create docker storage

On CentOS/RHEL 7, Docker uses the “devicemapper” storage driver by default to manage container images and disk layers. For production, the storage driver must be configured to use “direct-lvm” mode (The “loop-lvm” mode is acceptable for testing, but is not supported). The “direct-lvm” mode requires one free block device (a disk or a partition).

WARNING: This procedure will delete all existing Docker images and containers.

If a free block device is available, go to Step 1B.

1A. Create a block device

You can attach a new disk, or create a new partition. The block device should be at least 40 GB in size. Attaching a new disk is outside the scope of these instructions. To create a new partition, use fdisk. Set the partition type to 8e (Linux LVM). See http://tldp.org/HOWTO/Partition/fdisk_partitioning.html for detailed information on fdisk.

1B. Note the path of the block device, e.g., /dev/sdb for a disk, /dev/sdc1 for a partition.

2. Create an LVM thin pool

Ensure that LVM is installed on the host:

yum list lvm2

The “lvm2” package should be listed as installed. If it is not, then install it:

yum install lvm2

Invoke the following bash script with the block name noted in Step 1B:

bd2tp.sh BLOCK_DEVICE_NAME "docker-vg"
#!/usr/bin/env bash

function create_thinpool_from_block_device()
{
local block_dev="$1"
local vg_name="$2"

# Create physical volume
pvcreate "$block_dev"

# Create volume group
vgcreate "$vg_name" "$block_dev"

# Create logical volumes (one for data, another for metadata)
lvcreate --wipesignatures y -n thinpool "$vg_name" -l 95%VG
lvcreate --wipesignatures y -n thinpoolmeta "$vg_name" -l 1%VG

# Convert data volume to a thin volume, using metadata volume for thin volume metadata
lvconvert -y --zero n -c 512K --thinpool "$vg_name/thinpool" --poolmetadata "$vg_name/thinpoolmeta"

# Ensure both volumes are extended as necessary
# 1. Create a profile
cat > "/etc/lvm/profile/$vg_name-thinpool.profile" <<EOF
activation {
thin_pool_autoextend_threshold=80
thin_pool_autoextend_percent=20
}
EOF
# 2. Link profile to data volume
lvchange --metadataprofile "$vg_name-thinpool" "$vg_name/thinpool"
# 3. Enable monitoring of data volume size, so that extension is triggered automatically
lvs -o+seg_monitor
}

function usage()
{
cat >&2 <<EOF
Usage:

bd2tp.sh BLOCK_DEV VOL_GRP_NAME

Creates an lvm thin pool in the VOL_GRP_NAME volume group (e.g. docker-vg)
using the BLOCK_DEV block device (e.g. /dev/xvdb).

NOTE: There is a set of rules that determine valid volume group names. This
script does not validate the name. See the lvm manpage for details.
EOF
}

if [ "$#" -ne 2 ]; then
usage
exit 1
else
echo create_thinpool_from_block_device "$1" "$2"
fi

------------------------------

3. Remove old docker data

If you have ever run Docker on this host before, or if /var/lib/docker/ exists, move it out of the way so that Docker can use the new LVM pool to store the contents of image and containers.
  1. $ mkdir /var/lib/docker.bk
    $ mv /var/lib/docker/* /var/lib/docker.bk
    


4. Example trace of configuration
 

#yum install lvm2
#pvcreate /dev/sdb
#vgcreate docker_vg /dev/sdb
#lvcreate --wipesignatures y -n thinpool docker_vg -l 95%VG
#lvcreate --wipesignatures y -n thinpoolmeta docker_vg -l 1%VG
#lvconvert -y --zero n -c 512K --thinpool "docker_vg/thinpool" --poolmetadata "docker_vg/thinpoolmeta"
#vi /etc/lvm/profile/docker_vg-thinpool.profile
#lvchange --metadataprofile "docker_vg-thinpool" "docker_vg/thinpool"
#lvs -o+seg_monitor
#lvs
#vi /etc/docker/daemon.json
#cat /etc/docker/daemon.json
{
 "storage-driver": "devicemapper",
 "storage-opts": [
 "dm.thinpooldev=/dev/mapper/docker_vg-thinpool",
 "dm.use_deferred_removal=true",
 "dm.use_deferred_deletion=true"
 ]
}
#systemctl stop docker
#mkdir /var/lib/docker.bk
#mv /var/lib/docker/* /var/lib/docker.bk
#systemctl start docker
#docker info | less

5. List current lvm2 volumes and disk usage

# pvs
 PV VG Fmt Attr PSize PFree
 /dev/sda2 cl lvm2 a-- 277.93g 4.00m
 /dev/sdb1 docker lvm2 a-- 30.00g 928.00m
 /dev/sdb2 docker_vg lvm2 a-- 50.00g 1.51g