In the last blog, we stood up the CES nodes and created a small NFS export so here we will add one client to verify if we can actually mount anything via the floating IP’s in CES and attempt so test by shutting down one of the CES nodes and verify reachability from our client.
Before we test NFS mounts, we are going to need a few clients VM so we will just the template and create one client VM’s on each site
Clone both clients and size them down
qm clone 9002 117 --full --name sitea-client01
qm clone 9002 127 --full --name siteb-client01
qm set 117 --memory 2048 --net0 virtio,bridge=vmbr1 --net1 virtio,bridge=vmbr1,tag=1615
qm set 127 --memory 2048 --net0 virtio,bridge=vmbr1,tag=1612 --net1 virtio,bridge=vmbr1,tag=1616
qm start 117
qm start 127Set hostname, reset machine-id, SSH keys and IPs, then reboot.
while read -r ID H M C MGW; do
echo ">> $ID $H"
qm guest exec $ID --timeout 120 -- bash -c "
hostnamectl set-hostname $H
rm -f /etc/machine-id /var/lib/dbus/machine-id && systemd-machine-id-setup
rm -f /etc/ssh/ssh_host_* && ssh-keygen -A
sed -i '/127.0.1.1/d' /etc/hosts
rm -f /etc/netplan/*.yaml
cat > /etc/netplan/50-lab.yaml <<EOF
network:
version: 2
ethernets:
enp6s18:
addresses: [$M/24]
routes: [{to: default, via: $MGW}]
nameservers: {addresses: [172.16.11.5], search: [lab.internal]}
enp6s19:
addresses: [$C/24]
EOF
chmod 600 /etc/netplan/50-lab.yaml
netplan generate && echo done
" | grep -E '"out-data"|"exitcode"'
qm reboot $ID
done <<'LIST'
117 sitea-client01 172.16.11.17 172.16.15.17 172.16.11.1
127 siteb-client01 172.16.12.27 172.16.16.27 172.16.12.1
LISTInstall the NFS client on siteb-client01
ssh awk@172.16.11.17 'sudo apt-get install -y nfs-common'
Testing NFS Clients
Create a directory and mount it
sudo mkdir -p /mnt/nfstest && sudo mount -t nfs -o vers=4.0 sitea-nfs01.lab.internal:/nfstest /mnt/nfstestWrite a file and test
echo "hello from sitea-client01 $(date)" > /mnt/nfstest/hello.txt && cat /mnt/nfstest/hello.txt
Failover test of CES
Start a write loop on the client and leave it running:
echo "hello from sitea-client01 $(date)" > /mnt/nfstest/hello.txt && cat /mnt/nfstest/hello.txtShut one of the CES nodes by issuing a mmces suspend command
mmces address list
mmces node suspend -N sitea-ces01.lab.internal
mmces address list
We are able to write to our NFS path with one CES node down so that looks to be working okay
Just resume CES by running mmces node resume
mmces node resume -N sitea-ces01.lab.internal
We just need to ensure CES is evenly propagated so we issue a