Part 1 - Implement & Manage GlusterFS

Gluster FS is a high-performance distributed scalable network file system developed by RedHat that can handle petabytes of data from disks. It offers features like auto-tiering on disks, sharding, replication, etc. GlusterFS is best for large immutable files and less suitable for small files. Let’s take a look at how to get this network file system up and running.

Blog Series

Lab Configuration

I’ve deployed a base server edition of Ubuntu 20.04 on all these servers.

Master Nodes

172.16.11.157 gluster01.ash.local gluster01
172.16.11.158 gluster02.ash.local gluster02
172.16.11.159 gluster03.ash.local gluster03

Client Nodes

172.16.11.160 gluster04.ash.local gluster04

I’ve chosen to deploy it on Ubuntu but this can in fact be deployed on all other variants.

Verify Ubuntu Install

There are a number of steps involved to configure the GlusterFS system and the first of it the OS installation.

Update Ubuntu Repositories & Packages

Run this command to update the default repo and all current packages to the latest.

apt-get update && apt-get upgrade -y

Configure user access to sudoers

Add another user to login to the VM as a root user via editing the sudoers

Configure root access to all servers

For us to do root authentication, we will add these parameters into my sshd config file

Restart ssh

systemctl restart ssh

Ensure all hostnames are valid

Open /etc/hostname and Change the Hostname

Ensure the hostname is set correctly

vi /etc/hosts
172.16.11.157 gluster01.ash.local gluster01
172.16.11.158 gluster02.ash.local gluster02
172.16.11.159 gluster03.ash.local gluster03
172.16.11.160 gluster04.ash.local gluster04

hostnamectl set-hostname new-hostname

Configure Static IP Address on all hosts

This step is necessary to ensure we have a static address on all our servers. Once done, save the file and apply the changes by running the following command:

vi /etc/netplan/00-installer-config.yaml

# This is the network config written by 'subiquity'
network:
 version: 2
 renderer: networkd
 ethernets:
  ens160:
   dhcp4: no
   addresses: [192.168.11.109/24]
   gateway4: 192.168.11.253
   nameservers:
      addresses: [172.16.11.4]

sudo netplan apply

Configure password-less access auth

We will need to generate a key and share out with the public to all servers in the cluster so as to do passwordless authentication.

root@node01:~/.ssh# ssh-keygen
Generating public/private rsa key pair.
Enter file in which to save the key (/root/.ssh/id_rsa): yes
Enter passphrase (empty for no passphrase): 
Enter same passphrase again: 
Your identification has been saved in yes
Your public key has been saved in yes.pub
The key fingerprint is:
SHA256:PkdbnDukhr7t0r08wUsJZkfBZJqpRRZGZkN0G8IcjIA root@gluster01
The key's randomart image is:
+---[RSA 3072]----+
|     ... O#=*.   |
|    E   .*+Ooo   |
|          =..    |
|         o+...   |
|        So.+=.   |
|       . o ==.   |
|        +.=ooo   |
|       ..=..+.   |
|        o+o oo   |
+----[SHA256]-----+
root@gluster01:~#

Copy the public key to all our servers

root@gluster01:~# ssh-copy-id 192.168.11.112
/usr/bin/ssh-copy-id: INFO: Source of key(s) to be installed: "/root/.ssh/id_rsa.pub"
/usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed
/usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys
root@192.168.11.112's password: 

Number of key(s) added: 1

Now try logging into the machine, with:   "ssh '192.168.11.112'"
and check to make sure that only the key(s) you wanted were added.

Disable firewall

This is optional however I’ll do this in my lab

ufw disable

Install and Config GlusterFS

We will install the glusterfs server packages onto all our servers using this command.

 apt-get install glusterfs-server -y

Enable & Start GlusterFS

root@gluster02:~# sudo systemctl start glusterd && sudo systemctl enable glusterd
Created symlink /etc/systemd/system/multi-user.target.wants/glusterd.service → /lib/systemd/system/glusterd.service.

root@gluster02:~#  sudo systemctl status glusterd
● glusterd.service - GlusterFS, a clustered file-system server
     Loaded: loaded (/lib/systemd/system/glusterd.service; enabled; vendor preset: enabled)
     Active: active (running) since Thu 2022-11-17 18:29:21 UTC; 10s ago
       Docs: man:glusterd(8)
   Main PID: 3158 (glusterd)
      Tasks: 9 (limit: 11848)
     Memory: 4.8M
     CGroup: /system.slice/glusterd.service
             └─3158 /usr/sbin/glusterd -p /var/run/glusterd.pid --log-level INFO

Nov 17 18:29:19 gluster02 systemd[1]: Starting GlusterFS, a clustered file-system server...
Nov 17 18:29:21 gluster02 systemd[1]: Started GlusterFS, a clustered file-system server.
root@gluster02:~#

Create a trusted pool

Storage volume is formed by joining disks from multiple nodes by grouping them as a collection of volumes that are exported from every server in its trusted pool to form a storage pool. Use this command to add new nodes into our cluster

oot@gluster01:~# gluster peer probe gluster02
peer probe: success. 
root@gluster01:~# 
root@gluster01:~# gluster peer status
Number of Peers: 1

Hostname: gluster02
Uuid: 7f356799-1049-4c0b-a629-f4eafe610493
State: Peer in Cluster (Connected)


root@gluster01:~# gluster peer probe gluster03
peer probe: success.

Verify the status of the three peers

As seen gluster03 is still showing its state as an Accepted peer request and this should transition into Peer in the Cluster mode.

root@gluster01:~# gluster peer status
Number of Peers: 2

Hostname: gluster02
Uuid: 7f356799-1049-4c0b-a629-f4eafe610493
State: Peer in Cluster (Connected)

Hostname: gluster03
Uuid: 4eb96e86-a0ab-4f2b-997e-34d18ae96c2c
State: Accepted peer request (Connected)
root@gluster01:~#

GlusterFS distributed file system is finally up and running.