Virtualization

Automating Rhel and Ubuntu Image Builds with Packer

By Ash Thomas

Packer is an automated or unattended way of installing an OS from ISO image and converting it to a VM template. Packer creates a template VM automatically and then Terraform will build the VM.

We don’t need to master all these syntaxes at all but just need to understand what each file does so it can be customized later so I’ve made this very simple to do.

Terraform files end in .tf, Packer files end in .pkr.hcl . Here we are going to build a VM Template based on the Rocky 9 Image.

We will use a similar structure for both Rhel and Ubuntu based images so its really easy to follow.

Automating Rhel builds with Packer

This is the file structure I’ve used for Rhel

There are few files here and the customization files here are shared among all templates.

vSphere inventory

  • vsphere.pkrvars.hcl – where in vCenter: datacenter, cluster, datastore, folder, port group

Actual Build file

  • rhel.pkr.hcl – the build, one for all flavours: VM hardware (CPU, RAM, disk, NIC)

Variables

  • alma9.pkrvars.hcl – Alma 9 values: template name, OS type, ISO, repo URLs, temp build IP
  • rocky9.pkrvars.hcl – Rocky 9 values: template name, OS type, ISO, repo URLs, temp build IP

Customization

  • data/ks.pkrtpl.hcl – the VM installer config that includes name, timezone, dns, disk layout, user, key packages etc.
  • scripts/cleanup.sh – the sysprep for linux

How this works?

When the template is being created we can see two ISO’s mounted

CD 1: Rocky-9 ISO → the installer
CD 2: OEMDRV → contains ks.cfg, made by xorriso which is the answer file.

The order looks like this..

  1. Packer logs into vCenter and creates an empty VM.
  2. It attaches both CDs, powers the VM on, and presses Install in the boot menu.
  3. Rocky installs itself from ks.cfg, then reboots.
  4. Packer logs in over SSH as the awx user, using an automation key.
  5. It runs cleanup.sh: patch, then wipe anything that makes this VM unique.
  6. It shuts the VM down and converts it to the template tpl-rocky9.

Installing the Packages

Install Packer and Xorriso

wget -O- https://apt.releases.hashicorp.com/gpg | sudo gpg --dearmor -o /usr/share/keyrings/hashicorp-archive-keyring.gpg
echo "deb [signed-by=/usr/share/keyrings/hashicorp-archive-keyring.gpg] https://apt.releases.hashicorp.com $(lsb_release -cs) main" | sudo tee /etc/apt/sources.list.d/hashicorp.list
sudo apt update && sudo apt install -y packer xorriso

Create an Automation Key

We need an automation key and this format has to be in ed25519

wk@ansible:/etc/ansible/packer$
awk@ansible:/etc/ansible/packer$ ssh-keygen -t ed25519 -f ~/.ssh/automation_ed25519 -C "automation@lab" -N ""
Generating public/private ed25519 key pair.
Your identification has been saved in /home/awl/.ssh/automation_ed25519
Your public key has been saved in /home/awl/.ssh/automation_ed25519.pub
The key fingerprint is:
SHA256:lHK7r3CHxUiVo5qHNki7oKHQCqgk6yFeoFWtIGCq11k automation@lab
The key's randomart image is:
+--[ED25519 256]--+
|.. .. |
|+ . oo |
|o . . E =. . |
|.. +.+ =.+ |
|ooo.+o +S o |
|B++ o * .+ |
|X* o o.o+ . |
|O o . o o |
|.o ... |
+----[SHA256]-----+
awk@ansible:/etc/ansible/packer$ ls -l ~/.ssh/automation_ed25519*
-rw------- 1 awk awk 411 Oct 31 12:04 /home/awl/.ssh/automation_ed25519
-rw-r--r-- 1 awk awk 96 Oct 31 12:04 /home/awl/.ssh/automation_ed25519.pub
awk@ansible:/etc/ansible/packer$

Under vsphere.pkrvars.hcl , define the following

datacenter = "DC1"
cluster    = "Cluster01"
datastore  = "Datastore01"
folder     = "Templates"
network    = "VM-1611-mgmt"

rhel.pkr.hcl is the one build file shared by every flavour, here we need to put the automation key and vCenter variables needed.

packer {
  required_plugins {
    vsphere = {
      source  = "github.com/hashicorp/vsphere"
      version = ">= 1.4.0"
    }
  }
}

# login: from env.sh
variable "vcenter"  { default = env("VSPHERE_SERVER") }
variable "user"     { default = env("VSPHERE_USER") }
variable "password" {
  default   = env("VSPHERE_PASSWORD")
  sensitive = true
}

# where: from ../vsphere.pkrvars.hcl
variable "datacenter" {}
variable "cluster"    {}
variable "datastore"  {}
variable "folder"     {}
variable "network"    {}

# flavour: from rocky9.pkrvars.hcl, alma9.pkrvars.hcl ...
variable "vm_name"       {}
variable "guest_os_type" {}
variable "iso_path"      {}
variable "baseos_url"    {}
variable "appstream_url" {}
variable "build_ip"      {}

source "vsphere-iso" "linux" {
  vcenter_server      = var.vcenter
  username            = var.user
  password            = var.password
  insecure_connection = true
  datacenter          = var.datacenter
  cluster             = var.cluster
  datastore           = var.datastore
  folder              = var.folder

  vm_name              = var.vm_name
  guest_os_type        = var.guest_os_type
  firmware             = "efi"
  CPUs                 = 2
  RAM                  = 2048
  disk_controller_type = ["pvscsi"]
  storage {
    disk_size             = 20480
    disk_thin_provisioned = true
  }
  network_adapters {
    network      = var.network
    network_card = "vmxnet3"
  }

  iso_paths = [var.iso_path]
  cd_label  = "OEMDRV"
  cd_content = {
    "ks.cfg" = templatefile("${path.root}/data/ks.pkrtpl.hcl", {
      hostname      = var.vm_name
      build_ip      = var.build_ip
      baseos_url    = var.baseos_url
      appstream_url = var.appstream_url
      ssh_key       = trimspace(file(pathexpand("~/.ssh/automation_ed25519.pub")))
    })
  }
  boot_order   = "disk,cdrom"
  boot_wait    = "15s"
  boot_command = ["<up><enter>"]

  ssh_host             = var.build_ip
  ssh_username         = "awx"
  ssh_private_key_file = "~/.ssh/automation_ed25519"
  ssh_timeout          = "30m"
  shutdown_command     = "sudo shutdown -P now"

  convert_to_template = true
}

build {
  sources = ["source.vsphere-iso.linux"]

  provisioner "shell" {
    script = "scripts/cleanup.sh"
  }
}

Create alma9.pkrvars.hcl values file per flavour.

vm_name       = "tpl-alma9"
guest_os_type = "rhel9_64Guest"
iso_path      = "[nfs-iso] AlmaLinux-9-latest-x86_64-minimal.iso"
baseos_url    = "https://repo.almalinux.org/almalinux/9/BaseOS/x86_64/os/"
appstream_url = "https://repo.almalinux.org/almalinux/9/AppStream/x86_64/os/"
build_ip      = "172.16.11.250"

Create rocky9.pkrvars.hcl values file per flavour.

vm_name       = "tpl-rocky9"
guest_os_type = "rhel9_64Guest"
iso_path      = "[nfs-iso] Rocky-9-latest-x86_64-minimal.iso"
baseos_url    = "https://download.rockylinux.org/pub/rocky/9/BaseOS/x86_64/os/"
appstream_url = "https://download.rockylinux.org/pub/rocky/9/AppStream/x86_64/os/"
build_ip      = "172.16.11.250"

Create a clean-up script called cleanup.sh under scripts folder

#!/bin/bash
set -e
sudo dnf -y update
sudo dnf clean all
sudo truncate -s 0 /etc/machine-id
sudo rm -f /etc/ssh/ssh_host_*
sudo rm -f /etc/NetworkManager/system-connections/*.nmconnection

And the last file we need is ks.pkrtpl.hcl which is our kickstart file

text
url --url=${baseos_url}
repo --name=AppStream --baseurl=${appstream_url}
lang en_GB.UTF-8
keyboard gb
timezone Europe/London --utc
network --bootproto=static --ip=${build_ip} --netmask=255.255.255.0 --gateway=172.16.11.1 --nameserver=172.16.11.5 --activate --hostname=${hostname}
rootpw --lock
user --name=awx --groups=wheel --lock
sshkey --username=awx "${ssh_key}"
selinux --enforcing
firewall --enabled --ssh
zerombr
clearpart --all --initlabel
autopart --type=plain --nohome
bootloader --timeout=1
firstboot --disabled
services --enabled=vmtoolsd
reboot --eject

%packages
@^minimal-environment
open-vm-tools
perl
vim-enhanced
bind-utils
net-tools
tar
wget
%end

%post
echo 'awx ALL=(ALL) NOPASSWD:ALL' > /etc/sudoers.d/awx
chmod 440 /etc/sudoers.d/awx
%end

Then there is also the env.sh which has the vCenter credentials we used in Terraform

export VSPHERE_SERVER=vcenter.lab.internal
export VSPHERE_USER=svc-automation@vsphere.local
read -s -p "Password: " VSPHERE_PASSWORD; echo; export VSPHERE_PASSWORD

Now that we have all the required configurations in place we will progress with build.

To start the build, navigate to the following folder – cd /etc/ansible/packer/rhel and authenticate to vCenter.

source /etc/ansible/terraform/env.sh

Initialise packer on the folder using packer init .

packer init .

Validate the config using packer validate 

packer validate -var-file=../vsphere.pkrvars.hcl -var-file=rocky9.pkrvars.hcl .

Build the VM template using packer build command

packer build -force -on-error=ask -var-file=../vsphere.pkrvars.hcl -var-file=rocky9.pkrvars.hcl .

After about 10 mins our template creation is done and VM will shut

That’s how we do an unattended installation via packer

The template is now ready for VM builds.

Automating Ubuntu builds with Packer

Ubuntu gets its own folder structure so let’s a similar structure we used for Rhel . So, with Ubuntu, this follows a slightly different boot structure than rhel and it uses auto install than kickstart

This is the file structure I’ve used for Ubuntu

There are few files here and the customization files here are shared among all templates.

vSphere inventory

  • vsphere.pkrvars.hcl – where in vCenter: datacenter, cluster, datastore, folder, port group and its defined in root folder so nothing needs to be done.

Actual Build file

  • ubuntu.pkr.hcl – the build, one for all flavours: VM hardware (CPU, RAM, disk, NIC)

Variables

  • ubuntu2204.pkrvars.hcl – Ubuntu 2204 values: template name, OS type, ISO, repo URLs, temp build IP

Customization

  • data/user-data.pkrtpl.hcl – the VM installer config that includes name, timezone, dns, disk layout, user, key packages etc.
  • scripts/cleanup.sh – the sysprep for linux

ubuntu.pkr.hcl is the one build file shared by every ubuntu flavour, here we need to put the automation key and vCenter variables needed. This file is same as rhel..

packer {
  required_plugins {
    vsphere = {
      source  = "github.com/hashicorp/vsphere"
      version = ">= 1.4.0"
    }
  }
}

# login: from env.sh
variable "vcenter"  { default = env("VSPHERE_SERVER") }
variable "user"     { default = env("VSPHERE_USER") }
variable "password" {
  default   = env("VSPHERE_PASSWORD")
  sensitive = true
}

# where: from ../vsphere.pkrvars.hcl
variable "datacenter" {}
variable "cluster"    {}
variable "datastore"  {}
variable "folder"     {}
variable "network"    {}

# flavour: from rocky9.pkrvars.hcl, alma9.pkrvars.hcl ...
variable "vm_name"       {}
variable "guest_os_type" {}
variable "iso_path"      {}
variable "baseos_url"    {}
variable "appstream_url" {}
variable "build_ip"      {}

source "vsphere-iso" "linux" {
  vcenter_server      = var.vcenter
  username            = var.user
  password            = var.password
  insecure_connection = true
  datacenter          = var.datacenter
  cluster             = var.cluster
  datastore           = var.datastore
  folder              = var.folder

  vm_name              = var.vm_name
  guest_os_type        = var.guest_os_type
  firmware             = "efi"
  CPUs                 = 2
  RAM                  = 2048
  disk_controller_type = ["pvscsi"]
  storage {
    disk_size             = 20480
    disk_thin_provisioned = true
  }
  network_adapters {
    network      = var.network
    network_card = "vmxnet3"
  }

  iso_paths = [var.iso_path]
  cd_label  = "OEMDRV"
  cd_content = {
    "ks.cfg" = templatefile("${path.root}/data/ks.pkrtpl.hcl", {
      hostname      = var.vm_name
      build_ip      = var.build_ip
      baseos_url    = var.baseos_url
      appstream_url = var.appstream_url
      ssh_key       = trimspace(file(pathexpand("~/.ssh/automation_ed25519.pub")))
    })
  }
  boot_order   = "disk,cdrom"
  boot_wait    = "15s"
  boot_command = ["<up><enter>"]

  ssh_host             = var.build_ip
  ssh_username         = "awx"
  ssh_private_key_file = "~/.ssh/automation_ed25519"
  ssh_timeout          = "30m"
  shutdown_command     = "sudo shutdown -P now"

  convert_to_template = true
}

build {
  sources = ["source.vsphere-iso.linux"]

  provisioner "shell" {
    script = "scripts/cleanup.sh"
  }
}

Create ubuntu2204.pkrvars.hcl values file for Ubuntu 22.04

vm_name       = "tpl-ubuntu2204"
guest_os_type = "ubuntu64Guest"
iso_path      = "[nfs-iso] ubuntu-22.04.1-live-server-amd64.iso"
build_ip      = "172.16.11.250"

Create a clean-up script folder called cleanup.sh under scripts folder and this is different in Apt.

#!/bin/bash
set -e
sudo apt-get -o DPkg::Lock::Timeout=300 update
sudo apt-get -o DPkg::Lock::Timeout=300 -y upgrade
sudo apt-get -y autoremove
sudo apt-get clean

sudo tee /etc/systemd/system/regen-ssh-keys.service >/dev/null <<'UNIT'
[Unit]
Description=Regenerate SSH host keys
Before=ssh.service
ConditionPathExists=!/etc/ssh/ssh_host_ed25519_key
[Service]
Type=oneshot
ExecStart=/usr/bin/ssh-keygen -A
[Install]
WantedBy=multi-user.target
UNIT
sudo systemctl enable regen-ssh-keys.service

sudo cloud-init clean --logs
sudo touch /etc/cloud/cloud-init.disabled
sudo truncate -s 0 /etc/machine-id
sudo rm -f /var/lib/dbus/machine-id
sudo rm -f /etc/ssh/ssh_host_*
sudo rm -f /etc/netplan/*.yaml

Create user-data.pkrtpl.hcl under data folder and this is our kickstart or auto install versionn in Ubuntu

#cloud-config
autoinstall:
  version: 1
  locale: en_GB.UTF-8
  keyboard:
    layout: gb
  network:
    version: 2
    ethernets:
      nic0:
        match:
          name: "en*"
        addresses: [${build_ip}/24]
        routes:
          - to: default
            via: 172.16.11.1
        nameservers:
          addresses: [172.16.11.5]
  storage:
    layout:
      name: direct
  ssh:
    install-server: true
    allow-pw: false
  packages:
    - open-vm-tools
    - dnsutils
    - net-tools
    - wget
  early-commands:
    - systemctl stop ssh
  user-data:
    hostname: ${hostname}
    timezone: Europe/London
    disable_root: true
    users:
      - name: awx
        groups: [sudo]
        sudo: "ALL=(ALL) NOPASSWD:ALL"
        shell: /bin/bash
        lock_passwd: true
        ssh_authorized_keys:
          - ${ssh_key}

And the last file we need is ubuntu.pkr.hcl

packer {
  required_plugins {
    vsphere = {
      source  = "github.com/hashicorp/vsphere"
      version = ">= 1.4.0"
    }
  }
}

# login: from env.sh
variable "vcenter"  { default = env("VSPHERE_SERVER") }
variable "user"     { default = env("VSPHERE_USER") }
variable "password" {
  default   = env("VSPHERE_PASSWORD")
  sensitive = true
}

# where: from ../vsphere.pkrvars.hcl
variable "datacenter" {}
variable "cluster"    {}
variable "datastore"  {}
variable "folder"     {}
variable "network"    {}

# version: from ubuntu2204.pkrvars.hcl ...
variable "vm_name"       {}
variable "guest_os_type" {}
variable "iso_path"      {}
variable "build_ip"      {}

source "vsphere-iso" "ubuntu" {
  vcenter_server      = var.vcenter
  username            = var.user
  password            = var.password
  insecure_connection = true
  datacenter          = var.datacenter
  cluster             = var.cluster
  datastore           = var.datastore
  folder              = var.folder

  vm_name              = var.vm_name
  guest_os_type        = var.guest_os_type
  firmware             = "efi"
  CPUs                 = 2
  RAM                  = 2048
  disk_controller_type = ["pvscsi"]
  storage {
    disk_size             = 20480
    disk_thin_provisioned = true
  }
  network_adapters {
    network      = var.network
    network_card = "vmxnet3"
  }

  iso_paths = [var.iso_path]
  cd_label  = "cidata"
  cd_content = {
    "meta-data" = ""
    "user-data" = templatefile("${path.root}/data/user-data.pkrtpl.hcl", {
      hostname = var.vm_name
      build_ip = var.build_ip
      ssh_key  = trimspace(file(pathexpand("~/.ssh/automation_ed25519.pub")))
    })
  }
  boot_order = "disk,cdrom"
  boot_wait  = "10s"
  boot_command = [
    "c<wait>",
    "linux /casper/vmlinuz --- autoinstall ds=nocloud<enter><wait>",
    "initrd /casper/initrd<enter><wait>",
    "boot<enter>"
  ]

  ssh_host             = var.build_ip
  ssh_username         = "awx"
  ssh_private_key_file = "~/.ssh/automation_ed25519"
  ssh_timeout          = "30m"
  shutdown_command     = "sudo shutdown -P now"

  convert_to_template = true
}

build {
  sources = ["source.vsphere-iso.ubuntu"]

  provisioner "shell" {
    script = "scripts/cleanup.sh"
  }
}

Now that we have all the required configurations in place we will progress with build.

To start the build, navigate to the following folder – cd /etc/ansible/packer/ubuntu and authenticate to vCenter.

source /etc/ansible/terraform/env.sh

Authenticate to vCenter.

source /etc/ansible/terraform/env.sh

Initialise packer on the folder using packer init .

packer init .

Validate the config using packer validate 

packer validate -var-file=../vsphere.pkrvars.hcl -var-file=ubuntu2204.pkrvars.hcl .

Build the VM template using packer build command

packer build -force -on-error=ask -var-file=../vsphere.pkrvars.hcl -var-file=ubuntu2204.pkrvars.hcl .

After about 10 mins our template creation is done and VM will shut

That’s how we do an unattended installation via packer

The template is now ready for VM builds.

Creating Dev VM’s via Terraform

Our template is now ready, so let’s just build few VM’s from it and see how it goes in our Dev Cluster

This is actually pretty easy so again all we need to touch is just 1 file terraform.tfvars and we just need to add the new VM’s and just say place it in the folder so let’s give it a try.

Also, we need to do one more edit inside our data.tf to add the new OS we just added to template via packr so Terraform is aware of it. Left side: a short name you choose, and it must be unique. It’s what you type in tfvars: os = "ubuntu22". Right side: must match the template name in vCenter exactly, the same as vm_name in the Packer values file.

Navigate to /etc/ansible/terraform/dev$

Let’s re-run the terraform plan and it shows exactly what Terraform would do, without changing anything. It says 2 VM’s are going to be added

Run terraform plan -out=tfplan to do a save of our plan we are just about to deploy

Make the changes by running terraform apply tfplan.

All the VM’s are created to their folders

So that’s it about terraform and packer.

In the next series we will add a new Ubuntu OS to the packer and make all changes needed and deploy a VM and this is very critical task so let’s see how it’s done

(Visited 10 times, 2 visits today)

By Ash Thomas

Ash Thomas is a seasoned IT professional with extensive experience as a technical expert, complemented by a keen interest in blockchain technology.

vExpert community knowledge — protected from AI training, bots, and automated harvesting. | BlogArise by Themeansar.