Manage RDMs during OS Upgrades and VM Lifecycle

As technology evolves, upgrading your virtual machines (VMs) to newer operating systems (OS) such as Windows Server 2016 and 2019 is essential. However, managing Raw Device Mappings (RDMs) during these upgrades can be challenging, especially when dealing with VMs nearing their end of life or experiencing issues like the blue screen of death. In this blog post, we’ll explore a strategic approach to manage RDMs effectively, ensuring a smooth transition and minimizing the risk of data loss.

Approach 1 – The Centralized RDM-Mapping-VM Pointer File Management

To address this issue, we can use three routes, the first one ensures a seamless upgrade process and simplifying VM lifecycle management if handled well.

1. Centralize RDM Pointer File Management: Use a dedicated datastore or a VM to manage RDM pointer files. This way, the RDM pointers are not tied to any specific production VM, reducing the risk of data loss or configuration errors.
2. Create a Dedicated RDM Mapping VM: Set up a VM solely for the purpose of housing RDM pointer files eg: rdm_mapipng_vm. This VM will not be involved in production but will act as the central point of management for the RDMs.
3. Documentation: Ensure regular backups of the VM that holds the RDM pointer files. Maintain detailed documentation of the RDM mappings and their associations with production VMs to avoid any confusion during future upgrades or decommissioning.

Example Configuration in .vmx File

For the dedicated RDM Mapping VM (rdm-mapping-vm):

# RDM disks
scsi0:1.present = "true"
scsi0:1.fileName = "rdm-mapping-vm/rdm_pointer_file1.vmdk"
scsi0:1.deviceType = "rdmp"

scsi0:2.present = "true"
scsi0:2.fileName = "rdm-mapping-vm/rdm_pointer_file2.vmdk"
scsi0:2.deviceType = "rdmp"

For production VMs (virt-fil-03, virt-fil-04), update the .vmx files to reference the rdm-mapping-vm:

# RDM disks
scsi0:1.present = "true"
scsi0:1.fileName = "rdm-mapping-vm/rdm_pointer_file1.vmdk"
scsi0:1.deviceType = "rdmp"

scsi0:2.present = "true"
scsi0:2.fileName = "rdm-mapping-vm/rdm_pointer_file2.vmdk"
scsi0:2.deviceType = "rdmp"

Benefits

• Decouples RDMs from Production VMs: Reduces the risk of data loss or configuration errors during VM upgrades or decommissioning.
• Simplifies Management: Provides a single point of management for all RDM pointer files.
• Future-Proofs Environment: Eases the transition to future OS versions and infrastructure changes without disrupting production.

Approach 2 – Steps to Migrate Physical RDM Files From One Datastore to Another

This is a scenario where the storage array holding that datastore is being decommissioned so we need the rdm’s to be moved to another datastore. This write up assumes data has copied from one storage array to another storage array using some block-based copy and a new lun with the data is being mapped to the ESXI hosts.

1. Power Off the Virtual Machines in the cluster:
   • Ensure the virtual machine that uses the RDM is powered off to avoid any data corruption.
2. Remove the RDM from the Virtual Machine:
   • In the vSphere Client, right-click the virtual machine and select Edit Settings.
   • Locate the RDM disk, select it, and click Remove. Make sure you are removing the mapping and not the actual disk.
3. Identify the new RDM LUN:
   • Take note of the RDM LUN details (e.g., LUN ID and target).
4. Remove the RDM Mapping File from the Original Datastore:
   • Manually delete the RDM mapping file from the original datastore using the datastore browser.
5. Create a New RDM Mapping File in the Target Datastore:
   • Go to the target datastore where you want to move the RDM mapping file.
   • Right-click the datastore and select New Raw Device Mapping.
   • Follow the prompts to select the same LUN and create the RDM mapping file in the new datastore.
6. Re-Add the RDM to the Virtual Machine:
   • Go back to the virtual machine settings, click Add to add a new disk.
   • Select Raw Device Mapping and choose the newly created RDM mapping file from the target datastore.
7. Power On the Virtual Machine:
   • Power on the virtual machine and verify that it can access the RDM without any issues.

Approach 3 – Steps to Migrate OS Disk that Failed with Primary RDM Files

By keeping the old datastore location for the RDM pointer files and swapping the OS drive, you can quickly transition to the new OS without moving the RDM pointer files so it will still be contained within the primary VM within the cluster. This approach minimizes the risk of data corruption and simplifies the process.

The trick here is, first, copy the new VM OS disk (path_to_new_os_drive.vmdk) from the new OS location to the old datastore location, where the broken VM or primary RDM pointer holder is located. Then, rename the existing OS disk in the old datastore location to old_os_drive.vmdk. After that, rename the newly copied OS disk to new_os_drive.vmdk, giving it the same name as the old OS disk. Since the new OS disk has the same name as the old one, you won’t need to make any changes to the VMX configuration file. Once the VM powers up successfully, you can delete old_os_drive.vmdk as it becomes redundant. Since Storage vMotion (SvMotion) won’t work in this case, the manual copy at the datastore level is the way to go.

Set Up a New OS and Use Existing RDM Pointer Files

1. Set Up a New Virtual Machine with Windows 2016:
   • Create a new virtual machine (VM) and install Windows Server 2016 on it.
2. Power Off the Old Virtual Machine
   • ie : that is broken or that has an old OS
3. Swap the OS Drive on the Old Virtual Machines:
   • Edit the .vmx file of each old VM to point to the new Windows 2016 OS drive. For example:

ide0:0.fileName = "path_to_new_os_drive.vmdk"
ide0:0.present = "TRUE"

Ensure Existing RDM Pointer Files Remain:

• In the .vmx file of each old VM, keep the existing RDM pointer file paths. For example:

# OS drive
ide0:0.fileName = "path_to_new_os_drive.vmdk"
ide0:0.present = "TRUE"

# RDM disks
scsi0:1.present = "true"
scsi0:1.fileName = "path_to_rdm_pointer_file1.vmdk"
scsi0:1.deviceType = "rdmp"

scsi0:2.present = "true"
scsi0:2.fileName = "path_to_rdm_pointer_file2.vmdk"
scsi0:2.deviceType = "rdmp"

Power On the Virtual Machines:

• Power on the old VMs with the new OS drive and verify that they can access the RDM disks using the original datastore location.

Example Configuration in .vmx File for virt-fil-01:

# OS drive
ide0:0.fileName = "path_to_new_os_drive.vmdk"
ide0:0.present = "TRUE"

# RDM disks
scsi0:1.present = "true"
scsi0:1.fileName = "path_to_rdm_pointer_file1.vmdk"
scsi0:1.deviceType = "rdmp"

scsi0:2.present = "true"
scsi0:2.fileName = "path_to_rdm_pointer_file2.vmdk"
scsi0:2.deviceType = "rdmp"

Example Configuration in .vmx File for virt-fil-02:

# OS drive
ide0:0.fileName = "path_to_new_os_drive.vmdk"
ide0:0.present = "TRUE"

# RDM disks
scsi0:1.present = "true"
scsi0:1.fileName = "path_to_rdm_pointer_file1.vmdk"
scsi0:1.deviceType = "rdmp"

scsi0:2.present = "true"
scsi0:2.fileName = "path_to_rdm_pointer_file2.vmdk"
scsi0:2.deviceType = "rdmp"

You can now delete the new VM you created as it’s no longer required.

Consideration: Avoiding Multi-Write Issues with Clustering

While vSphere is effectively a cluster, it’s important to note that adding Microsoft clustering on top of vSphere is not recommended. The multi-write approach in such setups can lead to data corruption and is generally considered a bad design. Ensuring that your RDM setup does not introduce multi-write conflicts is crucial to maintaining a stable environment.

Additional Consideration: Clustering VMDKs

Using clustered VMDKs can also pose challenges, as these setups are prone to data corruption. It is advisable to avoid clustering VMDKs and rely on alternative solutions that do not introduce multi-write scenarios.

Moving Away from Physical RDMs

• Deprecation by VMware: There are talks that VMware may be deprecating support for physical Raw Device Mappings (RDMs). Moving away from using physical RDMs helps in future-proofing your environment such as a VM migration to VCF’s.
• Virtual RDMs and Alternatives: Consider using virtual RDMs or migrating to VMware vSAN or other storage solutions that provide the necessary clustering capabilities without the complexities and risks associated with physical RDMs.
• Enhanced Flexibility: By moving away from physical RDMs, you gain increased flexibility in managing storage, as well as simplifying backup and recovery processes.

Advantages of Virtual RDMs

1. Simplified Management:
   • Virtual RDMs allow for easier management of storage devices within the virtual infrastructure. They enable the use of VMware’s advanced features such as VM snapshots and Storage vMotion, which are not available with physical RDMs.
2. Improved Data Protection:
   • With virtual RDMs, you can leverage VM-level backups and replication solutions, enhancing the overall data protection strategy. This makes it easier to implement consistent backup policies and ensures better disaster recovery preparedness.
3. Enhanced Scalability:
   • Virtual RDMs offer greater scalability by allowing you to dynamically allocate storage resources based on workload demands. This flexibility helps in optimizing storage utilization and supports the growth of your virtual environment.
4. Increased Mobility:
   • Virtual RDMs enable VMs to be easily migrated between hosts without disrupting operations. This improves the overall mobility of your virtual machines, facilitating maintenance and reducing downtime