This blog will walk you through the procedure to configure multipathing in Linux using device mapper multipathing which is the native multipathing in Linux. Multipathing combines multiple I/O paths between servers and storage and creates a single device at the OS. As part of this install, I’ve already configured my ISCSI target server on a Microsoft Server and we will be attaching it to that ISCSI san.
1. Install ISCSI Initiator package.
|
1 |
<span class="line"><span style="color: #BABED8">yum </span><span style="color: #89DDFF">-</span><span style="color: #BABED8">y install iscsi</span><span style="color: #89DDFF">-</span><span style="color: #BABED8">initiator</span><span style="color: #89DDFF">-</span><span style="color: #BABED8">utils</span><span style="color: #89DDFF">.</span><span style="color: #BABED8">x86_64</span></span> |
2. Check installed HBA modules with lspci
|
1 2 |
<span class="line"><span style="color: #BABED8">yum install </span><span style="color: #89DDFF">-</span><span style="color: #BABED8">y pciutils</span></span> <span class="line"><span style="color: #BABED8">lspci </span><span style="color: #89DDFF">|</span><span style="color: #BABED8">grep </span><span style="color: #89DDFF">-</span><span style="color: #BABED8">i fibr</span></span> |
2. Start ISCSI Initiator service.
|
1 |
<span class="line"><span style="color: #BABED8">systemctl start iscsid</span><span style="color: #89DDFF">.</span><span style="color: #BABED8">service</span></span> |
3. Get ISCSI ID Info
|
1 2 |
<span class="line"><span style="color: #BABED8">[root@virt</span><span style="color: #89DDFF">-</span><span style="color: #BABED8">scale</span><span style="color: #89DDFF">-</span><span style="color: #F78C6C">03</span><span style="color: #BABED8"> </span><span style="color: #89DDFF">~</span><span style="color: #BABED8">]# cat </span><span style="color: #89DDFF">/</span><span style="color: #BABED8">etc</span><span style="color: #89DDFF">/</span><span style="color: #BABED8">iscsi</span><span style="color: #89DDFF">/</span><span style="color: #BABED8">initiatorname</span><span style="color: #89DDFF">.</span><span style="color: #BABED8">iscsi</span></span> <span class="line"><span style="color: #BABED8">InitiatorName</span><span style="color: #89DDFF">=</span><span style="color: #BABED8">iqn</span><span style="color: #89DDFF">.</span><span style="color: #F78C6C">1994</span><span style="color: #89DDFF">-</span><span style="color: #F78C6C">05</span><span style="color: #89DDFF">.</span><span style="color: #BABED8">com</span><span style="color: #89DDFF">.</span><span style="color: #BABED8">redhat:7f598a789f</span></span> |
4. Add the IQN to the SAN Array
4. Enable initiators to find targets (logical drives mapped to the host) by
specifying iSCSI host port addresses / Login to the ISCSI Server – Path A
|
1 2 3 |
<span class="line"><span style="color: #BABED8">root@virt</span><span style="color: #89DDFF">-</span><span style="color: #BABED8">scale</span><span style="color: #89DDFF">-</span><span style="color: #F78C6C">04</span><span style="color: #BABED8"> </span><span style="color: #89DDFF">~</span><span style="color: #BABED8">]# iscsiadm </span><span style="color: #89DDFF">-</span><span style="color: #BABED8">m discovery </span><span style="color: #89DDFF">-</span><span style="color: #BABED8">t st </span><span style="color: #89DDFF">-</span><span style="color: #BABED8">p </span><span style="color: #F78C6C">172.27</span><span style="color: #89DDFF">.</span><span style="color: #F78C6C">13.100</span></span> <span class="line"><span style="color: #F78C6C">172.27</span><span style="color: #89DDFF">.</span><span style="color: #F78C6C">13.100</span><span style="color: #BABED8">:</span><span style="color: #F78C6C">3260</span><span style="color: #89DDFF">,</span><span style="color: #F78C6C">1</span><span style="color: #BABED8"> iqn</span><span style="color: #89DDFF">.</span><span style="color: #F78C6C">1991</span><span style="color: #89DDFF">-</span><span style="color: #F78C6C">05</span><span style="color: #89DDFF">.</span><span style="color: #BABED8">com</span><span style="color: #89DDFF">.</span><span style="color: #BABED8">microsoft:win</span><span style="color: #89DDFF">-</span><span style="color: #BABED8">opjkarsc1hv</span><span style="color: #89DDFF">-</span><span style="color: #BABED8">esx</span><span style="color: #89DDFF">-</span><span style="color: #BABED8">san</span><span style="color: #89DDFF">-</span><span style="color: #BABED8">target</span></span> <span class="line"><span style="color: #F78C6C">172.27</span><span style="color: #89DDFF">.</span><span style="color: #F78C6C">13.101</span><span style="color: #BABED8">:</span><span style="color: #F78C6C">3260</span><span style="color: #89DDFF">,</span><span style="color: #F78C6C">1</span><span style="color: #BABED8"> iqn</span><span style="color: #89DDFF">.</span><span style="color: #F78C6C">1991</span><span style="color: #89DDFF">-</span><span style="color: #F78C6C">05</span><span style="color: #89DDFF">.</span><span style="color: #BABED8">com</span><span style="color: #89DDFF">.</span><span style="color: #BABED8">microsoft:win</span><span style="color: #89DDFF">-</span><span style="color: #BABED8">opjkarsc1hv</span><span style="color: #89DDFF">-</span><span style="color: #BABED8">esx</span><span style="color: #89DDFF">-</span><span style="color: #BABED8">san</span><span style="color: #89DDFF">-</span><span style="color: #BABED8">target</span></span> |
5. Enable initiators to find targets (logical drives mapped to the host) by
specifying iSCSI host port addresses/ Login to the ISCSI Server – Path B
|
1 2 3 |
<span class="line"><span style="color: #BABED8">[root@virt</span><span style="color: #89DDFF">-</span><span style="color: #BABED8">scale</span><span style="color: #89DDFF">-</span><span style="color: #F78C6C">04</span><span style="color: #BABED8"> </span><span style="color: #89DDFF">~</span><span style="color: #BABED8">]# iscsiadm </span><span style="color: #89DDFF">-</span><span style="color: #BABED8">m discovery </span><span style="color: #89DDFF">-</span><span style="color: #BABED8">t st </span><span style="color: #89DDFF">-</span><span style="color: #BABED8">p </span><span style="color: #F78C6C">172.27</span><span style="color: #89DDFF">.</span><span style="color: #F78C6C">13.101</span></span> <span class="line"><span style="color: #F78C6C">172.27</span><span style="color: #89DDFF">.</span><span style="color: #F78C6C">13.101</span><span style="color: #BABED8">:</span><span style="color: #F78C6C">3260</span><span style="color: #89DDFF">,</span><span style="color: #F78C6C">1</span><span style="color: #BABED8"> iqn</span><span style="color: #89DDFF">.</span><span style="color: #F78C6C">1991</span><span style="color: #89DDFF">-</span><span style="color: #F78C6C">05</span><span style="color: #89DDFF">.</span><span style="color: #BABED8">com</span><span style="color: #89DDFF">.</span><span style="color: #BABED8">microsoft:win</span><span style="color: #89DDFF">-</span><span style="color: #BABED8">opjkarsc1hv</span><span style="color: #89DDFF">-</span><span style="color: #BABED8">esx</span><span style="color: #89DDFF">-</span><span style="color: #BABED8">san</span><span style="color: #89DDFF">-</span><span style="color: #BABED8">target</span></span> <span class="line"><span style="color: #F78C6C">172.27</span><span style="color: #89DDFF">.</span><span style="color: #F78C6C">13.100</span><span style="color: #BABED8">:</span><span style="color: #F78C6C">3260</span><span style="color: #89DDFF">,</span><span style="color: #F78C6C">1</span><span style="color: #BABED8"> iqn</span><span style="color: #89DDFF">.</span><span style="color: #F78C6C">1991</span><span style="color: #89DDFF">-</span><span style="color: #F78C6C">05</span><span style="color: #89DDFF">.</span><span style="color: #BABED8">com</span><span style="color: #89DDFF">.</span><span style="color: #BABED8">microsoft:win</span><span style="color: #89DDFF">-</span><span style="color: #BABED8">opjkarsc1hv</span><span style="color: #89DDFF">-</span><span style="color: #BABED8">esx</span><span style="color: #89DDFF">-</span><span style="color: #BABED8">san</span><span style="color: #89DDFF">-</span><span style="color: #BABED8">target</span></span> |
6. Login to the iSCSI targets:
root@virt-scale-04 ~]# iscsiadm -m node -T iqn.1991-05.com.microsoft:win-opjkarsc1hv-esx-san-target -P 172.27.13.100 -l
7. Install Device Mapper Multipath package.
Install the device-mapper-multipath package.
|
1 |
<span class="line"><span style="color: #BABED8">[root@virt</span><span style="color: #89DDFF">-</span><span style="color: #BABED8">scale</span><span style="color: #89DDFF">-</span><span style="color: #F78C6C">04</span><span style="color: #BABED8"> </span><span style="color: #89DDFF">~</span><span style="color: #BABED8">]# yum </span><span style="color: #89DDFF">-</span><span style="color: #BABED8">y install device</span><span style="color: #89DDFF">-</span><span style="color: #BABED8">mapper</span><span style="color: #89DDFF">-</span><span style="color: #BABED8">multipath</span></span> |
8. Set the multipath daemon starts on bootup
|
1 |
<span class="line"><span style="color: #BABED8">chkconfig multipathd on</span></span> |
9. Basic Configuration of Linux Device Mapper Multipathing
Take a backup of the configuration file /etc/multipath.conf file and edit the configuration file to ensure you have the following entries
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 |
<span class="line"><span style="color: #BABED8">defaults </span><span style="color: #89DDFF">{</span></span> <span class="line"><span style="color: #BABED8">user_friendly_names</span><span style="color: #F07178"> </span><span style="color: #BABED8">yes</span></span> <span class="line"><span style="color: #BABED8">find_multipaths</span><span style="color: #F07178"> </span><span style="color: #BABED8">yes</span></span> <span class="line"><span style="color: #89DDFF">}</span></span> <span class="line"><span style="color: #BABED8">blacklist </span><span style="color: #89DDFF">{</span></span> <span class="line"><span style="color: #BABED8">devnode</span><span style="color: #F07178"> </span><span style="color: #89DDFF">"</span><span style="color: #C3E88D">^(ram|raw|loop|fd|md|dm-|sr|scd|st)[0-9]</span><span style="color: #89DDFF">"</span><span style="color: #F07178"> </span><span style="color: #BABED8">devnode</span><span style="color: #F07178"> </span><span style="color: #89DDFF">"</span><span style="color: #C3E88D">^hd[a-z]</span><span style="color: #89DDFF">"</span><span style="color: #F07178"> </span><span style="color: #BABED8">devnode</span><span style="color: #F07178"> </span><span style="color: #89DDFF">"</span><span style="color: #C3E88D">^cciss!c[0-9]d[0-9]</span><span style="color: #89DDFF">"</span></span> <span class="line"><span style="color: #89DDFF">}</span></span> <span class="line"><span style="color: #BABED8">multipaths </span><span style="color: #89DDFF">{</span></span> <span class="line"><span style="color: #F07178"> </span><span style="color: #BABED8">multipath</span><span style="color: #F07178"> </span><span style="color: #89DDFF">{</span></span> <span class="line"><span style="color: #F07178"> </span><span style="color: #BABED8">wwid</span><span style="color: #F07178"> 360003</span><span style="color: #BABED8">ff44dc75adc95698676b2fbf9a0</span></span> <span class="line"><span style="color: #F07178"> </span><span style="color: #BABED8">alias</span><span style="color: #F07178"> </span><span style="color: #BABED8">nsd</span><span style="color: #89DDFF">-</span><span style="color: #F78C6C">01</span></span> <span class="line"><span style="color: #F07178"> </span><span style="color: #89DDFF">}</span></span> <span class="line"></span> <span class="line"></span> <span class="line"><span style="color: #F07178"> </span><span style="color: #BABED8">multipath</span><span style="color: #F07178"> </span><span style="color: #89DDFF">{</span></span> <span class="line"><span style="color: #F07178"> </span><span style="color: #BABED8">wwid</span><span style="color: #F07178"> 360003</span><span style="color: #BABED8">ff44dc75adcbddbaa76e5d90368</span></span> <span class="line"><span style="color: #F07178"> </span><span style="color: #BABED8">alias</span><span style="color: #F07178"> </span><span style="color: #BABED8">nsd</span><span style="color: #89DDFF">-</span><span style="color: #F78C6C">02</span></span> <span class="line"><span style="color: #F07178"> </span><span style="color: #89DDFF">}</span></span> <span class="line"></span> <span class="line"><span style="color: #F07178"> </span><span style="color: #BABED8">multipath</span><span style="color: #F07178"> </span><span style="color: #89DDFF">{</span></span> <span class="line"><span style="color: #F07178"> </span><span style="color: #BABED8">wwid</span><span style="color: #F07178"> 360003</span><span style="color: #BABED8">ff44dc75adc92b0a3c4d0b82d17</span></span> <span class="line"><span style="color: #F07178"> </span><span style="color: #BABED8">alias</span><span style="color: #F07178"> </span><span style="color: #BABED8">nsd</span><span style="color: #89DDFF">-</span><span style="color: #F78C6C">03</span></span> <span class="line"><span style="color: #F07178"> </span><span style="color: #89DDFF">}</span></span> <span class="line"></span> <span class="line"><span style="color: #89DDFF">}</span></span> |
The blacklist includes the devices which are not to be configured for Multipathing. Typically our OS installed disk is /dev/sd so the first entry in the blacklist will exclude it.
10. Start and Enable multipath daemons.
|
1 |
<span class="line"><span style="color: #BABED8">[root@virt</span><span style="color: #89DDFF">-</span><span style="color: #BABED8">scale</span><span style="color: #89DDFF">-</span><span style="color: #F78C6C">04</span><span style="color: #BABED8"> </span><span style="color: #89DDFF">~</span><span style="color: #BABED8">]# systemctl start multipathd</span></span> |
Enable the multipath service to start on boot.
|
1 |
<span class="line"><span style="color: #BABED8">[root@virt</span><span style="color: #89DDFF">-</span><span style="color: #BABED8">scale</span><span style="color: #89DDFF">-</span><span style="color: #F78C6C">04</span><span style="color: #BABED8"> </span><span style="color: #89DDFF">~</span><span style="color: #BABED8">]# systemctl enable multipathd</span></span> |
Check the status of the multipath service
|
1 |
<span class="line"><span style="color: #BABED8">[root@virt</span><span style="color: #89DDFF">-</span><span style="color: #BABED8">scale</span><span style="color: #89DDFF">-</span><span style="color: #F78C6C">04</span><span style="color: #BABED8"> </span><span style="color: #89DDFF">~</span><span style="color: #BABED8">]# systemctl status multipathd</span></span> |
11. Check multipathing status.
Print the status of multipath devices using the command multipath -ll
12 Check if the device alias is appearing correctly
root@virt-scale-04 etc]# ll /dev/mapper/
12 Check multipathing IO by disabling a path on san
13 How to find the WWID of storage disk including a multipath device?
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 |
<span class="line"><span style="color: #F6F6F4">root@lnx03 doc]# lsscsi </span><span style="color: #F286C4">-</span><span style="color: #F6F6F4">i</span></span> <span class="line"><span style="color: #F6F6F4">[</span><span style="color: #BF9EEE">3</span><span style="color: #F6F6F4">:</span><span style="color: #BF9EEE">0</span><span style="color: #F6F6F4">:</span><span style="color: #BF9EEE">0</span><span style="color: #F6F6F4">:</span><span style="color: #BF9EEE">0</span><span style="color: #F6F6F4">] cd</span><span style="color: #F286C4">/</span><span style="color: #F6F6F4">dvd NECVMWar VMware SATA CD01 </span><span style="color: #BF9EEE">1.00</span><span style="color: #F6F6F4"> </span><span style="color: #F286C4">/</span><span style="color: #F6F6F4">dev</span><span style="color: #F286C4">/</span><span style="color: #F6F6F4">sr0 </span><span style="color: #F286C4">-</span></span> <span class="line"><span style="color: #F6F6F4">[</span><span style="color: #BF9EEE">32</span><span style="color: #F6F6F4">:</span><span style="color: #BF9EEE">0</span><span style="color: #F6F6F4">:</span><span style="color: #BF9EEE">0</span><span style="color: #F6F6F4">:</span><span style="color: #BF9EEE">0</span><span style="color: #F6F6F4">] disk LIO</span><span style="color: #F286C4">-</span><span style="color: #F6F6F4">ORG disk01 </span><span style="color: #BF9EEE">4.0</span><span style="color: #F6F6F4"> </span><span style="color: #F286C4">/</span><span style="color: #F6F6F4">dev</span><span style="color: #F286C4">/</span><span style="color: #F6F6F4">sda 360014053c7523e7faeb41f8bb8b107db</span></span> <span class="line"><span style="color: #F6F6F4">[</span><span style="color: #BF9EEE">33</span><span style="color: #F6F6F4">:</span><span style="color: #BF9EEE">0</span><span style="color: #F6F6F4">:</span><span style="color: #BF9EEE">0</span><span style="color: #F6F6F4">:</span><span style="color: #BF9EEE">1</span><span style="color: #F6F6F4">] disk LIO</span><span style="color: #F286C4">-</span><span style="color: #F6F6F4">ORG disk02 </span><span style="color: #BF9EEE">4.0</span><span style="color: #F6F6F4"> </span><span style="color: #F286C4">/</span><span style="color: #F6F6F4">dev</span><span style="color: #F286C4">/</span><span style="color: #F6F6F4">sdg 36001405a8213c3d09fb4f3992ea666fd</span></span> <span class="line"><span style="color: #F6F6F4">[</span><span style="color: #BF9EEE">33</span><span style="color: #F6F6F4">:</span><span style="color: #BF9EEE">0</span><span style="color: #F6F6F4">:</span><span style="color: #BF9EEE">0</span><span style="color: #F6F6F4">:</span><span style="color: #BF9EEE">2</span><span style="color: #F6F6F4">] disk LIO</span><span style="color: #F286C4">-</span><span style="color: #F6F6F4">ORG disk03 </span><span style="color: #BF9EEE">4.0</span><span style="color: #F6F6F4"> </span><span style="color: #F286C4">/</span><span style="color: #F6F6F4">dev</span><span style="color: #F286C4">/</span><span style="color: #F6F6F4">sdh 360014056d147dfeeca44034937600170</span></span> <span class="line"><span style="color: #F6F6F4">[</span><span style="color: #BF9EEE">33</span><span style="color: #F6F6F4">:</span><span style="color: #BF9EEE">0</span><span style="color: #F6F6F4">:</span><span style="color: #BF9EEE">0</span><span style="color: #F6F6F4">:</span><span style="color: #BF9EEE">3</span><span style="color: #F6F6F4">] disk LIO</span><span style="color: #F286C4">-</span><span style="color: #F6F6F4">ORG disk04 </span><span style="color: #BF9EEE">4.0</span><span style="color: #F6F6F4"> </span><span style="color: #F286C4">/</span><span style="color: #F6F6F4">dev</span><span style="color: #F286C4">/</span><span style="color: #F6F6F4">sdf 3600140540228bfc4b894a0b8a09a1f7a</span></span> <span class="line"></span> <span class="line"><span style="color: #F6F6F4">or </span></span> <span class="line"></span> <span class="line"><span style="color: #F6F6F4">[root@lnx03 doc]# sg_inq </span><span style="color: #F286C4">-</span><span style="color: #F6F6F4">p </span><span style="color: #BF9EEE">0x83</span><span style="color: #F6F6F4"> </span><span style="color: #F286C4">/</span><span style="color: #F6F6F4">dev</span><span style="color: #F286C4">/</span><span style="color: #F6F6F4">sda </span><span style="color: #F286C4">|</span><span style="color: #F6F6F4"> grep </span><span style="color: #DEE492">"</span><span style="color: #E7EE98">designator_type: NAA</span><span style="color: #DEE492">"</span><span style="color: #F6F6F4"> </span><span style="color: #F286C4">-</span><span style="color: #F6F6F4">A </span><span style="color: #BF9EEE">5</span></span> <span class="line"><span style="color: #F6F6F4"> designator_type: NAA, code_set: Binary</span></span> <span class="line"><span style="color: #F6F6F4"> associated </span><span style="color: #F286C4">with</span><span style="color: #F6F6F4"> the Addressed logical unit</span></span> <span class="line"><span style="color: #F6F6F4"> NAA </span><span style="color: #BF9EEE">6</span><span style="color: #F6F6F4">, IEEE Company_id: </span><span style="color: #BF9EEE">0x1405</span></span> <span class="line"><span style="color: #F6F6F4"> Vendor Specific Identifier: </span><span style="color: #BF9EEE">0x3c7523e7f</span></span> <span class="line"><span style="color: #F6F6F4"> Vendor Specific Identifier Extension: </span><span style="color: #BF9EEE">0xaeb41f8bb8b107db</span></span> <span class="line"><span style="color: #F6F6F4"> [</span><span style="color: #BF9EEE">0x60014053c7523e7faeb41f8bb8b107db</span><span style="color: #F6F6F4">]</span></span> <span class="line"><span style="color: #F6F6F4">or</span></span> <span class="line"></span> <span class="line"><span style="color: #F6F6F4"># scsi_id </span><span style="color: #F286C4">-</span><span style="color: #F6F6F4">g </span><span style="color: #F286C4">-</span><span style="color: #F6F6F4">u </span><span style="color: #F286C4">/</span><span style="color: #F6F6F4">dev</span><span style="color: #F286C4">/</span><span style="color: #F6F6F4">sdh</span></span> <span class="line"><span style="color: #F6F6F4">36000097123456789abcdeffedcba9876</span></span> |
(Visited 36 times, 1 visits today)