veritas: Disk mirroring using Veritas Volume Manager

This paper will present a short introduction to mirroring two disks using Veritas Volume Manager (VxVM)

In this case you will see that two separate set of Storage Array disks can be mirrored across a single disk group without interruptions.

The following vxprint shows the contents of disk group dgprdnfs. It has a number of small sets of disks and from a single controller/storage c3's . The disks a set across three sets of concatenated volumes volprdnfs1 volprdnfs2 & volprdnfs3. 

 # vxprint -qhtg dgprdnfs
 dg dgprdnfs     default      default  16000    1106535203.86.eap42

 dm c3t0d52      c3t0d52s2    auto     2048     53028096 -
 dm c3t0d53      c3t0d53s2    auto     2048     53028096 -
 dm c3t0d54      c3t0d54s2    auto     2048     53028096 -
 dm c3t0d55      c3t0d55s2    auto     2048     53028096 -
 dm c3t0d122     c3t0d122s2   auto     2048     53026176 -
 dm c3t0d123     c3t0d123s2   auto     2048     53026176 -
 dm c3t0d124     c3t0d124s2   auto     2048     53026176 -
 dm c3t0d125     c3t0d125s2   auto     2048     53026176 -
 dm c3t0d126     c3t0d126s2   auto     2048     53026176 -
 dm c3t0d127     c3t0d127s2   auto     2048     53026176 -
 dm c3t0d128     c3t0d128s2   auto     2048     53026176 -
 dm c3t0d129     c3t0d129s2   auto     2048     53026176 -
 dm c3t0d130     c3t0d130s2   auto     2048     53026176 -
 dm c3t0d131     c3t0d131s2   auto     2048     53026176 -
 dm c3t0d132     c3t0d132s2   auto     2048     53026176 -
 dm c3t0d133     c3t0d133s2   auto     2048     53026176 -
 dm c3t0d203     c3t0d203s2   auto     2048     53026176 -
 dm dgprdnfs01   c3t0d12s2    auto     2048     53028096 -
 dm dgprdnfs02   c3t0d13s2    auto     2048     53028096 -
 dm dgprdnfs03   c3t0d14s2    auto     2048     53028096 -
 dm dgprdnfs04   c3t0d18s2    auto     2048     53028096 NOHOTUSE

 v  volprdnfs1   -            ENABLED  ACTIVE   943718400 SELECT   -        fsgen
 pl volprdnfs1-01 volprdnfs1  ENABLED  ACTIVE   943718400 CONCAT   -        RW
 sd dgprdnfs01-01 volprdnfs1-01 dgprdnfs01 0    53028096 0         c3t0d12  ENA
 sd dgprdnfs02-01 volprdnfs1-01 dgprdnfs02 0    9886464  53028096  c3t0d13  ENA
 sd dgprdnfs02-03 volprdnfs1-01 dgprdnfs02 19772928 33255168 62914560 c3t0d13 ENA
 sd dgprdnfs04-01 volprdnfs1-01 dgprdnfs04 0    39544576 96169728  c3t0d18  ENA
 sd c3t0d52-01   volprdnfs1-01 c3t0d52 0        53028096 135714304 c3t0d52  ENA
 sd c3t0d53-01   volprdnfs1-01 c3t0d53 0        53028096 188742400 c3t0d53  ENA
 sd c3t0d54-01   volprdnfs1-01 c3t0d54 0        53028096 241770496 c3t0d54  ENA
 sd c3t0d55-01   volprdnfs1-01 c3t0d55 0        50630912 294798592 c3t0d55  ENA
 sd c3t0d126-02  volprdnfs1-01 c3t0d126 53024   48247168 345429504 c3t0d126 ENA
 sd c3t0d127-01  volprdnfs1-01 c3t0d127 0       53026176 393676672 c3t0d127 ENA
 sd c3t0d128-01  volprdnfs1-01 c3t0d128 0       53026176 446702848 c3t0d128 ENA
 sd c3t0d129-01  volprdnfs1-01 c3t0d129 0       53026176 499729024 c3t0d129 ENA
 sd c3t0d130-01  volprdnfs1-01 c3t0d130 0       53026176 552755200 c3t0d130 ENA
 sd c3t0d131-01  volprdnfs1-01 c3t0d131 0       53026176 605781376 c3t0d131 ENA
 sd c3t0d132-01  volprdnfs1-01 c3t0d132 0       53026176 658807552 c3t0d132 ENA
 sd c3t0d133-01  volprdnfs1-01 c3t0d133 0       53026176 711833728 c3t0d133 ENA
 sd dgprdnfs04-02 volprdnfs1-01 dgprdnfs04 39544576 13483520 764859904 c3t0d18 ENA
 sd c3t0d55-02   volprdnfs1-01 c3t0d55 50630912 1898016  778343424 c3t0d55  ENA
 sd c3t0d123-03  volprdnfs1-01 c3t0d123 132064  52894112 780241440 c3t0d123 ENA
 sd c3t0d124-03  volprdnfs1-01 c3t0d124 130320  52895856 833135552 c3t0d124 ENA
 sd c3t0d125-02  volprdnfs1-01 c3t0d125 65168   52961008 886031408 c3t0d125 ENA
 sd c3t0d126-03  volprdnfs1-01 c3t0d126 48300192 4725984 938992416 c3t0d126 ENA

 v  volprdnfs2   -            ENABLED  ACTIVE   115343360 SELECT   -        fsgen
 pl volprdnfs2-01 volprdnfs2  ENABLED  ACTIVE   115343360 CONCAT   -        RW
 sd dgprdnfs02-02 volprdnfs2-01 dgprdnfs02 9886464 9886464 0       c3t0d13  ENA
 sd dgprdnfs03-01 volprdnfs2-01 dgprdnfs03 0    53028096 9886464   c3t0d14  ENA
 sd c3t0d203-01  volprdnfs2-01 c3t0d203 0       52428800 62914560  c3t0d203 ENA

 v  volprdnfs3   -            ENABLED  ACTIVE   53026176 SELECT    -        fsgen
 pl volprdnfs3-01 volprdnfs3  ENABLED  ACTIVE   53026176 CONCAT    -        RW
 sd c3t0d122-01  volprdnfs3-01 c3t0d122 0       53026176 0         c3t0d122 ENA

To start the mirroring process I have been given a set of of new disks. These sets of disks need to be equal or large enough to complete the mirroring process. Notice how these disks are on a separate controller/storage c4's.

 # vxdisk -o alldgs list|grep c4
 c4t2d1s2     auto:none       -            -            online invalid
 c4t2d2s2     auto:none       -            -            online invalid
 c4t2d3s2     auto:none       -            -            online invalid

Add new disks for mirroring to the same disk group as source.

 # vxdg -g dgprdnfs adddisk c4t2d1=c4t2d1s2
 # vxdg -g dgprdnfs adddisk c4t2d2=c4t2d2s2
 # vxdg -g dgprdnfs adddisk c4t2d3=c4t2d3s2

 # vxdisk -o alldgs list|grep c4
 c4t2d1s2     auto:cdsdisk    c4t2d1       dgprdnfs     online
 c4t2d2s2     auto:cdsdisk    c4t2d2       dgprdnfs     online
 c4t2d3s2     auto:cdsdisk    c4t2d3       dgprdnfs     online

Execute and Create mirror specifying the new disks to mirror each volume.

 # vxassist -g dgprdnfs -b mirror volprdnfs1 alloc=c4t2d1
 # vxassist -g dgprdnfs -b mirror volprdnfs2 alloc=c4t2d2
 # vxassist -g dgprdnfs -b mirror volprdnfs3 alloc=c4t2d3

After the above steps the following vxprint shows the new set of c4 disks attached to dgprdnfs disk group. Notice the sectored size of each disk, which must equal or be larger than the total volume size will are mirroring.

With the above vxassist command along with the mirror option we have executed the mirroring process, notice it has created a new plex for each volume volprdnfs1-02 volprdnfs2-02 volprdnfs3-02, each new plex is in TEMPRMSD state , which means an attached new temporary plex state, its copying the volume contents to the existing new plex, it wont be considered enabled once the process has completed.

 # vxprint -qhtg dgprdnfs
 dg dgprdnfs     default      default  16000    1106535203.86.eap42

 dm c3t0d52      c3t0d52s2    auto     2048     53028096 -
 dm c3t0d53      c3t0d53s2    auto     2048     53028096 -
 dm c3t0d54      c3t0d54s2    auto     2048     53028096 -
 dm c3t0d55      c3t0d55s2    auto     2048     53028096 -
 dm c3t0d122     c3t0d122s2   auto     2048     53026176 -
 dm c3t0d123     c3t0d123s2   auto     2048     53026176 -
 dm c3t0d124     c3t0d124s2   auto     2048     53026176 -
 dm c3t0d125     c3t0d125s2   auto     2048     53026176 -
 dm c3t0d126     c3t0d126s2   auto     2048     53026176 -
 dm c3t0d127     c3t0d127s2   auto     2048     53026176 -
 dm c3t0d128     c3t0d128s2   auto     2048     53026176 -
 dm c3t0d129     c3t0d129s2   auto     2048     53026176 -
 dm c3t0d130     c3t0d130s2   auto     2048     53026176 -
 dm c3t0d131     c3t0d131s2   auto     2048     53026176 -
 dm c3t0d132     c3t0d132s2   auto     2048     53026176 -
 dm c3t0d133     c3t0d133s2   auto     2048     53026176 -
 dm c3t0d203     c3t0d203s2   auto     2048     53026176 -
 dm c4t2d1       c4t2d1s2     auto     2048     964654848 -
 dm c4t2d2       c4t2d2s2     auto     2048     125821696 -
 dm c4t2d3       c4t2d3s2     auto     2048     62909696 -
 dm dgprdnfs01   c3t0d12s2    auto     2048     53028096 -
 dm dgprdnfs02   c3t0d13s2    auto     2048     53028096 -
 dm dgprdnfs03   c3t0d14s2    auto     2048     53028096 -
 dm dgprdnfs04   c3t0d18s2    auto     2048     53028096 NOHOTUSE

 v  volprdnfs1   -            ENABLED  ACTIVE   943718400 SELECT   -        fsgen
 pl volprdnfs1-01 volprdnfs1  ENABLED  ACTIVE   943718400 CONCAT   -        RW
 sd dgprdnfs01-01 volprdnfs1-01 dgprdnfs01 0    53028096 0         c3t0d12  ENA
 sd dgprdnfs02-01 volprdnfs1-01 dgprdnfs02 0    9886464  53028096  c3t0d13  ENA
 sd dgprdnfs02-03 volprdnfs1-01 dgprdnfs02 19772928 33255168 62914560 c3t0d13 ENA
 sd dgprdnfs04-01 volprdnfs1-01 dgprdnfs04 0    39544576 96169728  c3t0d18  ENA
 sd c3t0d52-01   volprdnfs1-01 c3t0d52 0        53028096 135714304 c3t0d52  ENA
 sd c3t0d53-01   volprdnfs1-01 c3t0d53 0        53028096 188742400 c3t0d53  ENA
 sd c3t0d54-01   volprdnfs1-01 c3t0d54 0        53028096 241770496 c3t0d54  ENA
 sd c3t0d55-01   volprdnfs1-01 c3t0d55 0        50630912 294798592 c3t0d55  ENA
 sd c3t0d126-02  volprdnfs1-01 c3t0d126 53024   48247168 345429504 c3t0d126 ENA
 sd c3t0d127-01  volprdnfs1-01 c3t0d127 0       53026176 393676672 c3t0d127 ENA
 sd c3t0d128-01  volprdnfs1-01 c3t0d128 0       53026176 446702848 c3t0d128 ENA
 sd c3t0d129-01  volprdnfs1-01 c3t0d129 0       53026176 499729024 c3t0d129 ENA
 sd c3t0d130-01  volprdnfs1-01 c3t0d130 0       53026176 552755200 c3t0d130 ENA
 sd c3t0d131-01  volprdnfs1-01 c3t0d131 0       53026176 605781376 c3t0d131 ENA
 sd c3t0d132-01  volprdnfs1-01 c3t0d132 0       53026176 658807552 c3t0d132 ENA
 sd c3t0d133-01  volprdnfs1-01 c3t0d133 0       53026176 711833728 c3t0d133 ENA
 sd dgprdnfs04-02 volprdnfs1-01 dgprdnfs04 39544576 13483520 764859904 c3t0d18 ENA
 sd c3t0d55-02   volprdnfs1-01 c3t0d55 50630912 1898016  778343424 c3t0d55  ENA
 sd c3t0d123-03  volprdnfs1-01 c3t0d123 132064  52894112 780241440 c3t0d123 ENA
 sd c3t0d124-03  volprdnfs1-01 c3t0d124 130320  52895856 833135552 c3t0d124 ENA
 sd c3t0d125-02  volprdnfs1-01 c3t0d125 65168   52961008 886031408 c3t0d125 ENA
 sd c3t0d126-03  volprdnfs1-01 c3t0d126 48300192 4725984 938992416 c3t0d126 ENA
 pl volprdnfs1-02 volprdnfs1  ENABLED  TEMPRMSD 943718400 CONCAT   -        WO
 sd c4t2d1-01    volprdnfs1-02 c4t2d1  0        943718400 0        c4t2d1   ENA

 v  volprdnfs2   -            ENABLED  ACTIVE   115343360 SELECT   -        fsgen
 pl volprdnfs2-01 volprdnfs2  ENABLED  ACTIVE   115343360 CONCAT   -        RW
 sd dgprdnfs02-02 volprdnfs2-01 dgprdnfs02 9886464 9886464 0       c3t0d13  ENA
 sd dgprdnfs03-01 volprdnfs2-01 dgprdnfs03 0    53028096 9886464   c3t0d14  ENA
 sd c3t0d203-01  volprdnfs2-01 c3t0d203 0       52428800 62914560  c3t0d203 ENA
 pl volprdnfs2-02 volprdnfs2  ENABLED  TEMPRMSD 115343360 CONCAT   -        WO
 sd c4t2d2-01    volprdnfs2-02 c4t2d2  0        115343360 0        c4t2d2   ENA

 v  volprdnfs3   -            ENABLED  ACTIVE   53026176 SELECT    -        fsgen
 pl volprdnfs3-01 volprdnfs3  ENABLED  ACTIVE   53026176 CONCAT    -        RW
 sd c3t0d122-01  volprdnfs3-01 c3t0d122 0       53026176 0         c3t0d122 ENA
 pl volprdnfs3-02 volprdnfs3  ENABLED  TEMPRMSD 53026176 CONCAT    -        WO
 sd c4t2d3-01    volprdnfs3-02 c4t2d3  0        53026176 0         c4t2d3   ENA

 

Monitor the synchronization process using the vxtask command.

 # vxtask list
 TASKID  PTID TYPE/STATE    PCT   PROGRESS
    161           ATCOPY/R 21.47% 0/53026176/11386880 PLXATT volprdnfs3 volprdnfs3-02 dgprdnfs
    163           ATCOPY/R 05.95% 0/115343360/6862848 PLXATT volprdnfs2 volprdnfs2-02 dgprdnfs
    164           ATCOPY/R 00.52% 0/943718400/4921344 PLXATT volprdnfs1 volprdnfs1-02 dgprdnfs

Once the synchronization process has completed, you are safe to remove the old plexes from the original source copy, or in this case the plexes which contain the c3's subdisks.

 # vxplex -g dgprdnfs -o rm dis volprdnfs1-01
 # vxplex -g dgprdnfs -o rm dis volprdnfs2-01
 # vxplex -g dgprdnfs -o rm dis volprdnfs3-01

Now the mirroring is complete, the following vxprint shows the end result. Disk group dgprdnfs is no longer dependent on controller c3 subdisks and now runs on a new set of storage subdisks c4's .

 # vxprint -qhtg dgprdnfs
 dg dgprdnfs     default      default  16000    1106535203.86.eap42

 dm c4t2d1       c4t2d1s2     auto     2048     964654848 -
 dm c4t2d2       c4t2d2s2     auto     2048     125821696 -
 dm c4t2d3       c4t2d3s2     auto     2048     62909696 -

 v  volprdnfs1   -            ENABLED  ACTIVE   943718400 SELECT   -        fsgen
 pl volprdnfs1-02 volprdnfs1  ENABLED  ACTIVE   943718400 CONCAT   -        RW
 sd c4t2d1-01    volprdnfs1-02 c4t2d1  0        943718400 0        c4t2d1   ENA

 v  volprdnfs2   -            ENABLED  ACTIVE   115343360 SELECT   -        fsgen
 pl volprdnfs2-02 volprdnfs2  ENABLED  ACTIVE   115343360 CONCAT   -        RW
 sd c4t2d2-01    volprdnfs2-02 c4t2d2  0        115343360 0        c4t2d2   ENA

 v  volprdnfs3   -            ENABLED  ACTIVE   53026176 SELECT    -        fsgen
 pl volprdnfs3-02 volprdnfs3  ENABLED  ACTIVE   53026176 CONCAT    -        RW
 sd c4t2d3-01    volprdnfs3-02 c4t2d3  0        53026176 0         c4t2d3   ENA