Wednesday, March 28, 2012

VxFS 6.0 and compression (a real must have).

VxFS 6.0 and compression (a real must have).


Here's a little personal info: I like good movies and good anime (for the kids). I like them even better when stored as DVD5 or DVD9 iso images for playing them with VLC (VideoLan Client) on the wide-screen TV.
But as my list of DVD's grew, my VxFS 3Tb filesystem started to become full (about 400 DVDs). With VxFS version 9 compression feature, I gained some time and was able to reclaim some space..

First of all:

1) Upgrade to SF6.0
2) Upgrade your DG (vxdg upgrade <dg>)
3) Upgrade your filesystem (vxupgrade -n 9 <fs>)

Which gives:

[root@thorbardin ~]# vxdg  upgrade local00dg
[root@thorbardin ~]# vxupgrade -n 9 /export/home/raistlin/.private/movies0
UX:vxfs vxupgrade: ERROR: V-3-25236: /dev/vx/rdsk/local00dg/movies0_lv: already at disk layout version 9.

[root@thorbardin ~]# find /export/home/raistlin/.private/movies0/DVD -name \*.iso|wc -l
512

So we have 512 DVD images on this FS.

Let's start the compression process:
[root@thorbardin ~]# vxcompress -v -r /export/home/raistlin/.private/movies0/DVD

You may check the compression progress with:
[root@thorbardin ~]# /opt/VRTS/bin/fsadm -S compressed /export/home/raistlin/.private/movies0
 Mountpoint    Size(KB)    Available(KB)   Used(KB)   Logical_Size(KB) Space_Saved(KB)
/export/home/raistlin/.private/movies0  2929236384     146315977 2773199135       2928058770     154859635

You may also check the compression ratio for a file or a set of files with:
[root@thorbardin ~]# vxcompress -L /export/home/raistlin/.private/movies0/DVD/*/*.iso
[....]
   4%    6.42 GB    6.68 GB   100%   gzip-6  1024k  /export/home/raistlin/.private/movies0/DVD/Dessins_Animes/Barbapapa.iso
   7%     5.9 GB    6.33 GB   100%   gzip-6  1024k  /export/home/raistlin/.private/movies0/DVD/Dessins_Animes/Albator_78_-_D6.iso
   4%    7.59 GB    7.92 GB   100%   gzip-6  1024k  /export/home/raistlin/.private/movies0/DVD/Dessins_Animes/Albator_78_-_D2.iso
   4%    7.56 GB    7.88 GB   100%   gzip-6  1024k  /export/home/raistlin/.private/movies0/DVD/Dessins_Animes/Albator_78_-_D1.iso

Total:    512 files       147.7 GB (5%) storage savings

So here it is: compressing 'in-place' an existing VxFS filesystem saved me about 148Gb. Time to save more of my DVD's! Considering that DVD's are already compressed (MPEG2 streams), 5% gained over a 3Tb filesystem isn't that bad.

If you have static data or keep sparse files around (database datafiles come to mind), we've seen compression ratios of as much as 7-10x!

The nice thing is that this compression stuff is completely transparent; might even get you higher performance in some case (de-compression is done in-memory) and you can upgrade to compressed files in-place, without having to do the backup/restore dance. It's also revertable: just uncompress your files if you wish to go back.

My 2c, VxFS: good for home and enterprise use! :)

Friday, March 9, 2012

VxFS 6.0 comes with de-dupe!

For those of you who don't want to watch Symantec's video, here's a quick & cheap howto of a VxFS de-dupe test using SFHA 6.0 under RHEL5.8:

* First, here's our setup:
[root@vcs11 ~]# df
Filesystem           1K-blocks      Used Available Use% Mounted on
/dev/mapper/rootdg-lv_root
                      15109112   9258676   5070560  65% /
/dev/sda1               101086     20027     75840  21% /boot
tmpfs                  1029372         0   1029372   0% /dev/shm
tmpfs                        4         0         4   0% /dev/vx
/dev/vx/dsk/vcsdg/lv_vcs
                        131072      4473    118813   4% /shared/vcs



 * Let's check the initial state of this setup:
[root@vcs11 ~]# /opt/VRTS/bin/fsadm -S shared /shared/vcs
 Mountpoint    Size(KB)    Available(KB)   Used(KB)   Logical_Size(KB) Space_Saved(KB)
/shared/vcs      131072        118813       4473             4473             0


* Enable de-dupe for our test filesystem:
[root@vcs11 ~]# /opt/VRTS/bin/fsdedupadm enable /shared/vcs -c 4096

* Copy some data:
[root@vcs11 ~]# cp -a /etc/lvm /shared/vcs/lvm1

* Check the results (nothing happened):
[root@vcs11 ~]# /opt/VRTS/bin/fsadm -S shared /shared/vcs
 Mountpoint    Size(KB)    Available(KB)   Used(KB)   Logical_Size(KB) Space_Saved(KB)
/shared/vcs      131072        118724       4562             4562             0


* Copy some data again and again.
[root@vcs11 ~]# cp -a /etc/lvm /shared/vcs/lvm2

[root@vcs11 ~]# cp -a /etc/lvm /shared/vcs/lvm3

[root@vcs11 ~]# cp -a /etc/lvm /shared/vcs/lvm4


* Still no change!
[root@vcs11 ~]# /opt/VRTS/bin/fsadm -S shared /shared/vcs
 Mountpoint    Size(KB)    Available(KB)   Used(KB)   Logical_Size(KB) Space_Saved(KB)
/shared/vcs      131072        118457       4829             4829             0




* Enable De-dupe for our node.
[root@vcs11 ~]# /opt/VRTS/bin/fsdedupadm list /shared/vcs
Chunksize Enabled Schedule        NodeList        Filesystem
--------------------------------------------------------------------------------
4096      YES     NONE            vcs10.lasthome. /shared/vcs
[root@vcs11 ~]# /opt/VRTS/bin/fsdedupadm setnodelist vcs11.lasthome.solace.krynn /shared/vcs


* Start the de-dupe process.
[root@vcs11 ~]# /opt/VRTS/bin/fsdedupadm start /shared/vcs
UX:vxfs fsdedupadm: INFO: V-3-27767:  deduplication is started on /shared/vcs.


* Check the results
[root@vcs11 ~]# /opt/VRTS/bin/fsdedupadm status /shared/vcs
Saving    Status    Node            Type        Filesystem
--------------------------------------------------------------------------------
05%       COMPLETED vcs11           MANUAL      /shared/vcs
        2012/01/16 17:11:25 Begin full scan
        2012/01/16 17:11:29 End detecting duplicates and filesystem changes.

[root@vcs11 ~]# /opt/VRTS/bin/fsadm -S shared /shared/vcs
 Mountpoint    Size(KB)    Available(KB)   Used(KB)   Logical_Size(KB) Space_Saved(KB)
/shared/vcs      131072        118363       4906             5167           261


LVM2 bootdisk encapsulation on RHEL7/Centos7

Introduction Hi everyone, Life on overcloud nodes was simple back then and everybody loved that single 'root' partition on th...