Creating sparse file to show how thin-provisioning is possible on Linux

hi,

Recently I have a good lunch meet up with Wing Loon to investigate on some of the questions that I kept for weeks. We were discussing on how Docker can store the container with extra disk space (by default it is allocating 10Gb for each container), yet my VM has not been allocating such a huge disk space. We were pondering and have no clue, until we bumped into a computer science term called, Sparse file. In layman term, it is something like thin-provisioning allowing a system to allocate a temporarily space, but will not really claiming data block size until you write it onto the disk. This type of file will eventually grow as it goes.

Now, I would like to hand-held you on a few steps to demonstrate/create this type of file.


[root@localhost ~]# dd of=sparse-file bs=1M seek=102400 count=0
0+0 records in
0+0 records out
0 bytes (0 B) copied, 5.3053e-05 s, 0.0 kB/s
[root@localhost ~]# ls -al
total 64
dr-xr-x---.  5 root root         4096 Jun 13 11:10 .
drwxr-xr-x. 18 root root         4096 Jun 13 10:28 ..
-rw-------.  1 root root         1082 Jun  6 20:26 anaconda-ks.cfg
-rw-------.  1 root root        12916 Jun 13 10:06 .bash_history
-rw-r--r--.  1 root root           18 Dec  4  2013 .bash_logout
-rw-r--r--.  1 root root          262 Jun  9 11:12 .bash_profile
-rw-r--r--.  1 root root          176 Dec  4  2013 .bashrc
-rw-r--r--.  1 root root          100 Dec  4  2013 .cshrc
-rw-------.  1 root root           98 Jun 13 10:00 .lesshst
drwxr-xr-x.  3 root root         4096 Jun  6 21:01 .local
drwxr-xr-x.  4 root root         4096 Dec 12  2013 .mozilla
drwxr-----.  3 root root         4096 Jun  6 20:30 .pki
-rw-r--r--.  1 root root 107374182400 Jun 13 11:10 sparse-file
-rw-r--r--.  1 root root          129 Dec  4  2013 .tcshrc
[root@localhost ~]#
[root@localhost ~]# du -s sparse-file
0    sparse-file

If you understand these commands above, you will curious why ls -al sparse-file is showing such a huge numbers, and yet, du -s sparse-file is just a zeroed-file.That's what we called it sparse file.


So, next we will making use of this file, I can create a device mapper file, and put do a prank tricks to my friends. This is how it goes.


[root@localhost ~]# losetup -f
/dev/loop0
[root@localhost ~]# losetup /dev/loop0 /root/sparse-file
[root@localhost ~]# losetup
NAME       SIZELIMIT OFFSET AUTOCLEAR RO BACK-FILE
/dev/loop0         0      0         0  0 /root/sparse-file
[root@localhost ~]#
[root@localhost ~]#
[root@localhost ~]# blockdev --getsz /root/sparse-file
209715200
[root@localhost ~]#
[root@localhost ~]#
[root@localhost ~]# dmsetup create hiu_prank --table "0 209715200 linear /dev/loop0 0"
[root@localhost ~]#
[root@localhost ~]#
[root@localhost ~]# dmsetup ls
fedora-swap    (253:1)
fedora-root    (253:0)
hiu_prank    (253:2)
[root@localhost ~]#
[root@localhost ~]#
[root@localhost ~]# fdisk -l /dev/mapper/hiu_prank

Disk /dev/mapper/hiu_prank: 100 GiB, 107374182400 bytes, 209715200 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes


Now, i have a fake disk that claimed itself having 100Gb. I guess you know how to format it and mount it and write data onto it.

By the way, /var/log/lastlog is also another type of sparse file, which we use it daily.

Enjoy!
 (P.S. thank you wingloon.com)