Strict Standards: Declaration of Walker_Page::start_lvl() should be compatible with Walker::start_lvl($output) in /homepages/21/d19413512/htdocs/SITES/haegele.info/tim/wp-includes/classes.php on line 584

Strict Standards: Declaration of Walker_Page::end_lvl() should be compatible with Walker::end_lvl($output) in /homepages/21/d19413512/htdocs/SITES/haegele.info/tim/wp-includes/classes.php on line 584

Strict Standards: Declaration of Walker_Page::start_el() should be compatible with Walker::start_el($output) in /homepages/21/d19413512/htdocs/SITES/haegele.info/tim/wp-includes/classes.php on line 584

Strict Standards: Declaration of Walker_Page::end_el() should be compatible with Walker::end_el($output) in /homepages/21/d19413512/htdocs/SITES/haegele.info/tim/wp-includes/classes.php on line 584

Strict Standards: Declaration of Walker_PageDropdown::start_el() should be compatible with Walker::start_el($output) in /homepages/21/d19413512/htdocs/SITES/haegele.info/tim/wp-includes/classes.php on line 603

Strict Standards: Declaration of Walker_Category::start_lvl() should be compatible with Walker::start_lvl($output) in /homepages/21/d19413512/htdocs/SITES/haegele.info/tim/wp-includes/classes.php on line 702

Strict Standards: Declaration of Walker_Category::end_lvl() should be compatible with Walker::end_lvl($output) in /homepages/21/d19413512/htdocs/SITES/haegele.info/tim/wp-includes/classes.php on line 702

Strict Standards: Declaration of Walker_Category::start_el() should be compatible with Walker::start_el($output) in /homepages/21/d19413512/htdocs/SITES/haegele.info/tim/wp-includes/classes.php on line 702

Strict Standards: Declaration of Walker_Category::end_el() should be compatible with Walker::end_el($output) in /homepages/21/d19413512/htdocs/SITES/haegele.info/tim/wp-includes/classes.php on line 702

Strict Standards: Declaration of Walker_CategoryDropdown::start_el() should be compatible with Walker::start_el($output) in /homepages/21/d19413512/htdocs/SITES/haegele.info/tim/wp-includes/classes.php on line 727

Strict Standards: Redefining already defined constructor for class wpdb in /homepages/21/d19413512/htdocs/SITES/haegele.info/tim/wp-includes/wp-db.php on line 58

Deprecated: Assigning the return value of new by reference is deprecated in /homepages/21/d19413512/htdocs/SITES/haegele.info/tim/wp-includes/cache.php on line 99

Strict Standards: Redefining already defined constructor for class WP_Object_Cache in /homepages/21/d19413512/htdocs/SITES/haegele.info/tim/wp-includes/cache.php on line 404

Deprecated: Assigning the return value of new by reference is deprecated in /homepages/21/d19413512/htdocs/SITES/haegele.info/tim/wp-includes/query.php on line 21

Deprecated: Assigning the return value of new by reference is deprecated in /homepages/21/d19413512/htdocs/SITES/haegele.info/tim/wp-includes/theme.php on line 576
Tims Wein- und IT-Blog » Blog Archive » Free binary diff - A solution for diff based Image Backups?

Strict Standards: mktime(): It is not safe to rely on the system's timezone settings. You are *required* to use the date.timezone setting or the date_default_timezone_set() function. In case you used any of those methods and you are still getting this warning, you most likely misspelled the timezone identifier. We selected 'Europe/Berlin' for 'CET/1.0/no DST' instead in /homepages/21/d19413512/htdocs/SITES/haegele.info/tim/wp-includes/functions.php on line 19

Strict Standards: date(): It is not safe to rely on the system's timezone settings. You are *required* to use the date.timezone setting or the date_default_timezone_set() function. In case you used any of those methods and you are still getting this warning, you most likely misspelled the timezone identifier. We selected 'Europe/Berlin' for 'CET/1.0/no DST' instead in /homepages/21/d19413512/htdocs/SITES/haegele.info/tim/wp-includes/functions.php on line 28

Strict Standards: date(): It is not safe to rely on the system's timezone settings. You are *required* to use the date.timezone setting or the date_default_timezone_set() function. In case you used any of those methods and you are still getting this warning, you most likely misspelled the timezone identifier. We selected 'Europe/Berlin' for 'CET/1.0/no DST' instead in /homepages/21/d19413512/htdocs/SITES/haegele.info/tim/wp-includes/functions.php on line 30

Strict Standards: date(): It is not safe to rely on the system's timezone settings. You are *required* to use the date.timezone setting or the date_default_timezone_set() function. In case you used any of those methods and you are still getting this warning, you most likely misspelled the timezone identifier. We selected 'Europe/Berlin' for 'CET/1.0/no DST' instead in /homepages/21/d19413512/htdocs/SITES/haegele.info/tim/wp-includes/functions.php on line 32

Strict Standards: date(): It is not safe to rely on the system's timezone settings. You are *required* to use the date.timezone setting or the date_default_timezone_set() function. In case you used any of those methods and you are still getting this warning, you most likely misspelled the timezone identifier. We selected 'Europe/Berlin' for 'CET/1.0/no DST' instead in /homepages/21/d19413512/htdocs/SITES/haegele.info/tim/wp-includes/functions.php on line 33

Strict Standards: mktime(): It is not safe to rely on the system's timezone settings. You are *required* to use the date.timezone setting or the date_default_timezone_set() function. In case you used any of those methods and you are still getting this warning, you most likely misspelled the timezone identifier. We selected 'Europe/Berlin' for 'CET/1.0/no DST' instead in /homepages/21/d19413512/htdocs/SITES/haegele.info/tim/wp-includes/functions.php on line 19

Strict Standards: date(): It is not safe to rely on the system's timezone settings. You are *required* to use the date.timezone setting or the date_default_timezone_set() function. In case you used any of those methods and you are still getting this warning, you most likely misspelled the timezone identifier. We selected 'Europe/Berlin' for 'CET/1.0/no DST' instead in /homepages/21/d19413512/htdocs/SITES/haegele.info/tim/wp-includes/functions.php on line 28

Strict Standards: date(): It is not safe to rely on the system's timezone settings. You are *required* to use the date.timezone setting or the date_default_timezone_set() function. In case you used any of those methods and you are still getting this warning, you most likely misspelled the timezone identifier. We selected 'Europe/Berlin' for 'CET/1.0/no DST' instead in /homepages/21/d19413512/htdocs/SITES/haegele.info/tim/wp-includes/functions.php on line 30

Strict Standards: date(): It is not safe to rely on the system's timezone settings. You are *required* to use the date.timezone setting or the date_default_timezone_set() function. In case you used any of those methods and you are still getting this warning, you most likely misspelled the timezone identifier. We selected 'Europe/Berlin' for 'CET/1.0/no DST' instead in /homepages/21/d19413512/htdocs/SITES/haegele.info/tim/wp-includes/functions.php on line 32

Strict Standards: date(): It is not safe to rely on the system's timezone settings. You are *required* to use the date.timezone setting or the date_default_timezone_set() function. In case you used any of those methods and you are still getting this warning, you most likely misspelled the timezone identifier. We selected 'Europe/Berlin' for 'CET/1.0/no DST' instead in /homepages/21/d19413512/htdocs/SITES/haegele.info/tim/wp-includes/functions.php on line 33

Free binary diff - A solution for diff based Image Backups?

Xdelta is a free binary diff Software. The idea to try it out was finding a solution to backup Xen VM-Images more efficiently. Currently its mainly done with dd. Full Backups of the Images are expensive, and file-backup from inside the VM are unpractical in terms of restore or migration.

I built xdelta3 from source because the include xdelta1 in Debian etch didn’t work as expected.
In the meantime I took a offline copy of the image called disk.img-dist. This will be my “source” file. Afterwards I started the VM again and created some input inside the Wiki - actually I only deleted parts of a topic.

Off course the Image size remains the same:

monfortino:/srv/xen/domains/WikiBrain# ls -al
total 6281480
drwxr-xr-x  4 root root       4096 2008-02-27 17:41 .
drwxr-xr-x  9 root root       4096 2008-01-29 19:09 ..
-rw-r--r--  1 root root 3145728000 2008-01-29 21:13 disk.img
-rw-r--r--  1 root root 3145728000 2008-02-27 17:17 disk.img-dist
-rw-r--r--  1 root root  134217728 2007-09-24 21:44 swap.img

Creating a diff with xdelta3 is similar to the syntax of gzip, and creating a delta - or call it patch, diff - can be seen as compression, while the opposite, patching a file, is called decompression .
The following delta is created after minimal changes to the Wiki inside the disk.img (as already said).

monfortino:/srv/xen/domains/WikiBrain# time xdelta3 -s disk.img-dist disk.img delta
xdelta3: warning: input window 276824064..285212672 has no source copies
xdelta3: warning: input window 285212672..293601280 has no source copies
xdelta3: warning: input window 293601280..301989888 has no source copies
xdelta3: warning: input window 301989888..310378496 has no source copies

The resulting delta is just 315K in size:

-rw-r--r--  1 root root 315K 2008-02-27 17:43 delta

xdelta3 is rather quick:

real    1m34.007s
user    0m10.129s
sys     0m5.308s

In the other direction, applying the diff, xdelta3 is equally fast:

monfortino:/srv/xen/domains/WikiBrain# time xdelta3 -d -s disk.img-dist delta test.img

real    1m34.591s
user    0m3.640s
sys     0m11.505s

The md5sum of the test file and the source files are as expected the same:

monfortino:/srv/xen/domains/WikiBrain# md5sum disk.img && md5sum test.img
f34e91e42c00dacb37657b1e9a9e5657  disk.img
f34e91e42c00dacb37657b1e9a9e5657  test.img

Possible use of the Software can be, as already mentioned diff Backup of VM Images. If you use LVM you can create snapshots of the running System, read from the snapshot device and pipe it to xdelta3 just like to gzip - havent tested this yet, but I am sure it works :)

More benchmarking:

The results of a test running the same command on a 10GB VM Image where I upgrade 70MB Systemfiles and deleted the downloaded installationfiles afterwards are shown below:

# time xdelta3 -s Backup/disk.img-dist disk.img-test Backup/delta`date +%j`

real    7m19.994s
user    2m6.416s
sys     0m15.665s

monfortino:/srv/xen/domains/GarantDoc# ls -lah Backup/delta058
-rw-r--r-- 1 root root 160M 2008-02-27 21:25 Backup/delta058

With the maximum compression switch “-9″ top shows 234M virt Mem usage, without the switch it was about 140M

# time xdelta3 -9 -s Backup/disk.img-dist disk.img-test Backup/delta`date +%j_1`
real    9m34.542s
user    4m15.784s
sys     0m15.765s

But the filessize degrease is rather low:

-rw-r--r-- 1 root root 156M 2008-02-27 22:23 delta058_1

EDIT

from 21.03.08:

monfortino:/srv/xen/domains/GarantDoc# time /root/xdelta-read-only/xdelta3/xdelta3 -f -s Backup/disk.img-dist disk.img Backup/delta`date +%j`

real    8m59.031s
user    2m22.865s
sys     0m14.181s

monfortino:/srv/xen/domains/GarantDoc# ls -alh Backup/
-rw-r--r-- 1 root root 181M 2008-03-21 16:38 delta081

Leave a Reply

You must be logged in to post a comment.