ODP: [Bug-tar] GNU tar, star and BSD tar speed comparison +new script

Jan Psota
Thu, 25 Oct 2007 06:51:13 -0700

Latest TCP script at the bottom (3180 bytes).
4 tests: 64bit dual core Athlon tmpfs / disk (reiserfs) - 60MB/s,
        32bit Athlon tmpfs / disk (reiserfs) - 55MB/s
Both machines were idle -- used for testing only.
Tarball and extracted files were on different physical devices.
Test data: linux 2.6.22/3 kernel sources for memory operations,
for the other data average file size should bring enough info.

2 x [...] processor means 1 processor with 2 cores (2 entries in cpuinfo).
Archive format is set to pax (Joerg).
Let's end with it. I only wanted to send You a new version of TCP script :-).

--
Jan Psota

TCP, version 2007-10-25
Linux 2.6.22-suspend2-r2 / Gentoo Base System release 2.0.0_rc5
2012MB of memory, 2 x AMD Athlon(tm) 64 X2 Dual Core Processor 4200+ 2211.348 
512 KB 4426.24 bmips
gcc (GCC) 4.2.2 (Gentoo 4.2.2 p1.0)
CFLAGS="-O2 -march=k8 -pipe"

bsdtar: bsdtar 2.3.4 - libarchive 2.3.4
gnutar: tar (GNU tar) 1.19
star:   star: star 1.5a85 (x86_64-unknown-linux-gnu)

best time of 5 repetitions,
        src=linux-2.6.23, 291M in 23867 files, avg 13KB/file,
        archive=/tmp/tcp.tar, extract to /tmp/tcptmp
program operation       real    user    system  %CPU         speed
bsdtar  create          0.764   0.232   0.532   99.96       370308 KB/s
gnutar  create          0.743   0.200   0.512   95.87       380775 KB/s
star    create          0.587   0.040   0.820   100.00      441247 KB/s

bsdtar  list            0.164   0.096   0.068   99.84      1579341 KB/s
gnutar  list            0.218   0.064   0.152   98.92      1188128 KB/s
star    list            0.359   0.044   0.240   79.09       721481 KB/s

bsdtar  extract         0.733   0.200   0.504   96.02       353358 KB/s
gnutar  extract         0.625   0.092   0.508   96.02       414419 KB/s
star    extract         0.875   0.096   0.980   100.00      296013 KB/s

bsdtar  compare         0.001   0.000   0.000   0.00     259012000 KB/s
gnutar  compare         0.719   0.288   0.400   95.66       360239 KB/s
star    compare         0.695   0.224   0.636   100.00      372679 KB/s

[...]
best time of 3 repetitions,
        src=/home, 3.2G in 7447 files, avg 554KB/file,
        archive=/var/tcp.tar, extract to /mnt/a/tcptmp
program operation       real    user    system  %CPU         speed
bsdtar  create          184.680 0.552   13.365  7.53         17958 KB/s
gnutar  create          159.240 0.256   12.417  7.95         20827 KB/s
star    create          181.779 0.140   14.789  8.21         18203 KB/s

bsdtar  list            0.053   0.032   0.016   91.41     62435471 KB/s
gnutar  list            56.535  0.136   3.764   6.89         58531 KB/s
star    list            56.652  0.080   5.236   9.38         58410 KB/s

bsdtar  extract         78.914  0.820   15.149  20.23        41932 KB/s
gnutar  extract         78.480  0.196   14.197  18.33        42164 KB/s
star    extract         79.439  0.132   12.973  16.49        41655 KB/s

bsdtar  compare         0.001   0.000   0.000   0.00    3309080000 KB/s
gnutar  compare         61.771  3.464   8.905   20.02        53570 KB/s
star    compare         57.561  1.728   9.897   20.19        57488 KB/s


Linux 2.6.22-suspend2-smp / Gentoo Base System release 2.0.0_rc5
504MB of memory, 1 x AMD Athlon(tm) Processor 1500.033 256 KB 3002.55 bmips
gcc (GCC) 4.2.2 (Gentoo 4.2.2 p1.0)
CFLAGS="-O2 -march=athlon-xp -mfpmath=sse -frename-registers -pipe"

bsdtar: bsdtar 2.3.4 - libarchive 2.3.4
gnutar: tar (GNU tar) 1.19
star:   star: star 1.5a85 (i686-pc-linux-gnu)

best time of 3 repetitions,
        src=/usr/src/linux-2.6.22-suspend2/drivers, 119M in 5900 files,
        avg 21KB/file, archive=/tmp/tcp.tar, extract to /tmp/tcptmp
program operation       real    user    system  %CPU         speed
bsdtar  create          1.329   0.192   1.132   99.63        89784 KB/s
gnutar  create          1.223   0.124   1.092   99.46        97566 KB/s
star    create          1.848   0.036   1.708   94.36        61372 KB/s

bsdtar  list            0.167   0.060   0.108   100.00      679137 KB/s
gnutar  list            0.161   0.040   0.124   100.00      704447 KB/s
star    list            0.859   0.044   0.716   88.51       132032 KB/s

bsdtar  extract         1.186   0.172   1.012   99.87        95629 KB/s
gnutar  extract         1.064   0.056   1.004   99.63       106593 KB/s
star    extract         1.920   0.088   1.724   94.40        59070 KB/s

bsdtar  compare         0.002   0.000   0.000   0.00      56708000 KB/s
gnutar  compare         0.925   0.232   0.692   99.90       122611 KB/s
star    compare         1.569   0.376   1.096   93.79        72285 KB/s

[...]
best time of 3 repetitions,
        src=/home/jasiu, 2.1G in 8416 files, avg 277KB/file,
        archive=/home/j2/tcp.tar, extract to /mnt/a/tar/tcptmp
program operation       real    user    system  %CPU         speed
bsdtar  create          182.171 1.692   29.130  16.91        11584 KB/s
gnutar  create          174.999 0.632   27.450  16.04        12059 KB/s
star    create          180.004 0.360   41.795  23.41        11677 KB/s

bsdtar  list            0.214   0.076   0.136   99.04      9822294 KB/s
gnutar  list            0.210   0.076   0.136   100.00    10009385 KB/s
star    list            43.462  0.148   18.109  42.00        48363 KB/s

bsdtar  extract         94.912  4.476   31.574  37.98        22146 KB/s
gnutar  extract         94.657  0.396   29.462  31.54        22206 KB/s
star    extract         100.814 0.400   39.906  39.98        20849 KB/s

bsdtar  compare         0.003   0.000   0.004   100.00   700657000 KB/s
gnutar  compare         80.174  3.932   20.365  30.30        26217 KB/s
star    compare         73.911  8.341   27.670  48.72        28439 KB/s

=============================================================

Note by Tim Kientzle:  The "bsdtar compare" results here are
invalid since bsdtar does not support that operation.
For the list numbers, note that libarchive automatically optimizes
list operations on uncompressed tar archives on disk by using lseek()
to skip over the bodies of entries.  GNU tar added an option to
provide the same feature.

The biggest problem with these tests is that they only
cover uncompressed archives stored on disk.  The results for
compressed archives and/or archives stored on tape are
likely quite different.