libarchive, bsdtar, bsdcpio

Comments, questions, patches: libarchive-discuss@googlegroups.com

Source Code and Downloads

For up-to-date information about libarchive, bsdtar, and bsdcpio please go to libarchive.org. Materials here are useful only for archeological purposes.

Questions

Discussion about libarchive, including problems, questions, and ideas: http://groups.google.com/group/libarchive-discuss

Announcements

Subscribe here: http://groups.google.com/group/libarchive-announce

About versions: An a in the version indicates a test release (typically with known problems that make it unsuitable for production use). A b in the version indicates a test release that may not yet be fully tested. For production applications, please use the most recent non-alpha/non-beta release. Feedback on any version is always appreciated.

VERY OLD Documentation

bsdtar program documentation: bsdtar.1
bsdcpio program documentation: bsdcpio.1
General libarchive overview: libarchive.3
Summary of file formats supported by libarchive: libarchive-formats.5
Tar archive format: tar.5
Cpio archive format: cpio.5
Mtree format: mtree.5
Read API: archive_read.3
Write API: archive_write.3
Write-Disk API: archive_write_disk.3
Miscellaneous library docs: archive_entry.3, archive_util.3
Libarchive internals: libarchive_internals.3

What Is It?

Libarchive is a programming library that can create and read several different streaming archive formats, including most popular tar variants, several cpio formats, and both BSD and GNU ar variants. It can also write shar archives and read ISO9660 CDROM images and ZIP archives. The bsdtar program is an implementation of tar(1) that is built on top of libarchive. It started as a test harness, but has grown into a feature-competitive replacement for GNU tar. The bsdcpio program is an implementation of cpio(1) that is built on top of libarchive.

The bsdtar archiving program is built on libarchive, so offers a variety of modern features. One unusual feature it offers is the ability to function as a format-conversion filter, reading entries from one archive and emitting an archive in a different format with the same contents. This feature was simple to implement because libarchive's robust automatic format detection makes it unnecessary to specify the format of the input archive. More details are available in the bsdtar documentation.

Why?

Sometime around 2001, there was a debate on one of the mailing lists about the FreeBSD package tools. The debate concerned two interrelated questions:

What is the "right" format for a package system?
Why are FreeBSD's package tools slower than their counterparts on other systems?

After looking into it a bit, I concluded that tar/gzip and tar/bzip2 were fine formats and that the performance problems were purely implementation issues.

So, I started a project to rewrite the package tools, beginning with pkg_add. Key to this project is a library that understands tar/gzip and tar/bzip2 archives. I finished libtarfile in early 2003 and realized that much of the core infrastructure would easily generalize to handling other formats, so the library was renamed libarchive. Once I had built libarchive, I realized that my early test harness was very nearly a complete BSD-licensed replacement for GNU tar, hence bsdtar. The FreeBSD project adopted bsdtar and libarchive and allowed me to continue development within the FreeBSD source tree. Around 2007, libarchive was ported to other platforms and primary development moved to a separate repository, first on GoogleCode, then on Github where it is today.