[HEADSUP][CFT] pkgng beta1 is out ================================= Hi, pkgng has just reached the beta phase, and has now found its way to the ports tree (disabled by default). 1/ Why pkgng? ------------ Our current pkg_install tools are obsolete and hard to maintain, and they lack lots of features: - missing metadata - no upgrade support - no repository support - no fine dependency tracking - no modern binary package management And lots more. Having such old tools makes it hard to improve the ports infrastructure: lots of hacks have found their way into the different Mk/bsd.*.mk files to work around pkg_install limitations; there are lots of hacks in the packages metadata itself: @comment which are not comments; and so forth. We have some people writing tools to improve the situation (portmaster and portupgrade to name two), but they are limited by and can become quite complicated to maintain because of the pkg_install limitations. 2/ What it is? -------------- It is a tool that is designed to replace pkg_install and provide modern features to advance package management on FreeBSD. It has been done with compatibility in mind. Most of the ports tree is able to build on pkgng without modification (21500 succesful packages is the highest pkgng score so far). The missing ones will be easily fixed with pkgng in ports. It has been done with ease of migration in mind. It is easy to migrate from pkg_install to pkgng. (However, note that going backwards is not possible.) It has been done with FreeBSD features in mind: it supports chroot, jails, rcng, etc. It has been done with scripting features in mind: 'pkg query' will allow you to query almost everything from the pkgng database in a script friendly way. It has been done with improvement in mind: it doesn't need to be root to create packages with root files in it; it is already able to package from a stage/fakeroot/name_it_like_you_want directory; it is also able to fake the package creation to directly install the package from that fake/stage/whatever directory. It has been done with human readability in mind: the new metadata are stored in YAML format; the plist keywords can be extended with YAML (for the ports). It has been our thinking that the pkg binary is not able to please everyone's needs, so it has been written on top of a library which can be used by any other third party tools. (Think about packagekit, or ruby binding for portupgrade for example, or any other usage like these). 3/ Roadmap ---------- We plan a very long beta phase with lots of beta versions, released as often as possible to ease testing and help improve the tool as much as possible. The goal now that we are in beta is to not break anything for users, which means that pkgng will be able to safely upgrade itself. (No real breakage occurred during the alpha phase; expect even less in beta.) Most of the big features are implemented, so now if you have a revolutionary idea that breaks everything, it won't find its way into pkgng 1.0. You can still provide it for pkgng 2.0. 1.0 is not revolutionary because of the fact that it is full of workarounds to allow compatibility with the current ports tree. At some future time (TBD), once we have dropped pkg_install support, things will be able to move forward faster. pkgng will live in the ports tree, so it will evolve with the infrastructure, allowing us not to have to wait for the EOL of a release to be able to move forward to new features. The library API is currently not considered stable; it will be designated stable as of pkgng 2.0. Therefore, if you are going to use the library in a third party project, you can expect some breakage from time to time. Of course, we will avoid breakage as much as possible. The plan is to have pkgng 1.0 ready and rock solid for 10.0-RELEASE (and possibly 9.1-RELEASE). The more testers/contributors we have, the faster we can go, and the faster we go, the faster we can drop pkg_install and improve our port infrastructure. (Note: due to limitations in FreeBSD 7.x, we do not plan to backport there.) 4/ pkgng itself ---------------- pkg add: add packages the old way (should be avoided by users) pkg audit: audit the installed packages for vulnerabilities pkg autoremove: interactively propose packages to be removed that were installed automatically (as a dependency) and not depended on anymore pkg check: check the installed packages database, prompting for inconsistency and proposing to try to fix it pkg clean: cleanup the package cache from binary installation (from repositories) pkg delete: deinstall packages pkg info: query information from the installed packages in a user-friendly way pkg install: install packages from a remote repository pkg query: query information from the installed packages in a script-friendly way pkg register: register packages in the database, synchronising the files from a fake/stage directory, or with already installed files (like currently with ports) pkg search: search the remote database for packages pkg update: update the remote repositories databases pkg updating: scan the installed ports and show all UPDATING entries that affect one of the installed ports (same as old pkg_updating) pkg upgrade: perform a full binary upgrade of the installed packages pkg version: determine whether package(s) need to be updated (same as old pkg_version) pkg which: determine which package owns a file pkg2ng: convert a pkg_install installed database to a pkgng installed database (you would need the ports tree for pkg2ng to be able to gather missing information about packages) Sample output of pkginfo: $ pkg info -f libreoffice: Name : libreoffice Version : 3.4.4 Origin : editors/libreoffice Prefix : /usr/local Categories : editors Licenses : MPL & LGPL3 Maintainer : office@FreeBSD.org WWW : http://www.libreoffice.org/ Comment : Full integrated office productivity suite Options : DEBUG: off GNOME: off GTK: on JAVA: off KDE4: off MMEDIA: off PYUNO: off SDK: off SYSTRAY: off WEBDAV: off Flat size : 319 MB Description : LibreOffice is the free power-packed Open Source personal productivity suite for Windows, Macintosh and Linux, that gives you six feature-rich applications for all your document production and data processing needs: Writer, Calc, Impress, Draw, Math and Base. WWW: http://www.libreoffice.org/ 5/ what is missing ------------------ - for 1.0: * currently the user handling is done using the @exec/@unexec scripts from the ports; we need to finish the switch to the pw_/gr_util API to have cleaner handling. * better error reporting; lots of error/warning messages are currently technical and need to be improved to become more user-friendly. * better documentation; pkgng does a lot of things (more than what is described here) and needs to be documented. We lack enough native English speakers that are able to correctly document everything. * bug hunting and fixing. - for future: * capsicum: during EuroBSDCon some ideas were shared on how we can capsicumize pkgng, and more generally, increase security in package management; we need to have more thought about this subject. * improved arch support; currently pkgng is able to prevent installing amd64 packages into an arm box for examples, but we can go further and have real arch handling -- packaging noarch packages only once for all architectures (shell scripts, data, etc) and share them between the repository. This would reduce the size of the repositories and speed up the packages building process. * incrementally updated packages (diff packages): this is a really complicated task but could be done, would need a clean design. * asbtract/alternative packages (e.g. "provide http_server") * having a real sat solver for the dependency tree. Currently we have a really simple and minimalistic solver which works well but if we can to go to an even finer package management we would need a real solver. * many more to join the project to share your ideas. Keep in mind that in pkgng every thing should be and will remain simple, so when you come up with ideas, try providing a simple and clean design :)