This installment of the Bi-Monthly Status Report is a few days late, but I'm pleased to say that it is chocked full of over 30 articles. May and June were yet again busy months; the Netperf project passed major milestones and can now be run with the debug.mpsafenet tunable turned on from sources in CVS. The ARM, MIPS, and PPC ports saw quite a bit of progress, as did several other SMPng and Netgraph projects. FreeBSD 5.3 is just around the corner, so don't hesitate to grab a snapshot and test the progress!
On a more serious note, it's very important to remember that code freeze for FreeBSD 5.3 will happen on August 15, 2004. This is only a few weeks away and there is still a lot to do. The TODO list for the release can be found at http://www.freebsd.org/releases/5.3R/todo.html. If you are looking for a way to contribute to the release, this TODO list has several items that are in urgent and in need of attention. Testing is also very important. The tree has had some stability stability problems in the past few weeks, but there are work-arounds that should allow everyone to continue testing and using FreeBSD. We absolutely must have FreeBSD 5.3 be a rock-solid release, so every little bit of contributed effort helps!
Thanks,
Scott Long
Contact: Maksim Yevmenkin <m_evmenkin@yahoo.com>
Bluetooth code was marked as non-i386 specific. It is now possible to build it on all supported platforms. Please help with testing. Other then this there was not much progress during last few months. I've been very busy with Real Life.
URL: http://www.csl.sony.co.jp/person/kjc/kjc/software.html#ALTQ
URL: http://www.rofug.ro/projects/freebsd-altq/
URL: http://kerneltrap.org/node.php?id=505
URL: http://people.freebsd.org/~mlaier/ALTQ_driver/
Contact: Max Laier <mlaier@FreeBSD.org>
The ALTQ framework is part of KAME for more than 4 years and has been adopted by Net- and OpenBSD since more than 3 years. It provides means of managing outgoing packets to do QoS and bandwidth limitations. OpenBSD developed a different way to interact with ALTQ using pf, which was adopted by KAME as the "default for everyday use".
The Romanian FreeBSD Users Group has had a project to work towards integration of ALTQ into FreeBSD, which provided a very good starting point for the final import. The import only provides the "pf mode" configuration and classification API as the older ALTQ3 API does not suit to our SMP approach.
A reworked configuration API (decoupled from pf) is in the making as are additional driver modifications. Both should be done before 5-STABLE is branched, although additional drivers can be imported during the lifetime of 5-STABLE as well.
Contact: Poul-Henning Kamp <phk@FreeBSD.org>
The buf-junta project is underway, I am trying to bisect the code such that we get a struct bufobj which is the handle and method carrier for a buffer-cache object. All vnodes contain a bufobj, but as filesystems get migrated to GEOM backing, bufobj's will exist which do not have an associated vnode. The work is ongoing.
Contact: Scott Long <scottl@freebsd.org>
Not much coding has taken place on this lately, with the recent focus being on refining the design. We are currently investigating per-CPU completion queues and threads in order to reduce locks and increase concurrency. Also reviewing the BSD/OS CAM lockdown to see what ideas can be shared. Work should hopefully puck back up in late July. Development is taking place in the FreeBSD Perforce repository under the //depot/projects/scottl-camlock/... branch for now.
URL: http://www.cronyx.ru/hardware/wan.html
Contact: Roman Kurakin <rik@FreeBSD.org>
cp(4) driver for Cronyx Tau-PCI was added. Cronyx Tau-PCI is family of synchronous WAN adapters with various set of interfaces such as V.35, RS-232, RS-530(449), X.21, E1, E3, T3, STS-1. This is a third family of Cronyx adapters that is supported by FreeBSD now. Now all three drivers cx(4), ctau(4) and cp(4) are on both major branches (HEAD and RELENG_4).
Busdma conversion was recently finished. Current work is concentrated on locking both for adapters drivers and for sppp (see my other report for additional information).
URL: http://www.eurobsdcon2004.de/
Contact: Patrick M. Hausen <hausen@punkt.de>
Registration for EuroBSDCon 2004 taking place in Karlsruhe, Germany, from Oct. 29th to 31st has just opened. An early bird discount will be offered to all registering until Aug. 15th. Please see the conference website for details.
URL: http://doc.fugspbr.org
URL: http://lists.fugspbr.org/listinfo.cgi/doc-fugspbr.org
URL: http://developer.berlios.de/projects/doc-br/
Contact: DOC-BR Discussion List <doc@fugspbr.org>
The FreeBSD Brazilian Documentation Project is an effort of the Brazilian FreeBSD Users Group (FUG-BR) to translate the available documentation to pt_BR. We are proud to announce that we've finished the Handbook and FDP Primer translation and they are being revised. Both should be integrated to the FreeBSD CVS repository shortly.
There are many other articles being translated and their status can be checked at our website. If you want to help please create an account at BerliOS, since our CVS repository is being hosted there, and contact us through our mailing list. Any help is welcome!
URL: http://www.evilcoder.org/freebsd_html
URL: http://www.evilcoder.org/freebsd/handbook.tbz
URL: http://www.evilcoder.org/freebsd/html.tbz
Contact: Remko Lodder <remko@elvandar.org>
The FreeBSD Dutch Documentation project is a ongoing project translating the FreeBSD handbook {and others} to the dutch language. We are still on the look for translators and people that are willing to check the current html documentation. If you are interested, contact me at the email address shown above. We currently are reading for some checkups and then insert the first documents into the documentation tree.
URL: http://www.freebsd.org/docproj/handbook3.html
Contact: Murray Stokely <murray@FreeBSD.org>
The Third Edition of the FreeBSD Handbook has been split into two volumes. The first volume, the User Guide, has been published. Work is progressing on the second volume. The following chapters are included in the second volume : advanced-networking, network-servers, config, boot, cutting-edge, disks, l10n, mac, mail, ppp-and-slip, security, serialcomms, users, vinum, eresources, bibliography, mirrors. Please see the Task List for information about what work remains to be done. In addition to technical and grammatical review, a number of HTML output assumptions in the document need to be corrected.
URL: http://portsmon.firepipe.net/index.html
Contact: Mark Linimon <linimon_at_lonesome_dot_com>
The system continues to function well. The accuracy of the automatic classification algorithm has been improved by assigning a higher priority to port names found in pieces of Makefiles.
Several bugs had to be fixed due to the transition from bento to pointyhat. For about two weeks the URLs to the build errors were wrong. This has now been corrected (but note that some of the pointyhat summary pages themselves still show the broken links.)
A report was added to show only PRs in the 'feedback' state, so that committers can focus on maintainer and/or responsible timeouts. (As a reminder, the policy is 2 weeks). Another report on 'ports that are in ports/MOVED, but still exist' has also been added to the Anomalies page. Sometimes these are actual errors but not always.
Here are my latest observations about the trends in ports PRs:
URL: https://projects.fsck.ch/profile/
Contact: Tobias Roth <ports@fsck.ch>
FreeBSD profile.sh is an enhancement to the FreeBSD 5 rcng boot system, targeted at laptops. One can configure multiple network environments (eg, home, work, university). After this initial configuration, the laptop detects automatically in what environment it is started and configures itself accordingly. Not only network settings, but almost everything from under /etc can be configured per environment. It is also possible to suspend the machine in one environment and wake it up in a different one, and reconfiguration will happen automatically.
Contact: Olivier Houchard <cognet@FreeBSD.org>
Not much to report, Xscale support is in progress, and should boot at least single user really soon on an Intel IQ31244Evaluation board.
URL: http://www.FreeBSD.org/projects/mips/
URL: http://www.mdstud.chalmers.se/~md1gavan/mips64emul/
Contact: Juli Mallett <jmallett@FreeBSD.org>
In the past two months, opportunities to perform a good chunk of work on FreeBSD/MIPS have arisen and significant issues with context switching, clocks, interrupts, and kernel virtual memory have been resolved. A number of issues with caches were fixed, however those are far from complete and at last check, there were issues when running cached which would prevent booting sometimes. Due to toolchain issues in progress, current kernels are no longer bootable on real hardware.
A 64-bit MIPS emulator has arisen giving the ability to test and debug in an emulator, and much testing has taken place in it. It has been added to the FreeBSD ports tree, and the port will be actively tracking the main codebase as possible. In general, FreeBSD/MIPS kernels should run fine in it.
Before toolchain and cache issues, the first kernel threads would run, busses and some devices would attach, and the system would boot to a mountroot prompt.
URL: http://berklix.com/scanjet/
Contact: Julian Stacey <jhs@FreeBSD.org>
HP Network Scanjet 5 can unobtrusively run FreeBSD inside the scanner. Those who miss their Unix at work can have a FreeBSD box, un-noticed & un-challenged by blinkered managers who block any non Microsoft PC in the building. http://berklix.com/scanjet/
Contact: John Baldwin <jhb@FreeBSD.org>
Support for programming the polarity and trigger mode of interrupt sources at runtime was added. This includes a mini-driver for the ELCR register used to control the configuration for ISA and EISA interrupts. The atpic driver reprograms the ELCR as necessary, while the apic driver reprograms the interrupt pin associated with an interrupt source as necessary. The information about which configuration to use mostly comes from ACPI. However, non-ACPI systems also force any ISA interrupts used to route PCI interrupts to use active-low polarity and level trigger.
Support for suspend and resume on i386 was also slightly improved. Suspend and resume support was added to the ELCR, $PIR, and apic drivers.
The ACPI PCI-PCI bridge driver was fixed to fall back to the PCI-PCI bridge swizzle method for routing interrupts when a routing table was not provided by the BIOS.
Mixed mode can now be disabled or enabled at boot time via a loader tunable.
Contact: Tim Robbins <tjr@FreeBSD.org>
Many more text-processing utilities in the FreeBSD base system have been updated to work with multibyte characters, including comm, cut, expand, fold, join, paste, unexpand, and uniq. New versions of GNU grep and GNU sort (from coreutils) have been imported, together with multibyte support patches from developers at IBM and Red Hat.
Future work will focus on modifying the regular expression functions to work with multibyte characters, improving performance of the C library routines, and updating the remaining utilities (sed and tr are two important ones still remaining).
URL: http://coombs.anu.edu.au/~avalon/ip-filter.html
Contact: Darren Reed <darrenr@FreeBSD.org>
IPFilter has been upgraded in both FreeBSD-current and 4-STABLE (post 4.10) from version 3.4.31 to 3.4.35.
Contact: Michael Nottebrock <lofi@FreeBSD.org>
The work on converting the build switches/OPTIONS currently present in the ports of the main KDE modules into separate ports in order to make packages available for the software/features they provide is progressing. Porting of KOffice 1.3.2 are nearly completed. The Swedish FreeBSD snapshot server http://snapshots.se.freebsd.org, operated and maintained by members of the KDE/FreeBSD team, is back up and running at full steam. Additional amd64 hardware has been added and amd64 snapshots will be available soon.
URL: http://www.FreeBSD.org/~nsouch/kgi4BSD
Contact: Nicholas Souchu <nsouch@FreeBSD.org>
KGI is going slowly but surely. The port of the KGI/Linux accel to FreeBSD is in progress. It's no more than a double buffering API for graphic command passing to the HW engine.
Most of the work in the past months was about console management and more especially dual head console. Otherwise a new driver building tree is now ready to compile Linux and FreeBSD drivers in the same tree.
Documentation about KGI design is in progress.
URL: http://people.freebsd.org/~jkoshy/projects/perf-measurement/
Contact: Joseph Koshy <jkoshy@FreeBSD.org>
The current design attempts to support both per-process and system-wide statistical profiling and per-process "virtual" performance counters. The userland API libpmc(3) is somewhat stable now, but the kernel module's design is being redone to handle MP better. Initial development is targeting the AMD Athlon CPUs, but the intent is to support all the CPUs that FreeBSD runs on.
An early prototype is available under Perforce [under //depot/user/jkoshy/projects/pmc/].
Contact: Brooks Davis <brooks@FreeBSD.org>
An enhanced network interface cloning API has been
committed. It allows interfaces to support more complex names
then the current name# style. This functionality
has been used to enable interesting cloners like
auto-configuring vlan interfaces. Other features include
locking of cloner structures and the ability of drivers to
reject destroy requests.
Work on userland support for this functionality is ongoing.
URL: http://www.freebsd.org/smp/
URL: http://www.watson.org/~robert/freebsd/netperf/
Contact: Robert Watson <rwatson@FreeBSD.org>
This project is aimed at converting the FreeBSD network stack from running under the single Giant kernel lock to permitting it to run in a fully parallel manner on multiple CPUs (i.e., a fully threaded network stack). This will improve performance/latency through reentrancy and preemption on single-processor machines, and also on multi-processor machines by permitting real parallelism in the processing of network traffic. As of FreeBSD 5.2, it was possible to run low level network functions, as well as the IP filtering and forwarding plane, without the Giant lock, as well as "process to completion" in the interrupt handler. This permitted both inbound and outbound traffic to run in parallel across multiple interfaces and CPUs.
Work continues to improve the maturity and completeness of the locking (and performance) of the network stack for 5.3. The network stack development branch has been updated to the latest CVS HEAD, as well as the following and more. Many but not all of these changes have been merged to the FreeBSD CVS tree as of the writing of this report. Complete details and more minor changes are documented in the README file on the netperf web page.
URL: http://www.benzedrine.cx/pf.html
Contact: Max Laier <mlaier@FreeBSD.org>
Contact: Daniel Hartmeier <dhartmei@FreeBSD.org>
We imported pf as of OpenBSD 3.5 stable on June, 17th which will be the base for 5-STABLE pf (according to the current schedule). The most important improvement in this release is the new interface handling which makes it possible to write pf rule sets for hot-pluggable devices and pseudo cloning devices, before they exist. The import of the ALTQ framework enabled us to finally provide the related pf functions as well.
Before 5-STABLE we will import some bug fixes from OpenBSD-current, which have not been merged to their stable branch, as well as some FreeBSD specific features. The planned ALTQ API make-over will also affect pf.
We are (desperately) looking for non-manpage documentation for FreeBSD pf and somebody to write it. Few things have changed so a port of the excellent "PF FAQ" on the OpenBSD homepage should be fitting. There are, however, a couple of points that need conversion. A simple tutorial how to setup a NAT gateway with pf would also help. The in-kernel NAT engine is very easy to use, we should tell people about this alternative. This is even more true since the pf module now plugs into GENERIC without modifications.
Contact: Peter Grehan <grehan@FreeBSD.org>
The port has been moving along steadily. There have been reports of buildworld running natively. Works is almost complete on make release so there will be bootable CD images in the near future.
Contact: Scott Long <scottl@freebsd.org>
Project Mini-Evil is an attempt to extend Bill Paul's 'Project Evil' Windows NDIS wrapper layer to the SCSI MiniPort and StorePort layers. While drivers exist for most storage controllers that are on the market today, many companies are integrating software RAID into their products but not providing any source code or design specs. Instead of constantly reverse-engineering these raid layers and attempting to shoehorn them into the ata-raid driver, Project Mini-Evil will run the Windows drivers directly. It will hopefully also run most any SCSI/ATA/RAID drivers that conform to the SCSI Miniport or Storeport specification.
Work on this project is split between making the NDIS wrapper code more general and implementing the new APIs. Development is taking place in the FreeBSD Perforce repository under the //depot/projects/sonofevil/... branch.
URL: http://www.FreeBSD.org/smp/
Contact: John Baldwin <jhb@FreeBSD.org>
Contact: <smp@FreeBSD.org>
Not a lot happened on the SMPng front outside of the work on locking the network stack (which is a large amount of work). The priorities of the various software interrupt threads were corrected and locking for taskqueues was improved. The return value of the sema_timedwait() function was adjusted to be more consistent with cv_timedwait(). A small fix was made to the sleepqueue code to shorten the amount of time that a sleepqueue chain lock is held when waking up threads. Some simple debug code for profiling the hash tables used in the sleep queue and turnstile code was added. This will allow developers to measure the impact of any tweaks to the hash table sizes or the hash algorithm.
URL: http://www.freebsd.org/~rik
Contact: Roman Kurakin <rik@FreeBSD.org>
Currently I work on two directions: if_spppfr.c and sppp locking (on behalf of netperf). At the moment of writing this sppp locking is not ready yet. But it would be ready in couple of days. Also you may find as a part of this work some user space fixes for rwatson netperf code (Only that I was able to catch while world compilation. If you know some others let me know and I'll try to fix them too).
Since sppp code is quite big and state machine is very complicated, it would be difficult to test all code paths. I will glad to get any help in testing all this stuff. More tester more probability to test all possible cases.
Work on FRF.12 (ng_frf12) is frozen since of low interest and lack of time. Current state of stable code: support of FRF.12 End-to-End fragmentation. Support of FRF.12 Interface (UNI and NNI) fragmentation is not tested.
Contact: Poul-Henning Kamp <phk@FreeBSD.org>
An effort to get the tty subsystem out from under Giant has morphed into an more general effort to eliminate a lot of code which have been improperly copy & pasted into device drivers. In an ideal world, tty drivers would never get near a cdevsw, but since some drivers are more than just tty drivers (for instance sync) a more sensible compromise must be reached. The work is ongoing.
Contact: Pawel Jakub Dawidek <pjd@FreeBSD.org>
I'm working on various GEOM classes. Some of them are already committed and ready for use (GATE, CONCAT, STRIPE, LABEL, NOP). The MIRROR class is finished in 90% and will be committed in very near future. Next I want to work on RAID3 and RAID5 implementations. Userland utility to control GEOM classes (geom(8)) is already in the tree.
URL: http://www.vuxml.org
URL: http://vuxml.FreeBSD.org
URL: http://www.freebsd.org/ports/portaudit/
Contact: Tom Rhodes <trhodes@FreeBSD.org>
The portaudit utility is currently an add-on to FreeBSD designed to give administrators and users a heads up with regards to security vulnerabilities in third party software. The VuXML database keeps a record of these security vulnerabilities along with internal security holes. When installed, the portaudit utility periodically downloads a database with known issues and checks all installed ports or packages against it; should it find vulnerable software installed the administrator or user is notified during the daily run output of the periodic scripts.
These utilities are considered to be of production quality and discussion is taking place over whether or not they should be included as part of the base system. All ports committers are urged to add entries when when a vulnerability is discovered; any questions may be sent to eik@ or myself.