If you're using Netscape, turn style sheets off.


Dag-Erling Smørgrav


2001-12-18: Twenty-four

A quick note before leaving on Christmas vacation—I'm going up north with my fiançée to visit her mother. Sickness and other obligations have kept me from significant FreeBSD work the past few weeks, but I hope this vacation will replenish my batteries so I can make a strong comeback come the new year. Happy holidays!


2001-12-12: Sick of being sick

I've been bedridden for four straight days now, venturing out in the upright positions only for special occasions, such as my NUUG presentation about PAM, from which I've just returned. It went rather well, I think—they asked me back, anyway. In the meantime, I have a huge backlog of unread mail, not to mention all the work that didn't get done. Oh well...


2001-12-06: To do or not to do, that is the question

I said yesterday that I was going to update the to-do list, but I haven't had time. I've been too busy doing the stuff on the list :-)

Linux-PAM progress

I started off by checking out the head of the Linux-PAM CVS tree and studying the changes since 0.75. The changes to the acutal PAM library were disappointingly small; slightly more than 700 lines in unified diff format. The FreeBSD RTLD_NOW problem has been fixed—badly; pam_handlers.c no longer redefines RTLD_NOW, but the headers are still in the wrong order, so dlfcn.h is not included, so now the code won't even build on FreeBSD. The funny thing is that as far as I can figure out, it won't build on Linux either, unless header pollution in glibc is even worse than I think it is.


I also followed up on yesterday's idea of writing the skeleton of an XSSO-compliant PAM library (which I've dubbed OpenPAM). I've created headers with all the types, constants and prototypes listed in the XSSO preliminary spec, and written a Perl script that auto-generates stubs for all the API functions (each in a separate file). I also implemented (partly or completely) pam_start(3), pam_end(3), pam_strerror(3), pam_set_item(3) and pam_get_item(3). All other functions return PAM_SYSTEM_ERR. Not very useful, but I can now build (and run, though all requests fail) simple PAM applications with my reference library and thus get an idea of how far we've drifted from standard PAM. An

I discovered quite a number of errors in XSSO in the process of writing the OpenPAM skeleton, and I'm going to try to start a dialog with The Open Group about XSSO and OpenPAM. I'm also considering shoving OpenPAM into the projects subdirectory of the FreeBSD CVS repository, but haven't made up my mind about it yet.

More documentation

I've started writing an article about PAM for the FreeBSD Documentation Project. It's far from done, but I've checked in what I've written so far (doc/en_US.ISO8859-1/articles/pam/) and an HTML rendition is available online here. It's looking pretty good so far, if I may say so myself.


Of course, it turned out that the pam.d issue wasn't as simple as I originally thought, and people started having trouble with sudo, so I spent two hours tracking the problem down and ended up reorganizing _pam_init_handlers(). It's even semi-readable now. And it's way past my bedtime.


2001-12-05: Small progress

I spent most of the day cleaning up some docs for a customer, then meeting with them, then meeting with another customer, but I've also had time to commit most of what I've done so far on PAM, plus clean up some namespace pollution issues in the headers.

I've started looking into the pam.conf / pam.d issue, especially since I want to add support for a /usr/local/etc/pam.d directory, and in the process, I started wondering how much work it is to write an XSSO-compliant PAM library from scratch—just the library, not the modules—because Linux-PAM has severe maintainability issues, and is not being developed very actively. It's mostly idle wondering right now, but I think I'm going to throw together a skeleton just to get an idea of how much work it would be.

Not so small after all

I found and fixed a bug in _pam_init_handlers() in pam_handlers.c which prevented /etc/pam.conf from working as a fallback for /etc/pam.d if the latter exists but is empty. Then I wrote a Perl script that created pam.d configuration files from an existing pam.conf. My desktop now lives happily with a fully populated /etc/pam.d and no /etc/pam.conf, and most of this work has already been committed. All in all, not a bad day :-)

What's the point?

Ashley asked me on IRC today what the aim of my work on PAM was. Here's the exact wording from my contract with NAI:

Complete and extend the Pluggable Authentication Module (PAM) implementation for FreeBSD. Modify all major base system network daemons involved in authentication to properly employ PAM. Employ PAM as widely as possible throughout the system to correctly manage credentials, authorize login, and account for resources consumed. Introduce support for additional PAM authentication modules, including support for at least two low-cost cryptographic peripherals.

Keeping score

Looking at my original to-do list, I have now completed or rendered irrelevant items 1, 2 and 6, and made some progress on items 4 and 5. I've barely scratched the surface of item 3, and there are some items missing from the list (support for hardware authentication devices, for instance) which are nonetheless included in my contract. I'll revise the to-do list tomorrow to take all this into account.

Oh, and I need to work on my NUUG presentation—it's next wednesday.

Double-oh, I must remember to write a pam_sasl module.


2001-12-03: R&R

I was in Bergen on R&R from Thursday until early this afternoon, and came back nicely rested from a weekend with my fiancée. This doesn't mean I've been idle, though. We spent the first day and night babysitting a friend's son, and he (the father, not the son) has a microwave link to the University of Bergen, so I got some work done, mostly on a Perl script for indexing Majordomo archives, which I wrote for the buggers archive.

Upon my return, I had quite a bit of catch-up to do, most of it Bugmeister-related. I started by implementing something I've always missed: a tool to perform simple PR operations without going through the hassle of edit-pr(1). I ended up with a pair of scripts, one of which sets some environment variables and invokes edit-pr(1) with EDITOR pointing to the second script, which makes the required changes to the PR and fills in the audit trail entries with the reason provided. Very nifty:

des@freefall ~% pradm close 32294 'MFCed, thanks!'
vipr: editing PR 32294...
vipr: editing audit trail entry...
edit-pr: filing PR docs/32294 back into the database...

I've also decided to hand out bugbuster hats to the following persons:

Oh, and I closed or feedback'ed a handful of PRs. Attaboy.

I miss my cats though.


2001-11-27: Frustration

I'm broke, I'm tired, I have a headache, I'm coming down with a cold, and everything I've done today (well, almost everything) has turned out wrong. Grrrr. Be nice to me Or Else.

More documentation

I've written and committed a pam_set_item(3) man page. I think it turned out quite well—I seem to be getting the hang of mdoc(7).

I've also spent some time reading Andrew Morgan's PAM whitepaper and browsing through the original OSF RFC and The Open Group's preliminary draft for PAM.. The OSF and X/Open documents describe PAM as it was originally defined and implemented by Sun, while Andrew Morgan tries to go one step further by separating out the conversation functions so they can run on the client rather than on the server. It's a good idea, but the design is flawed (one problem is that the proposed protocol does not include mechanisms for exchanging information about library versions and supported features; the paper suggests that this should be negotiated out of band!) and it does not address or even acknowledge the event vs. callback problem I discussed in an earlier issue. One very good thing about Morgan's paper, though, is that it establishes terminology for most of the various components and actors involved in a PAM session. Where previous writers used user and remote user, for instance, which is confusing not only because remote user includes the word user but also because the remote user is not necessarily remote, Morgan chooses to use user and applicant. Not quite perfect, but certainly a huge step in the right direction.

SourceForge tribulations

I tried to submit patches to Linux-PAM through their SourceForge site, and came away quite rankled and frustrated. Using SourceForge's bug- and patch-tracking system is a gruesome experience. There's no way to preview a ticket before it's filed, so you can't tell if you goofed, and there's no way to correct a ticket after you've filed it either; though if you're logged in, SourceForge will helpfully present you with the full ticket-editing interface with lots of controls for modifying the ticket, removing attached files etc., which—unless you're an administrator for the project to which you submitted the ticket—do absolutely nothing except clutter your screen and render the ticket essentially unreadable. Never mind the fact that SourceForge can easily verify that you are the original submitter, since you were logged in all along. To top it all off, if you screw up while submitting a ticket, SourceForge will present you with a nice, big, red error message, but it will not return you to the submission form and let you correct your mistake—nor will it tell you that it has in fact already filed the erroneous ticket! You are then left to choose between: filing a new (hopefully correct) ticket, and hoping the project admins won't be ticked off at your spammage; submitting a follow-up to the incorrect ticket explaining what is wrong, and hoping the admins won't be ticked off at having to manually effect the changes you specify; or throwing up your arms in disgust and going out for a beer.

I used to think SourceForge was a neat idea—then I started using it.

Yes, you may quote me on that.

Bug-tracking explorations

In the past few days, I've looked (or at least glanced cursorily) at four different open-source bug-tracking systems: Request Tracker, Mantis, Bugzilla and Scarab. They all have their strength and weaknesses. Here's a summary of my impressions:

So where does this all lead to? I think my feelings about these systems can be summarized as follows: if I needed a ticket tracking system for my company, I'd use Request Tracker. If I were picking a bug tracking system for an open source project that didn't already have one, I'd choose Mantis, or Bugzilla if I were worried about not knowing PHP well enough to make local hacks to Mantis. But if I were picking a bug tracking system for an open source project that had previously relied on Gnats and had a database of 30,000 problem reports, I wouldn't pick any of these systems. They are all, in different ways, so much of a departure from Gnats that the transition would be too painful - and they would all require a significant amount of glue code to integrate with the existing infrastructure (such as the installed base of FreeBSD systems with Gnats' send-pr(1) utility). FreeBSD's problem report database is such an important part of the project's workflow and methodology that replacing Gnats with a system designed to fit somebody else's workflow and methodology is simply not an option.

Bugs, Darwin and thermodynamics

I won't even try to explain. Just read the paper.


2001-11-26: Documentation

I spent some time today writing a man page for the pam_ssh(8) module, and updating the pam_unix(8) man page. Next on my list is a quick scan of the other existing man pages, just to check for any glaring errors or omissions, then PR docs/32294, then any other missing man pages, then a pam_module(8) man page explaining how to write a PAM module. That done, I'll return to PAMifying userland programs like passwd(1) etc.


2001-11-25: Intermission

Long time no write! I've spent the last couple of days playing around with neural networks and writing a neural network engine in Java which I've tentatively christened Jinn. It's far from done yet, but I'm going to have to shelve it for a little while and get back to the harsh reality of PAM and problem reports.

The next item on my Bugmeister to-do list is to write a set of guidelines for handling problem reports; I'm going to air this on the list and see what the buggers come up with. As for PAM, I'm going to stray from my to-do list for a while once I've committed the stuff I did last week; first because I've received some external input I want to look at and reflect upon, and second because I want to take some time to look at the newest Linux-PAM and see if they've done anything worth nabbing since we imported PAM into our tree.

I've been tinkering with a script that converts Gnats PRs (either single PRs or entire directories) to some sort of XML. I haven't decided on the schema yet, though, so it's still quite unfinished. Next on the list will be to import new PRs into an XML database (a related yet slightly different problem).

Oh, and I'm starting to think Rik Rose was right about using an SQL database as a backend for the PR system. It does simplify things a lot. The requirement for easy distribution can be satisified by regularly exporting the database to some human-readable format and distributing that.

Finally, memo to self: I still have to make slides for my NUUG presentation, and try to write that PAM-NG paper I've been thinking about.


2001-11-22: Buggers

Mailing list

I've created a mailing list at buggers@ofug.org for informal discussions about the PR system. The charter is:

The purpose of this list is to serve as an informal forum for discussing policies and mechanisms for PR handling in the context of the FreeBSD Project.

It's open, so go ahead and subscribe, if you think you have something to contribute. To subscribe, send mail to majordomo@ofug.org with subscribe buggers in the body.


I spent a good chunk of the evening writing an article about how to write good problem reports. It's not really done yet — amongst other things, I should add a section about what information to provide when reporting a kernel panic — but it's a good start.

The article will eventually reside under the the FreeBSD documentation page, but I haven't connected it to the doc or www build yet, so for the time being it's here.


2001-11-21: Bugmeister reprise


Jordan finally posted the announcement to -developers:

From: Jordan Hubbard <jkh@winston.freebsd.org>
Subject: Announcing the creation of "The Bugmeister"
To: developers@freebsd.org
Date: Wed, 21 Nov 2001 08:14:30 -0800
Message-ID: <38374.1006359270@winston.freebsd.org>
Sender: owner-developers@FreeBSD.org

Core is pleased to announce that as of Tuesday, November 20, we've
created the position of "Bugmeister", along with a really outlandish
looking fruit-covered hat to go with it, with the following charter:

  The Bugmeister manages the Project's bug tracking system.  His
  primary responsibilities are to define a consistent policy for
  handling problem reports, and to develop and maintain tools that
  will encourage and facilitate handling problem reports in accordance
  with this policy.  The Bugmeister may delegate some of his tasks to
  a team of Bugbusters, whom he will appoint from the developer and user

  The Bugmeister and Bugbuster positions are intended to supplement,
  rather than replace, the important part played by CVS committers in
  handling problem reports.

Dag-Erling Smørgrav (des@FreeBSD.org) has kindly volunteered for,
helped define, and rammed through core with sheer bloody-minded
persistence, the Bugmeister position.  We thank him for his
perseverance and willingness to serve!

- Core

A heavy hat indeed

Now that I'm a Bugmeister, I'm going to have to start acting like one. Here's a rough prioritized list of tasks for the near future, starting with some administrativia:

  1. Document the position in all the right places in the Handbook etc.
  2. Create a -bugbusters mailing list and start recruiting Bugbusters.
  3. Write a set of guidelines for handling PRs.
  4. Write a set of guidelines for submitting PRs.
  5. Organize a systematic sweep of the PR database to close stale or bogus PRs.

How to Report Bugs Effectively

Tony Finch sent me a link to a very interesting essay on how to write good problem reports.


2001-11-20: Bugmeister

Although it hasn't been announced officially yet, -core has accepted my application for the position of Bugmeister, which was created with the following charter:

The Bugmeister manages the Project's bug tracking system. His primary responsibilities are to define a consistent policy for handling problem reports, and to develop and maintain tools that will encourage and facilitate handling problem reports in accordance with this policy. The Bugmeister may delegate some of his tasks to a team of Bugbusters, whom he appoints from the developer and user communities.

The Bugmeister and Bugbuster positions are intended to supplement, rather than replace, the important part played by CVS committers in handling problem reports.

There's not much else to report — I spent most of my waking hours today playing Werewolf: the Apocalypse — but I did come up with a couple more loose ideas about the Bugmeister position and the new PR system:


2001-11-19: Follow-through

Hacking PAM modules

Having spent a lovely weekend R&R'ing with my girlfriend (and freezing my nads off trying to spot Leonids), I'm ready to get down and dirty with PAM again. Today's agenda is item 1 from the original to-do list, completing existing modules so they all provide all four facilities. I'm also planning to spend some time cleaning up any (subjective) ugliness I encounter in the process, and consider possible feature improvements to the existing modules. I also have some ideas for additional modules, but I'll get back to that at a later date.

Although there are only four facilities, they translate into six functions, because two of the facilities are split into two functions each:

All PAM modules should implement all six functions, even if they don't make any sense for that particular module, so we'll just add dummies that return PAM_SUCCESS.

FreeBSD currently has the following PAM modules:


2001-11-17: En passant


Short entry today, my girlfriend is visiting and has set a quota on tube time, but I'd like to discuss some of the feedback I've received on yesterday's entry.

One thing everybody I've heard from seems to agree on is that a web interface (not just for queries, but for actual maintenance and administration) is a strong requirement, whereas I had placed it in the "would be nice" category. Actually, this suits me fine, as the framework I was thinking of using for the PR system is one I originally wrote for web applications.

I'd like to elaborate on the idea of to-do lists or progress reports. I like the idea of a PR system that allows a committer or PR worker to define a list of tasks that need to be performed before the PR is closed (for instance, "reproduce the bug", "develop a patch", "get feedback from originator", "commit to -CURRENT", "commit to -STABLE"). The system might even automatically close the PR when the last task is completed. One obvious extension of this idea is that a committer can open a PR (assigned to himself) that describes a project he is working on, and use the task list feature to set up a list of milestones, and close each task as the milestone is reached.

Richard Rose suggested a hybrid model for the PR database — an SQL database with references to separate files that contain the actual PR text. XML would only be used as a transport format. Personally, I think distributability (if such a word exists) is more important than performance or ease of implementation. Also, we're looking at a very simple schema, with practically no relations between tables, so SQL won't really give us much. To me, the ability to just add gnats release=current prefix=/usr/local/gnats to your supfile and then use grep(1) and less(1) to browse PRs, or to quickly write simple PR-manipulation tools in Perl without having to install MySQL and a heap of DBI and DBD modules, is more than I'm willing to give up.

There are a few synchronization and performance issues an SQL database would help with, and we'll have to handle those manually, so to say, but they're well-understood problems with straightforward solutions (lock files for synchronization, separate index files for faster searches).

Richard also suggested writing the system in Java, though he mentioned a few caveats that might tip the balance in favor of Perl. I've also thought about Java, which is a great language for certain purposes (and has very good XML support), but the facts that Perl is in our base system whereas Java is not, that Perl is almost consistently faster than Java, and that Perl CGI scripts can run as-is on the FreeBSD Project's web server while Java programs would require installing a JDK plus either the addition of JServ or wrapper scripts to start the virtual machine, all speak in Perl's favor. Not to mention that I already have a non-trivial amount of Perl code I may want to reuse for this project :-)


I'd like to take this opportunity to speak out against the harassment -core have been subject to on the mailing lists these past few days. Yes, some degree of dissention is always good in a democracy, but Poul-Henning's way of going about it is often, and especially in this case, destructive rather than productive. I agree that -core is not perfect — frankly, I think they stink at communicating with the user and developer community, and they really need to get better at that — but I don't believe this is indicative of general incompetence. I think that on the whole, -core is doing an OK job; we just never get a chance to really appreciate it because most of it happens behind closed doors and they haven't sent out monthly status reports like they were supposed to.

People have suggested an early election — well, frankly, do you really think it will do any good? I doubt you'll get many credible candidates other than the current -core members, and once the election is over — at least two months from now — we either end up with practically the same team, only now they're pissed off that we've wasted their time with pointless elections, or we end up with a green team, all wet behind the ears, who have no idea what -core is about or how much effort it requires, and six months later Poul-Henning is back up on his soap box yelling that -core are incompetent and we need an early election.

Some people also suggested doing away with -core altogether. Tcha, you know what? Uh-uh. We need -core, even if it's only to appoint "hats" and serve as a conflict-resolution body. The only alternative I see is holding a referendum every other week or so, and that gets old real quick, as anyone who's ever tried to get more than three committers to agree on something should know. We need a body of elected officials, and if we really don't like the way they're running things, well, we have bylaws which spell out the mechanism for calling an early election: just get one-third of all active committers to sign a petition. But don't do it on a whim. The whole point with electing leaders is continuity and stability, and we'll never get that if we keep calling early elections every six months.


2001-11-16: Divergence

That which we call a rose...

I found out that I enjoyed writing this diary so much I'm going to expand its scope to, basically, "any technical subject I feel like writing about". Obviously, I couldn't go on calling it "The PAM Diaries", so after some thinking and a chat with my girlfriend, I changed the name to "Descrypture". For the lexicologically impaired out there, it's a portmanteau of the words "DES" (my moniker) and "scripture" (meaning "that which is written"), with the "i" changed to a "y" to imply a connection to information security.

Of problem reports

As I mentioned earlier on this page, I have volunteered to take responsibility for FreeBSD's problem report system (a position which Jordan calls "Bugmeister", and Poul-Henning "PR Khan"). There are several things I have in mind that I want to do if I get the assignment. One of them is to set up a better PR statistics system, with regular mailings, to keep people's attention focused on PRs. Another is to create a team of PR screeners whose job it will be to handle bogus PRs, assign PRs to committers, close PRs that time out in feedback state, etc. The third, and (to me) most interesting, is to develop a replacement for GNATS. I can't stop thinking about this issue, and I have a kind of requirements list running through my head that keeps growing and growing, so I thought I'd write it down while I could remember it all.

First of all, there is a list of requirements that grow out of the need for compatibility with our installed base and existing infrastructure:

Then there's everything we'd like to be able to do, except GNATS won't let us:

Finally, there are a few features that Would Be Nice, but we can live without:

Of course, different people will have different opinions of which features are "must have" and which would just be "nice to have", but you can never please everybody. If I get a majority of users and committers to agree that the new system is better than the old, it'll be a huge step in the right direction.

On a lighter note, I just thought of a great name for a bug tracking system: Bugger. The command-line interface would of course be bugger(1), and the command to list all open PRs would be bugger --all. Whaddyall think? :-)


I spent some more time thinking about the PR problem after writing the above section, and came up with a few things I'd missed:

I also have a quite definite vision of the underlying architecture for the system. I won't bring it up here and now — I want to develop it a little further first — but I'll make it the topic of a later Descrypture.

Addendum addendi

Warner asked me to provide Core with a definition of the Bugmeister position. Here's what I wrote:

First, I should point that this came out of a discussion at the Q&A session at BSDCon Europe, where I suddenly found myself standing up with a microphone in my hand, and, not knowing exactly what to do with it, I was about to tap it and say "one two, one two", but I had a Freudian slip and it came out as "I'm willing to take responsibility for the FreeBSD PR system, assuming -core is willing to give me a free hand to do what I deem necessary to resolve the issue".

I see the Bugmeister's responsibilities as:

  1. Define (subject to core's approval) and enforce a consistent policy for handling PRs.
  2. Set goals for the PR system (rate of closing PRs, maximum number of outstanding PRs, average age of outstanding PRs etc.); report on the project's performance relative to those goals; research and implement policies and tools that encourage and facilitate meeting those goals.
  3. Form and oversee a team of PR handlers ("screeners") who are responsible for the maintenance of the PR database (closing bogus or stale PRs etc.)
  4. Develop and maintain, or oversee the development and maintenance of, the PR handling infrastructure.
  5. Assume responsibility for any fuckups.

Note that item 2 partly summarizes items 1, 3 and 4.


2001-11-15: First blood

Fixing the "required" bug

There's a comment at the top of pam.conf that says:

# Each final entry must say "required" -- otherwise, things don't
# work quite right.  If you delete a final entry, be sure to change
# "sufficient" to "required" in the entry before it.

What it doesn't say is precisely what doesn't work quite right, so I started browsing the libpam code with special attention to the dispatcher (the bit that actually goes through the module chain and runs each module), to determine exactly what the difference between a "required" rule and a "sufficient" rule is, from the library's point of view.

It didn't take long to find out what the bug was and what caused it. In pam_handlers.c, the actions for a "sufficient" module are defined as follows:

                actions[PAM_SUCCESS] = _PAM_ACTION_DONE;
                actions[PAM_NEW_AUTHTOK_REQD] = _PAM_ACTION_DONE;
                _pam_set_default_control(actions, _PAM_ACTION_IGNORE);

This means that if a "sufficient" module succeeds (whether or not the user is required to change her password), the dispatcher will immediately stop and return PAM_SUCCESS, and in all other cases, this module will simply be ignored. Fair enough — this is what "sufficient" is supposed to mean. For reference, the actions for a "required" module are defined as follows:

                actions[PAM_SUCCESS] = _PAM_ACTION_OK;
                actions[PAM_NEW_AUTHTOK_REQD] = _PAM_ACTION_OK;
                actions[PAM_IGNORE] = _PAM_ACTION_IGNORE;
                _pam_set_default_control(actions, _PAM_ACTION_BAD);

This means that the failure of a "required" module is interpreted as negative, but does not terminate the chain.

The problem lies in how the dispatcher behaves when it "falls off" the end of the chain. Let's look at a sample module chain for the su service:

su      auth    sufficient      pam_rootok.so   no_warn
su      auth    requisite       pam_wheel.so    no_warn auth_as_self noroot_ok
su      auth    sufficient      pam_unix.so     no_warn try_first_pass nullok

What happens when you try to run su(1)? First, you need to understand a little more about how the dispatcher works. The dispatcher is basically a loop that runs each module in the chain in order and records the result. This loop can terminate early if a module returns a status that resolves to _PAM_ACTION_DONE (e.g. success of a "sufficient" module) or _PAM_ACTION_DIE (e.g. success of a "requisite" module). The result of the entire chain is based on the value of two variables, impression and status. The former indicates which way the wind blows — as in "so far, I think I'll allow this, but I'm not quite sure yet". The latter simply indicates the rationale behind that impression. These variables are initialized to _PAM_UNDEF and PAM_MUST_FAIL_CODE, respectively.

Let's assume a regular user who is a member of wheel runs su(1) to become root. Here's what happens when the dispatcher runs:

  1. The pam_rootok module runs and returns PAM_AUTH_ERR since the current user is not root. This maps to _PAM_ACTION_IGNORE, so nothing happens.
  2. The pam_wheel module runs and returns PAM_SUCCESS, since the current user is a member of the wheel group. This maps to _PAM_ACTION_OK, which sets impression to _PAM_POSITIVE ("OK so far") and status to PAM_SUCCESS.
  3. The pam_unix module runs and asks the user to type her password. She mistypes, and pam_unix returns PAM_AUTH_ERR, which maps to PAM_IGNORE, so nothing happens.
  4. The dispatcher has reached the end of the chain, and looks at impression, which is still _PAM_POSITIVE, so it returns PAM_SUCCESS. Oops.

So how do we fix this? Well, having consulted the original PAM paper (available from Sun's PAM page), I'm not even sure it's a bug. But it's certainly non-intuitive, and can have catastrophic consequences if the admin is not aware of the exact semantics of "sufficient". I see four ways to avoid this pitfall:

  1. Don't change the code, but document the pitfall, and recommend that module chains be terminated with a "requisite" pam_deny.
  2. Hack the dispatcher to recognize this situation and return PAM_AUTH_ERR.
  3. Change pam_wheel to return PAM_IGNORE instead of PAM_SUCCESS.
  4. Add a new control flag (called "necessary", maybe?) that means "terminate on failure, ignore success", to mirror "sufficient" (which has the semantics of "ignore failure, terminate on success").

The root of the problem is that the PAM control flags are poorly named and too large-grained to provide sufficient control of the flow of execution. Like everything in life, it's a trade-off: simplicity versus flexibility. In this case, simplicity has, in my opinion, been carried too far.

Events vs. callbacks

The PAM that we all know and love is based on a callback system, where the application calls PAM library functions, which call functions in the various modules, which sometimes call application-defined conversation functions to ask the user for a password or other authentication token. This works for simple, linear, text-based applications like login(1) and su(1), but it doesn't mix well with event-driven applications like sshd(8). Such applications need an event-driven model, where the application and the module can engage in an asynchronous exchange of information.

Unfortunately, it is very difficult to implement an event model on top of the existing callback model. Eivind Eklund (eivind@freebsd.org) has had some success with sshd(1), using a wrapper library that forks off a child process that handles the callback stuff and communicates with the parent process using IPC, but this is a horrible hack.

On the other hand, had the PAM library been based on the event model in the first place, it would have been trivial to implement the callback model on top of it (for simplicity and compatibility with legacy applications), but no such luck.

I could, of course, rewrite PAM from scratch using an event model, but that would be a lot of work and totally outside the scope of my contract, so unless I can get additional funding (whether from DARPA or from another source) or convince the University of Oslo to let me do it for my master's degree, it won't happen in the foreseeable future. Would be nice, though, especially when you consider what a mess our current PAM implementation (which was derived from Linux-PAM) is.


2001-11-14: The hows and whys

Nothing really interesting today — this is just a quick note to explain how this diary is generated, and why it looks like crap in some browsers.

I write the diary in XML using a schema I created specially for this purpose, basically by just starting to type and marking up the text in what seemed a reasonable way. Then I transform the XML source into XHTML using Sablotron and a custom XSLT stylesheet. The resulting HTML code is totally devoid of layout information — it simply uses DIV and SPAN elements with different CLASS attributes to group parts of the text together and indicate their semantic role, and relies on a CSS1 stylesheet to tell the browser how the various elements and classes are to be formatted.

The net result is that the diary not only looks great in a modern browser, but it should also look OK in any browser without CSS support (including text-only browsers), and present no significant trouble for screen readers or HTML-to-speech converters. Unfortunately, there are browsers out there — like Netscape 4 — that claim to support CSS, but make a total hash out of it. Amongst other bugs, Netscape 4 gets confused by DIV elements and ends up rendering successive DIVs as if they were nested, so if you read this diary using Netscape 4, you're better off with the "Enable Style Sheets" option in the "Advanced" preferences dialog turned off. Sucks to be you. Get a real browser.


2001-11-13: So it begins

BSDCon Europe

I went to BSDCon Europe this weekend, which was a ball — it was great to finally meet all those people I've known for so long but, with a few exceptions, never actually met — Paul, Nik, Brian and Josef, of course, and Poul-Henning, Greg and Mark, and many others. Too bad Robert couldn't make it — I was really looking forward to meeting him too! The conference was a smash hit, three days of non-stop fun and fascinating presentations and most of all lots of socializing. Oh, and it finished with a flourish — I stood up in the middle of a near-plenary session and volunteered to take on responsibility for the FreeBSD bug tracking system. I'm still wondering if that was really such a good idea :-)

I arrived late on Thursday night with a growing headache and a horrible pain down the left side of my neck and down to my left shoulder blade. Joe met me in the lobby and brought me to the terminal room, where Paul, Nik, Brian and he were busy setting up the network. Having played around with MacOS X a little, I offered to assist with the network, and they set me to crimping network cables, as I seemed to be the only one around who had done it before — that, or the others had more important things to do.

Two Ibuprofen, a few hundred feet of Cat 5, ten RJ45 connectors and half a roll of Gaffer tape later, the terminal room was fully wired up and connected to the Internet through twelve bundled ISDN channels, so I hooked up my laptop and sat down to read my email. The first item in my FreeBSD mailbox was the contract from NAI Labs! Finally, more than four months late, all the paperwork had been sorted out and we could get to work.

Chatting with Mark

For obvious reasons, I spent quite a bit of my time in Brighton chatting with Mark Murray about PAM — how it works, how it doesn't, where it's headed and what needs to be done to get it there. I attended his presentation, of course, and late on Sunday — as Mark was getting ready to drive back — we sat down in the lobby and wrote up a to-do list that, after some cleaning up and minor additions, looks like this:

  1. Complete all existing modules so that they all provide all four facilities, even if some are just dummies.
  2. Find and fix the bug that forces us to always end the rule list with a "required" entry.
  3. PAMify the userland, including but not limited to chfn(1), chpass(1), chsh(1), ftpd(1), login(1), ppp(8), sshd(8), su(1) and telnetd(8), as well as a variety of ports, including but not limited to sudo(1), FTP, HTTP, POP and IMAP servers, login dialogs in windowing systems, and various authentication-related add-on modules to scripting languages such as Perl, Ruby and Python.
  4. In parallel with the previous task, add more PAM modules that implement the functionality that these programs require, including but not limited to calling the appropriate login_cap(3) functions and updating accounting databases on login and logout.
  5. Write documentation.
  6. Issue a warning if both pam.conf and pam.d exist.

Papers and presentations

A couple of months ago, I agreed to hold a FreeBSD-related presentation at the December meeting of NUUG, the Norwegian Unix User Group, and selected "FreeBSD as a security platform" as the topic of my presentation. Then, when it became clear that I would be able to afford going to BSDCon Europe (mostly thanks to Josef Karthauser, who generously offered me to crash at his place!), I used that as the topic of my BSDCon Europe talk (and ended up rewriting it from scratch on the morning of the day I was to give it). Since NUUG hadn't yet announced my presentation on their web site, I decided to change the topic and mailed them a new abstract for a presentation entitled "PAM in FreeBSD". I expect that I'll have more interesting things to say about that than I had about "FreeBSD as a security platform".

I'm also considering submitting a paper to BSDCon which is coming up next February, but the deadline for submitting the complete paper is December 4, which might be tight, and I'm not even sure they're still accepting new papers. I won't be able to afford going to BSDCon unless I'm presenting (and possibly not even then), but there are other interesting events coming up in the same timeframe, such as NordU, and later on there's SANE, USENIX and the Security Symposium. The latter is the only one where the deadline for initial submissions hasn't yet expired.

Planning ahead

The contract is for roughly 600 hours, and I'll be doing most of the work myself (Eivind will do some review work as well as handle anything that relates to OpenSSH). This means I might be working with PAM nearly full-time for the next four to six months! The good news is that I can get a lot of work done; the bad news is that I run the risk of getting monumentally tired of PAM. The obvious solution is to vary my tasks, but this means doing non-PAM work in between, which means we're looking at six to eight rather than four to six months.

I don't expect to follow the above to-do list slavishly; I expect that I'll start off trying to familiarize myself with the code and fix any immediate bugs, then take a break from the code to write some basic documentation, and maybe a Dæmon News article on PAM. Then (by mid- or late December), I'll start working on missing functionality (with login(1), su(1) and chpass(1) and friends at the top of my list), writing the associated documentation along the way.

I'll be documenting my work in this diary, both so the community can track my progress and as documentation for the invoices I send to NAI. If it works out well, I'll probably start writing diary entries about FreeBSD work I do as well, and maybe even other Open Source work (I'm mightily tempted to volunteer for Conglomerate... especially as I use XML a lot these days, for papers, slides, reports and even this diary!) I don't know how many people will actually be interested in this diary, but if I get much of a readership, I'm also quite willing to take requests from my readers for topics they'd like to see me discuss.