 |
FreeBSD SysVR4 EmulationMark Newton, newton@atdot.dotat.org |
|
FreeBSD SysVR4 EmulationMark Newton, newton@atdot.dotat.org |
Description
This page describes an SysVR4 emulator for FreeBSD. It is currently
capable of running (or walking, in some cases) a wide-ish variety of
SysV executabls taken from Solaris/x86 2.5.1 and 2.6 systems. I have
reason to believe that it will also run SCO UnixWare and SCO OpenServer
binaries.
The emulator has been produced by a process of distillation, starting with the Linux and iBCS2 emulators from FreeBSD and the SVR4 emulator from NetBSD. HUGE thanks to Christos Zoulas and Todd Vierling for the NetBSD stuff, and David Greenman and Soren Schmidt for the FreeBSD stuff. I haven't so much written this stuff as integrated and debugged it, and couldn't have done it without the efforts of those individuals.
The emulator has some limitations, and needs a lot of work. The most most pressing items at the top of my to-do list are presented below; if you are interested in lending a hand, it'd probably be best to start with those.
If you use it, please provide feedback. If something doesn't work, ktrace output describing the failure would also be appreciated (although I'll get very rude if you bombard my mailbox with 50Mbyte ktrace.out files, so please be considerate).
I've tested the emulator with a variety of hand-rolled test programs and actual utilities from the Solaris 2.5.1 CD. The most complicated shell-based program which runs is "vi". Networking programs such as "telnet" and "ftp" work fine too, which means the SysVR4 STREAMS emulation is at least pretending to be functional.
1999-Jan-28: Committed the LDT patch as a
lead-up to committing the whole emulator. Also cleaned up a crapload
of warnings. There are still other cosmetic nits and major functionality
botches, but it's better now than it was. svr4_sys_getdents64() and
poll() are still broken, incidentally.
1999-Jan-11: I've merged most of Christos Zoulas' changes to the NetBSD
SysVR4 emulator into this one so that it can be relatively functional with
Solaris 2.6 libraries. Changes involved unstaticizing dosetrlimit()
in sys/kern/kern_resource.c, which is taken care of by the updated
installation patch. Despite the update, svr4_sys_poll()
and svr4_sys_getdents64() syscalls don't seem to be working as expected,
so more work needs to be done here. NetBSD's VOP_READDIR and
native poll() syscalls are different enough from FreeBSD's that
they can't JustWork(tm).
1999-Jan-06: The emulator is now a KLD ready for ELF 3.0.x (rather than an LKM, the
FreeBSD 2.2.x kernel loadable object format). See the download
section to get it. You'll need to re-fetch the svr4.tar.gz source module if you have an earlier version of the emulator already installed.
The distribution is broken into three parts. You'll need all
three parts if you expect to get this bucket of bolts working:
Each distribution contains a README file in some semi-obvious location;
The README files document
things you'll need to do to make that sub-part do its thing (for
instance, things you'll need to change in your kernel source tree to
make the "streams" pseudo-device driver work). That information is
reproduced below in the INSTALLATION section.
To make the emulator work you'll need a SysVR4 CD-ROM. I've been
using Solaris CDs for testing, but there shouldn't be anything preventing
SCO binaries from running (crosses fingers -- I've only tried a small
subset of SCO binaries because SCO's habit of putting billions of symlinks
all over the filesystem makes it all too hard to do large-scale testing
at this point). SCO and Sun both give
their x86 versions away for free for educational or private individual
use, so it shouldn't be all that difficult to come across them. See
http://www.sco.com and http://www.sun.com to find out how to get
free copies of licenses and cheap installation media.
If you have everything working, you should be able to run the "hello"
program provided in /usr/src/sys/modules/svr4.
Developers are extremely welcome, and actively encouraged. I've set
up a majordomo mailing list to discuss the technical issues of SVR4
emulation; You can put yourself onto it by sending email to
majordomo@atdot.dotat.org with the text "subscribe freebsd-svr4" in
the BODY of the message.
Eventually this distribution will get rolled into the -current CVS repository.
That'll make installing it sooo much easier!
I have some ideas on how to make that simple, but I'll need time. If
anyone thinks they can assist, let me know. It should be a relatively
light weight project technically speaking, and could be an ideal way for
someone who hasn't hacked kernels before to make a contribution.
My development methodology has been something like, "xxx doesn't work.
Find out what services xxx needs. Emulate them. xxx works now, therefore
those services must be emulated correctly."
While the first three steps are ok, the fourth represents a false
conclusion. More extensive testing than I've carried out really needs
to be done. Call me lazy :-)
In most cases I'm building on a fantastic foundation provided by
Christos Zoulas from the NetBSD Project (this is really just a FreeBSD
port of his code). In some cases, however (parts of signal delivery,
the LKM framework, etc) there is code I've written which has been
tested somewhat less rigourously.
You'll need to make other changes
"All the SysVR4 programs I try say, `ELF binary type not known'"
[ substitute the pathname of your SysV program in place of "program" ]
yes, that means you can't run things straight off a SysV CD-ROM. Cope.
"X11 clients hang or tell me they can't open server :0.0"
"Crudbox 3.1 from Foobar Enterprises doesn't work!"
I don't have huge numbers of SysVR4 apps, so I can't test everything.
At this stage I expect people who submit bug reports to be able to send
'em in with patches. Sorry if this sounds harsh, but if I spend 24 hours
per day working on everyone else's bugs I'll end up like Mike Smith :-)
kdump(1) doesn't seem to be reporting the right system calls
Example: kdump tells you that a Solaris/x86 program has executed
getrusage(0x28113000,0x1ea8). If you grep for "getrusage" in
/sys/kern/syscalls.c, you get:
So grep for 117 in /sys/svr4/syscalls.master:
... which tells you that the Solaris program was really running an
"munmap" system call.
It sucks, but I've spent so much time on the emulator itself that
I haven't finished porting linux_ktrace. I've done most of it; if
anyone wants to take the bits I've done and finish them off, let me
know.
Extract svr4.tar.gz and streams.tar.gz into /usr/src. Then follow
the steps below. They take about half an hour to run through (including
kernel building), and only need to be done once.
You can simplify your task by taking this patch, which
was generated against 3.0-RELEASE. It promotes static symbols to
global scope and edits /sys/conf/files as described below,
and ends up modifying /sys/kern/kern_exit.c, ,
/sys/sys/socketvar.h and /sys/conf/files to do its work
(so you'll need write permission on those files and should expect them to
change as a result of running the patch). You can
install it by running patch -p < svr4patch in /sys.
If you don't trust foreign patches and want to do things by hand, see
below.
The symbols concerned are listed below, along with the files they
belong to. Edit each of the files and "unstaticize" the appropriate
symbols.
Note that function call declarations need to be declared non-static
both in their prototypes and the actual definition of the function.
It's easier and quicker than it sounds.
(it should go just underneath "so_gen_t so_gencnt")
Expect it to compile with quite a few warnings at this stage (anyone who
wants to fix them and submit patches is welcome to do so).
4.0 Type svr4 instead of the
commands above to load the emulator. To make it automatically load
at boot time, set the svr4_enable flag to YES in
The tar archive contains symlinks which expect you to have a Solaris
for Intel CD-ROM mounted on /cdrom -- If that's what you actually
have, congratulations, installation is completed and you can start
seeing what applications you can actually run. If you only have v2.6
or v7 media, stay with us, we're working to improve support for that (2.5.1
and earlier is presently more "capable").
If you have some other OS CD, remove the symlinks and install
that OS's /usr/lib, /lib, and so on within the /compat/svr4 heirarchy.
Make sure you run the SVR4_MAKEDEV script as shown above, though,
otherwise code which tries to do networking or UNIX-domain sockets
won't work.
This will remove the need to brand executables, because it will make the
kernel assume that unbranded executables belong to the SVR4 emulator.
If you get "ELF binary type not known" messages, see the mini-FAQ
above.
News
1999-Jan-30: Committed the rest of the emulator to FreeBSD 4.0-CURRENT.
Blaz Zupan kindly sent me info about a glitch he had with it shortly
thereafter: The emulator will fail to load with an "Exec format error"
if your kernel hasn't been compiled with options KTRACE. Since
you'd be mad to attempt to debug an emulator without options KTRACE
anyway, I don't see that this is a huge problem at present (although we'll
have to fix it eventually).
Lots of #if defined(NOTYET) stuff was removed too -- Solaris 2.6
libraries need a reasonable number of 64-bit system calls to work, so I've
had to bring 'em in. DNS name resolution doesn't work at present, but I
think that's related to the svr4_sys_poll() problem outlined above.
Downloading
IGNORE THIS SECTION IF YOU'RE RUNNING 4.0-CURRENT. ALTERNATIVELY, IF YOU'RE
NOT RUNNING -CURRENT THE CODE BELOW MAY NOT NECSSARILY WORK ANYMORE WITHOUT
A BIT OF MAINTENANCE, SO ONLY ATTEMPT TO INSTALL THE EMULATOR IF YOU KNOW WHAT
YOU'RE DOING.
A kernel pseudo-device which provides services needed for
STREAMS-based networking (unpack in /usr/src)
Alternate download site here
The bulk of the emulator: Source code for the svr4_mod LKM
which emulates a couple of hundred SysVR4 system calls. Includes
a dozen or so .h files which are needed by the streams driver.
(unpack in /usr/src)
Alternate download site here
Things you should put into /compat/svr4 to expect the emulator
to work. (unpack in /compat)
Alternate download site here
Even more alternate: if you have a Solaris 2.6 or Solaris 7 CD-ROM
you'll really want this or this, not the compat_svr4.tar.gz file mentioned above.
To-Do list
Current priorities are listed below, in what I think is their order of
importance (YMMV, though).
I'm told that 2.6 executables used to fail utterly with Bus Error signals
shortly after startup. Sun changed the syscall
conventions used by Solaris between 2.5.1 and 2.6. We've dealt
with that problem by patching FreeBSD's LDT table, but other Solaris
2.6 changes in userland exercise bugs in the emulator (so hostname
resolution doesn't work right and poll() acts bogusly).
getdents64() doesn't work right either - test prints reveal
that the emulator copies dirent structures into process address
space once, but many filenames show up twice! Bizarre.
Oh well, at least filemgr runs now...
NOTE: X windows clients do work, you just can't use :0.0
as your $DISPLAY. Change it to unix:0.0; Solaris wants
to use a named pipe to talk to the X server if you use :0.0, but
XFree86 provides a socket (which can't be open()'ed). If you use
unix:0.0 the Solaris libX11 will use the socket.
Currently disabled with #ifdef (NOTYET) bits. Needs to
be fixed so we can see if Wabi will work...
Signals seem to work pretty well: I've tested alternate signal stacks,
catching and ignoring signals, etc. Part of the implementation of
svr4_sys_context() required redesign of the sendsig() routine and
the sigcode() assembly code, however, and that can probably do with
some verification. That's the most recent bit of work I've done on
it, so it's the bit that has had the least amount of testing. It
seems to work for most cases (i.e.: all the ones I've tried), though.
There should be no reason why a single source tree can't compile
on NetBSD and FreeBSD. I've done something a bit dumb by making my
code diverge from Christos', and to repent I really should fold my
differences back into the NetBSD codebase.
Related to (2) above.
Interface BSD's SysV shared memory emulation with SysVR4 system calls
Daniel O'Connor
Can't offend the style(9) gods, can we? :-p (hi, Bruce!)
Probably lots of other crap. SysV is wierd compared to BSD, after all.THIS SOFTWARE WILL NOT BUILD CLEANLY!!
THIS SOFTWARE WILL NOT BUILD CLEANLY!!
THIS SOFTWARE WILL NOT BUILD CLEANLY!!
to your kernel source code before
it'll work. See "INSTALLATION" below
for more information.Mini-FAQ
If it doesn't build at all
If it complains about undefined symbols, they're probably declared static
in the kernel sources. Grep through /sys/kern/* and /sys/sys/* for the
symbols involved, and change static definitions to non-static. Rebuild
your kernel, recompile the emulator, and see if it works.
A peculiarity of the FreeBSD ELF subsystem is that ELF programs need to
be "branded". Use the following command:
brandelf -f -t SVR4 program
X windows clients do work, you just can't use :0.0
as your $DISPLAY. Change it to unix:0.0; Solaris wants
to use a named pipe to talk to the X server if you use :0.0, but
XFree86 provides a socket (which can't be open()'ed). If you use
unix:0.0 the Solaris libX11 will use the socket.
Unfortunately the aforementioned poll() bug means TCP connections
to X servers hang. Let's just fix poll(), ok?
Tell the mailing list you're working on it; Modify the emulator to make it work; Submit
patches to me.
That's because ktrace(1) is seeing FreeBSD syscall numbers when you
ktrace a SysV executable. To find out what the SysV executable is
really doing, grep for the syscall name provided by ktrace in the
file /sys/kern/syscalls.c. That'll give you the system call number.
Then grep for that number in /sys/svr4/syscalls.master -- that'll
tell you the SysV system call that was really running.
"getrusage", /* 117 = getrusage */
117 NOPROTO SVR4 { int munmap(void *addr, int len); }
[ idea for the future: perhaps kdump should know how to
grovel through kmem to find a syscall_names struct for
each emulator so that you don't need to build a new version
of it for each OS you support... after all, if the names
are in the LKM that provides the emulation, it seems senseless
to duplicate 'em in a kdump utility. I've had some more thoughts on this. ]
Installation
Most of this is already done if you're running 4.0-CURRENT. If
uname -r reports 4.0-CURRENT you should only carry out
the steps marked 4.0.
Symbol Source File M_ZOMBIE sys/kern/kern_exit.c soo_read() sys/kern/sys_socket.c soo_write() sys/kern/sys_socket.c soo_close() sys/kern/sys_socket.c dosetrlimit() sys/kern/kern_resource.c
void *so_emuldata; /* private data for emulators */
dev/streams/streams.c optional streams device-driver
103 streams SysV STREAMS network driver
pseudo-device streams 1
freebsd% cd /sys/modules/svr4
freebsd% make depend
freebsd% make
freebsd# make install
freebsd# kldload /modules/svr4.ko
freebsd# cd /compat/svr4/dev
freebsd# sh SVR4_MAKEDEV all