Skip site navigation (1) Skip section navigation (2)

Introduction

This report covers FreeBSD related projects between June and October 2006. This includes the conclusion of this year's Google Summer of Code with 13 successful students. Some of last year's and the current SoC participants have meanwhile joined the committer ranks, kept working on their projects, and improving FreeBSD in general.

This year's EuroBSDCon in Milan, Italy has meanwhile published an exciting program. Many developers will be there to discuss these current and future projects at the Developer Summit prior the conference. Next year's conference calendar has a new entry - in addition to the now well established BSDCan in Ottawa - AsiaBSDCon will take place in Tokyo at the beginning of March.

As we are closing in on FreeBSD 6.2 release many bugs are being fixed and new features have been MFCed. On the other hand a lot of the projects below already are focusing on FreeBSD 7.0 and promise a lot of exciting news and features to come.

Thanks to all the reporters for the excellent work! We hope you enjoy reading.

Google Summer of Code

Projects

FreeBSD Team Reports

Network Infrastructure

Kernel

Documentation

Userland Programs

Architectures

Ports

Miscellaneous


    Google Summer of Code


    Analyze and Improve the Interrupt Handling Infrastructure

    Links
    SoC Student Wiki URL: http://wiki.freebsd.org/Interrupts

    Contact: Paolo Pisati <[email protected]>
    Contact: John Baldwin <[email protected]>

    This project consisted in the improvement of the Interrupt Handling System in FreeBSD: while retaining backward compatibility with the previous models (FAST and ITHREAD), a new method called 'Interrupt filtering' was added. With interrupt filtering, the interrupt handler is divided into 2 parts: the filter (that checks if the actual interrupt belong to this device) and the ithread (that is scheduled in case some blocking work has to be done). The main benefits of interrupt filtering are:

    • Feedback from filters (the system finally knows if any handler has serviced an interrupt or not, and can react consequently).
    • Lower latency/overhead for shared interrupt line.
    • Previous experiments with interrupt filtering showed an increase in performance against the plain ithread model

    Moreover, during the development of interrupt filtering, some MD dependent code was converted into MI code, PPC was fixed to support multiple FAST handlers per line and an interrupt stray storm detection logic was added. While the framework is done, there are still machine dependent bits to be written (the support for ppc, sparc64, arm and itanium has to be written/reviewed) and a serious analysis of the performance of this model against the previous one is a work-in-progress

    Bundled PXE Installer

    Links
    SoC Student Wiki URL: http://wiki.freebsd.org/MarkusBoelter

    Contact: Markus Boelter <[email protected]>
    Contact: Paul Saab <[email protected]>

    For me, the Google Summer of Code was a new and very exciting experience. I got actively involved in doing Open Source Software and giving something back to the community. Facing some challenges within the project forced me to look behind the scenery of FreeBSD. The result was a better understanding of the overall project. Working with a lot of developers directly also gave a very special spirit to the Google Summer of Code.

    I really enjoyed the time and will continue to work on the project after the deadline. For me, it was a great chance to get involved in active development and not just some scripts and hacks at home. Getting paid for the work was just a small part of the overall feeling.

    Thanks to the people at the FreeBSD Project and Google for the really, really great time!

    Gvirstor

    Links
    gvirstor home page URL: http://wiki.freebsd.org/gvirstor

    Contact: Ivan Voras <[email protected]>

    Gvirstor is a GEOM class providing virtual ("overcommit") storage devices larger than physical available storage, with possibility to add physical storage on-line when the need arises. Current status is that it's done and waiting commit to HEAD, scheduled for some time after 6.2 is released.

    Open tasks:

    1. The project is in need of testing! If you have the equipment and time, please give it a try so possible bugs can be fixed before it goes into -CURRENT.

    IPv6 Stack Vulnerabilities

    Links
    SoC Student Wiki URL: http://wiki.freebsd.org/ClementLecigne
    PCS Library URL: http://pcs.sf.net

    Contact: George Neville-Neil <[email protected]>
    Contact: Clement Lecigne <[email protected]>

    The focus of this project was to review past vulnerabilities, create vulnerability testing tools and to discover new vulnerabilities in the FreeBSD IPv6 stack which is derived from the KAME project code. During the summer Clement took two libraries, the popular libnet, and his mentor's Packet Construction Set (PCS) and created tools to find security problems in the IPv6 code. Several issues were found, bugs filed, and patches created. At the moment Clement and George are editing a 50 page paper that describes the project which will be submitted for conference publication.

    All of the code from the project, including the tools, is online and is described in the paper.

    By all measures, this was a successful project. Both student and mentor gained valuable insight into a previously externally maintained set of code. In addition to the new tools development in this effort, the FreeBSD Project has gained a new developer to help work on the code.

    Jail Resource Limits

    Links
    SoC Student Wiki URL: http://wiki.freebsd.org/JailResourceLimits

    Contact: Chris Jones <[email protected]>
    Contact: Kip Macy <[email protected]>

    We now have support for limiting CPU and memory use in jails. This allows fairer sharing of a systems' resources between divergent uses by preventing one jail from monopolizing the available memory and CPU time, if other users and jails have processes to run.

    The code is currently available as patches against RELENG_6, and Chris is in the process of applying it to -CURRENT. More details can be found at JailResourceLimits on the wiki.

    Open tasks:

    1. Port patches against -CURRENT.

    Nss-LDAP importing and nsswitch subsystem improvement

    Links
    SoC Student Wiki URL: http://wiki.freebsd.org/MichaelBushkov
    Original Project Proposal URL: http://wiki.freebsd.org/LdapCachedOriginalProposal
    Detailed Description of the Completed Project URL: http://wiki.freebsd.org/LdapCachedDetailedDescription

    Contact: Michael Bushkov <[email protected]>
    Contact: Hajimu UMEMOTO <[email protected]>

    The Project consisted of five parts:

    1. Nsswitch modules and libc separation. The idea was to move the source code for different nsswitch sources (such as "files", "dns", "nis") out of the libc into the separate shared libraries. This task was successfully finished and the patch is available.
    2. Regression tests for nsswitch. A set of regression tests to test the correctness of all nsswitch-related functions and the invariance of their behavior between system upgrades. The task can be considered successfully completed, the patch is available.
    3. Rewriting nss_ldap. Though, this task was not clearly mentioned in the original proposal, during the SoC we found it would be easier, not to simply import PADL's nss_ldap, but to rewrite it from scratch (licensing issues were among the basic reasons for this). The resulting module behaves similarly to PADL's module, but has a different architecture that is more flexible. Though it's basically finished, several useful features from the PADL's nss_ldap still need to be implemented. Despite the lack of some features, this task can be considered successfully completed. Missing features will be implemented as soon as possible, hopefully during September.
    4. Importing nss_ldap into the Base System. The task was to prepare a patch, that will allow users to use nss_ldap from the base system. The task was successfully completed (the patch is available), but required importing OpenLDAP into the base in order for nss_ldap to work properly, and it had led to a long discussion in the mailing list. This discussion, however, have concluded with mostly positive opinions about nss_ldap and OpenLDAP importing.
    5. Cached performance optimization. The caching daemon performance needs to be as high as possible in order for cached to be as close (in terms of speed) to "files" nsswitch source as possible. Cached's performance analysis was made and nsswitch database pre-caching was introduced as the optimization. This task was completed (the patch is available). However there is room for improvement. More precise and extensive performance analysis should be made and more optimizations need to be introduces. This will be done in the near future.

    Though none of the code was committed yet into the official FreeBSD tree, my experience from the previous year makes me think that this situation is normal. I hope, that the code will be reviewed and committed in the coming months.

    Porting the seref policy and setools to SEBSD

    Links
    SoC Student Wiki URL: http://wiki.freebsd.org/DongmeiLiu

    Contact: Dongmei Liu <[email protected]>
    Contact: Christian Peron <[email protected]>

    Dongmei Liu spent the summer working on the basic footwork required to port the SEREF policy to SEBSD. This work has been submitted and can be viewed in the soc2006/dongmei_sebsd Perforce branch. This work was originated from the SEBSD branch: //depot/projects/trustedbsd/sebsd. Additionally setools-2.3 was ported from Linux and can be found in contrib/sebsd/setools directory. It is hoped that this work will be merged into the main SEBSD development branch.

    Porting Xen to FreeBSD

    Links
    Step-by-step tutorial for installing and using FreeBSD as domU URL: http://www.yuanjue.net/xen/howto.html
    Wiki page for this project URL: http://wiki.freebsd.org/YuanJue

    Contact: Jue Yuan <[email protected]>

    As a participant of Google's Summer of Code 2006, I am focusing on porting Xen to FreeBSD these months. The result of this summer's work include a domU kernel that could be used for installation, a guide for getting started with FreeBSD on Xen, and some other trivial improvements. But there are still a lot of work needing to be done in this area, e.g, the long-expeted dom0 support. So I will continue my work here and try to keep up with the update of Xen itself.

    Open tasks:

    1. dom0 support is the most urgent

    SNMP monitoring (BSNMP)

    Links
    P4 workspace URL: http://perforce.freebsd.org/depotTreeBrowser.cgi?FSPC=//depot/user/soc%2dshteryana/bsnmp&HIDEDEL=NOe
    SNMP-related pages on FreeBSD Wiki URL: http://wiki.freebsd.org/CategorySNMP
    A wiki page on if_bridge(4) monitoring module URL: http://wiki.freebsd.org/SnmpBridgeModule
    bsnmptools port URL: http://www.freshports.org/net-mgmt/bsnmptools/

    Contact: Shteryana Shopova <[email protected]>

    Contact: Bjoern A. Zeeb <[email protected]>

    A BRIDGE monitoring module for FreeBSD's BSNMP daemon has been implemented. In addition to RFC 4188 single bridge support and extending the kernel to get access to all the information, a private MIB was designed in order to be able to monitor multiple bridges supported by FreeBSD. The kernel part has already been committed to -CURRENT (thanks to thompsa@), for -STABLE a patch is available (see the wiki), code has already been reviewed.

    SoC 2005 work on SNMP client tools is now available too via port (net-mgmt/bsnmptools), thanks to Andrew Pantyukhin for the port.

    Open tasks:

    1. More testing is very welcome.
    2. if_vlan(4) monitoring module.
    3. jail(8) monitoring module.

    Summer of Code Summary

    Links
    FreeBSD Summer of Code 2006 URL: http://www.FreeBSD.org/projects/summerofcode-2006.html
    SoC 2006 Student wiki URL: http://wiki.freebsd.org/SummerOfCode2006
    SoC 2006 Perforce trees URL: http://perforce.freebsd.org/depotTreeBrowser.cgi?FSPC=//depot/projects/soc2006/

    Contact: Murray Stokely <[email protected]>

    We had another successful summer taking part in the Google Summer of Code. By all accounts, the FreeBSD participation in this program was an unqualified success. We received over 150 applications for student projects, amongst which 13 were selected for funding. All successful students received the full $4,500.

    These student projects included security research, improved installation tools, new utilities, and more. Many of the students have continued working on their FreeBSD projects even after the official close of the program. At least 2 of our FreeBSD mentors will be meeting with Google organizers in Mountain View this month to discuss the program at the Mentor Summit.

    Update of the Linux compatibility environment in the kernel

    Links
    Wiki page about the linux compatibility environment. URL: http://wiki.FreeBSD.org/linux-kernel

    Contact: Alexander Leidinger <[email protected]>
    Contact: Roman Divacky <[email protected]>
    Contact: Emulation Mailinglist <[email protected]>

    Roman Divacky participated in the Google Summer of Code 2006 and implemented a major part of the syscall compatibility to the 2.6.16 Linux kernel. The work has been committed to -CURRENT (the default compatibility still being a 2.4.2 Linux kernel) and we are working on fixing the remaining bugs as time permits.

    "Intron" submitted an implementation for the linux aio syscalls. His work has been committed to the Perforce repository.

    We also started to consolidate a list of known bugs, open issues and helpful stuff (e.g. regression tests and their status) in -CURRENT on a page in the FreeBSD wiki (see the links-section). It also contains a link to a more or less up-to-date patch with stuff we have in the Perforce repository so that interested people can help with testing. Thanks to the help of Marcin Cieslak we already fixed some bugs (some of the fixes are already MFCed to -STABLE).

    Thanks to the nice regression tests of the Linux Test Project (LTP) we have a list of small (and not so small) things which need to be looked at. This list makes up for a quick start into kernel hacking. So if you have a little bit of knowledge about C programming, and if you want to help us a little bit in improving FreeBSD, feel free to have a look at the list and to try to fix a problem or two. Sometimes it is as easy as "if (error condition) return Esomething;" (but you should coordinate with the emulation mailinglist, so that nobody does some work someone else just did too). Even if you do not know how to program, you can help. Have a look at the wiki page and tell us about things which should get mentioned there too. Or download the patch and test it.


    Projects


    CScout on the FreeBSD Source Code Base

    Links
    The CScout project page on the FreeBSD wiki. URL: http://wiki.freebsd.org/CScout

    Contact: Diomidis Spinellis <[email protected]>

    CScout is a refactoring editor and source code browser for collections of C code. The aim of the project is to make it easy for FreeBSD developers to use CScout and to improve the FreeBSD source code quality through CScout-based queries and refactorings.

    CScout was first applied to the FreeBSD kernel in 2003. Its application at that point involved substantial tinkering with the build system. The version released in October 2006 makes the running of CScout on the three Tier-1 architectures a fairly straightforward procedure. The current version can also draw a number of call graphs; this might help developers better understand foreign code.

    Open tasks:

    1. Use CScout to locate problematic code areas (for example unused or too liberaly visible objects).
    2. Use CScout to globaly rename identifiers in a more consistent fashion.
    3. Apply CScout to the userland code.
    4. Identify CScout extensions that would help us improve the quality of our code.
    5. Arrange for the continuous availability of a live CScout kernel session on the current version of the source code.

    DTrace

    Links

    Contact: John Birrell <[email protected]>

    Progress this month has been limited due to my sea-change, moving house to the country.

    Sun's OpenSolaris developers have followed through and released the DTrace test suite as part of the OpenSolaris distribution.

    jkoshy@'s work on libbsdelf is nearing feature completion for DTrace and will make life easier in FreeBSD for DTrace, given that we have more architectures to support than Sun has.

    The FreeBSD project has made available a dual processor AMD64 machine for DTrace porting.

    I am currently working through the diffs between the DTrace project in P4 and -current, committing files to -current if they are ready.

    Embedded FreeBSD

    Links
    URL: http://www.embeddedfreebsd.org/

    Contact: George Neville-Neil <[email protected]>

    Moved the HTML pages into the project CVS tree.

    Open tasks:

    1. Setup the web site to be served from projects CVS so that it can be updated by others.
    2. Complete the ARM port.
    3. Work on the MIPS port.
    4. Update the documentation to include common tasks for embedded engineers.

    FreeSBIE

    Links
    FreeSBIE Website URL: http://www.FreeSBIE.org
    FreeSBIE ML Subscription Form URL: http://liste.gufi.org/mailman/listinfo/freesbie
    FreeSBIE GMV Announcement URL: http://people.freebsd.org/~matteo/GMV/GMVAnnounce.txt

    Contact: FreeSBIE Staff <[email protected]>
    Contact: Matteo Riondato <[email protected]>

    FreeSBIE is a FreeBSD based LiveCD.

    On August 19th, Matteo Riondato, a member of the FreeSBIE staff, released an unofficial ISO, codename FreeSBIE GMV, based on FreeBSD -CURRENT (read the Announcement to download it). This is supposed to be the first in a series of four ISOs that will end up with the release of FreeSBIE 2.0. Matteo is now working on another ISO, codename FreeSBIE LVC, which is scheduled to be released October 12th.

    FreeSBIE 2.0 will be based on FreeBSD 6.2-RELEASE and will hopefully be released at EuroBSDCon 2006 in Milan. It will be available for the i386 and AMD64 platforms.

    Open tasks:

    1. Test the released ISO in preparation for the release.
    2. Suggest software to include in the ISO.
    3. Submit a simple and clear but complete fluxbox configuration.

    GJournal

    Links
    Patches against HEAD. URL: http://people.freebsd.org/~pjd/patches/gjournal_20060930.patch
    Patches against RELENG_6. URL: http://people.freebsd.org/~pjd/patches/gjournal6_20060930.patch

    Contact: Pawel Jakub Dawidek <[email protected]>

    GJournal seems to be finished. I fixed the last serious bug and it is now stable and reliable in our tests. I'm planning to commit it really soon now.

    The work was sponsored by home.pl

    iSCSI Initiator

    Links
    URL: ftp://ftp.cs.huji.ac.il/users/danny/freebsd/iscsi-17.5.tar.bz2

    Contact: Damiel Braniss <[email protected]>

    This iSCSI initiator kernel module and its companion control program are still under development, but the main parts are working.

    Open tasks:

    1. Network Disconnect Recovery.
    2. Sysctl Interface and Instrumentation.
    3. Rewrite the userland side of iscontrol.

    Porting ZFS to FreeBSD

    Links
    Source code. URL: http://perforce.freebsd.org/depotTreeBrowser.cgi?FSPC=//depot/user/pjd/zfs
    ZFS porting site. URL: http://www.opensolaris.org/os/community/zfs/porting/
    ZFS port announce. URL: http://docs.freebsd.org/cgi/mid.cgi?20060822104516.GB16033

    Contact: Pawel Jakub Dawidek <[email protected]>

    My work is moving slowly forward. ZVOL is, I believe, fully functional (I recently fixed snapshots and clones on zvols), which means you can put UFS on top of RAID-Z volume, take a snapshot of the volume, clone it if needed, etc. Very cool. The hardest part is the ZPL layer, I'm still working on it. Most file system methods work, but probably need detailed review and many fixes. Most of the time these days I'm spending on implementing mmap(2) correctly. It works more or less in simple tests but fails under fsx program. On the other hand, 'fsx -RW' works very stable and reliable. Other test programs (those that don't use mmap(2)) also work quite well. There is still a lot of work to do, mostly in ZPL area, many clean-ups, etc. Some functionality (like ACLs) I haven't even tried to touch yet.

    Summer of FreeBSD security development

    Links
    URL: http://people.freebsd.org/~cperciva/funding.html
    URL: http://www.daemonology.net/freebsd-upgrade-6.0-to-6.1/

    Contact: Colin Percival <[email protected]>

    I spent the months of May through August working on improving Portsnap, FreeBSD Update, and devoting more time to my (continuing) role as Security Officer. FreeBSD Update is now part of the FreeBSD base system and is fully supported by the FreeBSD Security Team; updates are currently only being built for the i386 architecture, but AMD64 updates will become available soon.

    In an attempt to reduce the number of people running out of date (and unsupported) FreeBSD releases, I wrote an automatic binary upgrade script for upgrading systems from FreeBSD 6.0 to FreeBSD 6.1; I will be releasing a new script for upgrading to FreeBSD 6.2-(RC*|RELEASE) soon (possibly before this status report is published).

    Further improvements to Portsnap are still ongoing.

    TrustedBSD Audit

    Links
    TrustedBSD Audit Page URL: http://www.TrustedBSD.org/audit.html
    OpenBSM Page URL: http://www.OpenBSM.org/

    Contact: Robert Watson <[email protected]>
    Contact: Christian Peron <[email protected]>
    Contact: Wayne Salamon <[email protected]>

    The TrustedBSD audit implementation provides fine-grained security event logging throughout the FreeBSD operating system. The big news for the last quarter is that the TrustedBSD audit implementation has been merged into RELENG_6 branch, and appeared in 6.2-BETA2. Over the past few months, work has also occurred in the following areas:

    • OpenBSM 1.0 alpha 8 through alpha 12 have been released and merged into FreeBSD CVS. Changes include significant numbers of bug fixes, documentation improvements, and feature enhancements. These include regular expression based matching for auditreduce, auditd management of kernel audit policy (such as maximum trail file size), improvements in printing support for a variety of tokens including execve argument support.
    • Significant enhancements to the FreeBSD Handbook chapter on Audit.
    • Full audit support for execve events, including optional auditing of command line arguments and environmental variables, as well as audit support for a broad range of other additional kernel events.
    • Kqueue support for audit pipes.
    • Robustness improvements in the presence of low disk space conditions.
    • Support for system call capture on additional platforms, such as ppc and ia64.
    • Improved support for very large audit record sizes (as required for extensive execve support).
    • id(1) now supports a -A argument to query audit state for the process.
    • An audit_warn(5) event for trail rotation, which can be used for archiving, reduction, and other administrative activities.

    Lots of testing as part of the 6.2-BETA cycle would be much appreciated. Audit support will be considered an experimental feature in FreeBSD 6.2-RELEASE, but we hope that it will be a production feature in 6.3-RELEASE.

    Open tasks:

    1. Continue expanding auditing of syscall arguments.
    2. Continue expanding auditing of administrative tools.
    3. More testing!
    4. Continue to explore improvements of the administrative model for audit trails, etc.

    USB

    Links
    Current USB files URL: http://perforce.freebsd.org/depotTreeBrowser.cgi?FSPC=//depot/projects/usb/src/sys/dev/usb&HIDEDEL=NO
    My USB homepage URL: http://www.turbocat.net/~hselasky/usb4bsd

    Contact: Hans Petter Sirevaag Selasky <[email protected]>

    During the last three months I have finished reworking nearly all USB device drivers found in FreeBSD-7-CURRENT. Only two USB drivers are left and that is ubser(4) and slhci. Some still use Giant, but most have been brought out of Giant. At the moment I am looking for testers that can test the various USB device drivers. Some have already been tested, and confirmed to work, while others have problems which need to be fixed. If you want to test, checkout the USB perforce tree or download the SVN version of the USB driver that is available on my homepage. At the moment the tarballs are a little out of date.

    Ideas and comments with regard to the new USB API are welcome at: [email protected].


    FreeBSD Team Reports


    FreeBSD Security Officer and Security Team

    Links
    URL: http://www.freebsd.org/security/
    URL: http://www.freebsd.org/administration.html#t-secteam
    URL: http://vuxml.freebsd.org/

    Contact: Security Officer <[email protected]>
    Contact: Security Team <[email protected]>

    In the time since the last status report, six security advisories have been issued concerning problems in the base system of FreeBSD; of these, five problems were in "contributed" code, while one was in code maintained within FreeBSD. The Vulnerabilities and Exposures Markup Language (VuXML) document has continued to be updated by the Security Team and Ports Committers documenting new vulnerabilities in the FreeBSD Ports Collection; since the last status report, 57 new entries have been added, bringing the total up to 814.

    The following FreeBSD releases are supported by the FreeBSD Security Team: FreeBSD 4.11, FreeBSD 5.3, FreeBSD 5.4, FreeBSD 5.5, FreeBSD 6.0, and FreeBSD 6.1. The respective End of Life dates of supported releases are listed on the web site; of particular note, FreeBSD 5.3 and FreeBSD 5.4 will cease to be supported at the end of October 2006, while FreeBSD 6.0 will cease to be supported at the end of November 2006 (or possibly a short time thereafter in order to allow time for upgrades to the upcoming FreeBSD 6.2).

    Ports Collection

    Links
    The FreeBSD Ports Collection URL: http://www.freebsd.org/ports/
    Contributing to the FreeBSD Ports Collection URL: http://www.freebsd.org/doc/en_US.ISO8859-1/articles/contributing-ports/
    FreeBSD ports unfetchable distfile survey (Bill Fenner's report) URL: http://people.freebsd.org/~fenner/portsurvey/
    FreeBSD ports monitoring system URL: http://portsmon.FreeBSD.org/index.html
    The FreeBSD Ports Management Team URL: http://www.freebsd.org/portmgr/index.html
    marcuscom tinderbox URL: http://tinderbox.marcuscom.com/

    Contact: Mark Linimon <[email protected]>

    The ports PRs surged (especially due to a large number of new port submissions), but with some hard work we have been able to get back down to around 900. We are rapidly approaching 16,000 ports.

    Due to this acceleration in adding new ports, portmgr is now very concerned that we are outstripping the capacity of both the build infrastructure and our volunteers to keep up with build errors and port updates. Accordingly, we've added a guideline (not a rule) that ports should be of more than just theoretical use to be added to the Ports Collection (e.g. we can't support all of CPAN + all of Sourceforge + everything else). Basically, use common sense as a guideline; certainly no one wants to see any kind of "gateway" procedure to get incoming ports approved.

    Seven sets of changes have been added to the infrastructure, mostly refactoring and bugfixing.

    As part of a Summer of Code project, we have also incorporated some of gabor@'s changes to incorporate better DESTDIR support. However, due to some unanticipated side-effects, more work is going to be needed in this area. gabor@ is continuing to work on the changes.

    netchild@ and bsam@ have been doing a great deal of work to bring the linux emulator ports closer to sanity, including bringing up a regression-test suite.

    The long-anticipated import of X.Org 7 has stalled due to developer time, mostly to deal with documentation and upgrade instructions. Hopefully this can get done in the early 6.3 development cycle. See the wiki for more information.

    As a part of that work, the decision has been made to move away from using X11BASE and just put everything into LOCALBASE; /usr/X11R6 is simply an artifact at this point. A plan for a transition process is underway; a great deal of testing will need to be done, but in the end the ports tree will be much cleaner. The GNOME team has already done the work to move all of their ports over, and it will be incorporated after the 6.2 release is shipped.

    tmclaugh@ is looking for someone to take over the C# ports. He has maintained them for over a year and wants more time to be able to work on other projects.

    Some work has been done to get rid of FreeBSD 2.X cruft in ports. Further work is needed to get the 3.X cruft removed.

    linimon@ did another pass through resetting inactive maintainers. Another list is waiting in the wings.

    linimon@ is also working on adding the ability for portsmon to analyze successful packages (not just failed ones), so that queries such as "show me packages that build on i386 but not amd64" and "show me why dependent package foo was not built on bar". This is currently in alpha testing.

    We have added 4 new committers since the last report.

    Open tasks:

    1. We still need help getting back to our modern low of 500 PRs.
    2. We have nearly 4400 unmaintained ports (see, for instance, the list on portsmon ). Although there has been a welcome upsurge in new maintainers recently which has dropped the percentage down below 28%, we still need much more help.
    3. A test run of gcc4.1 on the ports tree showed around 1000 new build errors. Kris@ has posted some results so that people can start working on the problems now. In particular, it seems that certain older versions of GCC cannot be built with GCC 4.1, so ports that depend on those older versions are going to have to be fixed as well. Although the import of GCC 4.1 to -CURRENT is not imminent, the time to start planning is now.
    4. The state of the packages on AMD64 and sparc64 significantly lags that of i386. In many of these cases, packages are not attempted because NOT_FOR_ARCH is used instead of more accurately only setting BROKEN based on ARCH. (pointyhat can be forced to build packages that are marked BROKEN, but not NOT_FOR_ARCH). NOT_FOR_ARCH is supposed to denote only "will never work on this ARCH". Although we have volunteers who have expressed interest in sparc64 (and ia64), we need more people who are running amd64 (especially as a desktop) to help us get more packages working.

    Release Engineering

    Links
    URL: http://www.FreeBSD.org/releng/
    URL: http://www.FreeBSD.org/releases/
    URL: http://www.FreeBSD.org/snapshots/

    Contact: Release Engineering Team <[email protected]>

    The FreeBSD Release Engineering team is currently working on FreeBSD 6.2-RELEASE, which is scheduled for release in early November 2006. Some notable features of this release include the debut of security event auditing as an experimental feature, Xbox support, the FreeBSD Update binary updating utility, and of course many fixes and updates for existing programs. Pre-release images for all Tier-1 architectures are available for testing now; feedback on these builds is greatly appreciated. More information about release engineering activities can be found at the links above.

    The FreeBSD Foundation

    Links
    URL: http://www.freebsdfoundation.org

    Contact: Deb Goodkin <[email protected]>

    The FreeBSD Foundation continued to support the FreeBSD project and community through various activities. These activities include creating strategies for fund development and actively seeking funding for the FreeBSD community, coordinating a new IBM Bladeserver project, and protecting the image and integrity of FreeBSD by governing the use of the trademarks. We are pleased to be a sponsor of EuroBSDCon and will be sponsoring a few developers to attend the conference through our travel grant program. And finally, we have secured funds for a major project that will be announced later this month.


    Network Infrastructure


    Bridge Spanning Tree Protocol Improvements

    Contact: Andrew Thompson <[email protected]>

    Work is almost finished to implement the Rapid Spanning Tree Protocol (RSTP) which supersedes Spanning Tree Protocol (STP). RSTP has a much faster link failover time of around one second compared to 30-60 seconds for STP, this is very important on modern networks. The code will be posted shortly for testing and feedback.

    FAST_IPSEC Upgrade

    Links
    CURRENT patch to enable FAST_IPSEC and IPv6 URL: www.freebsd.org/~gnn/fast_ipv6.patch

    Contact: George Neville-Neil <[email protected]>
    Contact: Bjoern Zeeb <[email protected]>

    First working version of code. Does not pass all TAHI tests, but does pass packets correctly and does not panic.

    Open tasks:

    1. More testing of the patch needed.

    Highly improved implementations of sendfile(2), sosend_*() and soreceive_stream()

    Links
    sendfile(2) patch with detailed performance figures URL: http://lists.freebsd.org/pipermail/freebsd-current/2006-September/065997.html
    sosend_*() patch with detailed performance figures URL: http://lists.freebsd.org/pipermail/freebsd-current/2006-September/066199.html
    Combined sendfile(2), sosend_*() and soreceive_stream() patch URL: http://people.freebsd.org/~andre/sendfile+sosend+soreceive-20061006.diff

    Contact: Andre Oppermann <[email protected]>

    The addition of TSO (TCP Segmentation Offload) has highlighted some shortcomings in the sendfile(2) and sosend_*() kernel implementations.

    The current sendfile(2) code simply loops over the file, turns each 4K page into an mbuf and sends it off. This has the effect that TSO can only generate 2 packets per send instead of up to 44 at its maximum of 64K. kern_sendfile() has been rewritten to work in two loops, the inner which turns as many pages into mbufs as it can -- up to the free send socket buffer space. The outer loop then drops the whole mbuf chain into the send socket buffer, calls tcp_output() on it and then waits until 50% of the socket buffer are free again to repeat the cycle. This way tcp_output() gets the full amount of data to work with and can issue up to 64K sends for TSO to chop up in the network adapter without using any CPU cycles. Thus it gets very efficient especially with the readahead the VM and I/O system do.

    Looking at the benchmarks we see some very nice improvements: 181% faster with new sendfile vs. old sendfile (non-TSO), 570% faster with new sendfile vs. old sendfile (TSO).

    The current sosend_*() code uses a sosend_copyin() function that loops over the supplied struct uio and does interleaved mbuf allocations and uiomove() calls. m_getm() has been rewritten to be simpler and to allocate PAGE_SIZE sized jumbo mbuf clusters (4k on most architectures). m_uiotombuf() has been rewritten to use the new m_getm() to obtain all mbuf space in one go. It then loops over it and copies the data into the mbufs by using uiomove(). sosend_dgram() and sosend_generic() have been changed to use m_uiotombuf() instead of sosend_copyin().

    Looking at the benchmarks we see some very nice improvements: 290% faster with new sosend vs. old sosend (non-TSO), 280% faster with new sosend vs. old sosend (TSO).

    Newly written is a specific soreceive_stream() function for stream protocols (primarily TCP) that does only one socket buffer lock per socket read instead of one per data mbuf copied to userland. When doing netperf tests with WITNESS (full lock tracking and validation enabled) the receive performance increases from ~360Mbit/s to ~520Mbit/s. Without WITNESS I could not measure any statistically significant improvement on a otherwise unloaded machine. The reason is two-fold: 1) per packet we do a wakeup and readv() is pretty much as many times as packets come it, thus the general overhead dominates; 2) the packet input path has a pretty high overhead too. On heavily loaded machines which do a lot of high speed receives a performance increase should be measureable.

    The patches are scheduled to be committed to FreeBSD-current at end of October or early November 2006.

    This work was sponsored by the TCP/IP Optimization Fundraiser 2005.

    SCTP Integration

    Links
    URL: http://www.sctp.org/

    Contact: Randall Stewart <[email protected]>
    Contact: George Neville-Neil <[email protected]>

    There are currently patches available for testing. A planned integration to HEAD is set to happen in October.

    Open tasks:

    1. The code still needs plenty of testing. See patches on sctp.org and in -CURRENT soon.

    TSO - TCP Segmentation Offload committed

    Links
    TSO commit to tcp_output.c URL: http://lists.freebsd.org/pipermail/cvs-src/2006-September/068524.html
    TSO em(4) hardware support URL: http://lists.freebsd.org/pipermail/cvs-src/2006-September/068610.html
    Enhanced em(4) TSO hw setup for IPv6 and future protocols URL: http://lists.freebsd.org/pipermail/cvs-src/2006-September/069493.html

    Contact: Andre Oppermann <[email protected]>

    TSO - TCP Segmentation Offload support has been committed to the network stack of FreeBSD-current in September 2006. With TSO, TCP can send data in the send socket buffer in bulk down to the network card which then does the splitting into MTU sized packets. On bulk high speed sending the performance is increased by 25% (normal writes) to 108% (sendfile). Jack Vogel and Prafulla Deuskar of Intel committed the driver changes for TSO hardware support of em(4) based network cards.

    These changes are scheduled to be backported to FreeBSD 6-STABLE shortly after FreeBSD 6.2-RELEASE is published to appear in upcoming FreeBSD 6.3 early next year.

    This work was sponsored by the TCP/IP Optimization Fundraiser 2005.

    Open tasks:


      Kernel


      Gvinum improvements

      Links
      URL: http://folk.ntnu.no/lulf/patches/freebsd/gvinum/gvinum_all_current.diff

      Contact: Ulf Lilleengen <[email protected]>

      I thought that since I sent a status report the last time, I might as well send one now.

      Since the last status report I have done work on several of the remaining commands as attach, detach, and finally the concat command to be able to create concatenated volumes with one easy command. The mirror and stripe commands are the next step after this.

      The most important thing I've been working on is maybe the implementation of drivegroups. I have posted a bit information on this mailinglists, but basically, it's a way to group drives with the same configuration. This way, you can make many commands operate on groups instead of drives, and the group-abstraction will handle how the underlying subdisks are created on the drives. In the future one will be able to move groups to different machines, etc.

      I've created a patch of all my work that is not in HEAD yet here (this is a snapshot of my development branch, so how thing's are done might be changed quite fast): http://folk.ntnu.no/lulf/patches/freebsd/gvinum/gvinum_all_current.diff

      Be aware that a there will probably be bugs in the code, so don't use it in production yet!

      Thanks to Greg Lehey for offering to help me on getting this into CVS, and all feedback on this has been good.

      Open tasks:

      1. Remaining components, mirror, stripe and some info commands.

      MMC/SD Support

      Links

      Contact: Warner Losh <[email protected]>
      Contact: Bernd Walter <[email protected]>

      The MMC/SD stack got a significant boost this quarter. Warner Losh and Bernd Walter have written a generic MMC/SD flash card stack for FreeBSD, and have implemented a host controller for the AT91RM9200 embedded ARM controller they are each using in separate projects.

      The stack is presently experimental in quality. It is being used as the root file system for these embedded projects. There's been no work done to support hot insertion and removal of cards (neither board wires up the pins necessary, and besides, / disappearing is very bad). There are still many rough edges.

      This is a freshly written stack. It has been written using the SD 1.0 (and recently 2.0) simplified specification, with the SanDisk MMC application notes supplementing. The Linux stack looks good, although not entirely standards conforming (there's work in progress that I've not seen that is supposed to fix this) and it is contaminated with the GPL. The OpenBSD stack also looks interesting, but Warner's experience porting NEWCARD over from NetBSD suggested that a fresh rewrite may be faster, at least for the bus and driver level. Since MMC is fairly simple, a port of the sdhci driver might be possible.

      Please see the open tasks list.

      Open tasks:

      1. Write sdhci driver, and integrate it into the current stack.
      2. Add support for hot plugging of cards.
      3. Add support for MMC cards (SD cards were the first target).
      4. Expand SD support to include SDIO cards as well as the new SDHC standard cards.
      5. Export stats via sysctl for each of the cards that are found as a debugging and usage monitoring aid.
      6. Add support for reading/writing multiple blocks at a time to improve performance.
      7. Implement any other host controller.
      8. Add proper support for timeouts.

      Sound Subsystem Improvements

      Links
      Some patches. URL: http://people.FreeBSD.org/~ariff/
      The FreeBSD Project Ideas List. URL: http://www.FreeBSD.org/projects/ideas/
      Wiki page about the sound system. URL: http://wiki.FreeBSD.org/soundsystem

      Contact: Ariff Abdullah <[email protected]>
      Contact: Alexander Leidinger <[email protected]>
      Contact: Ryan Beasley <[email protected]>
      Contact: Multimedia Mailinglist <[email protected]>

      Since the last status report we added basic support for envy24ht chips, imported the emu10kx driver into the base system and added support for High Definition Audio (HDA) compatible chips.

      Additionally the work of Ryan Beasley as part of his Google Summer of Code 2006 participation is committed. It adds compatibility to the Open Sound System (OSS) v4 API as far as this was possible. This allows for more sophisticated programs to be written. For example it is now possible to synchronize the start of multiple sound channels. It is also possible for a driver to support more than the AC97 mixer devices, but so far no driver has been extended to support this yet. More about it can be found in the wiki and in the official OSS documentation.

      The wiki page about the sound system was started to describe the current status of the sound system and to provide some information about where we are heading. But more work needs to be done to reach this goal. So far we collected some information about the status of the most recent work in the soundsystem. So if you have a look at it and you think that something important is missing, just tell us about it. While fully prepared content is very welcome, we are even happy about some ideas what we should list on the wiki page.

      Open tasks:

      1. Have a look at the sound related entries on the ideas list.
      2. sndctl(1): tool to control non-mixer parts of the sound system (e.g. spdif switching, virtual-3D effects) by an user (instead of the sysctl approach in -current); pcmplay(1), pcmrec(1), pcmutil(1).
      3. Plugable FEEDER infrastructure. For ease of debugging various feeder stuff and/or as userland library and test suite.
      4. Extend the wiki page.

      Documentation


      Chinese (Simplified) Project

      Links
      Latest snapshot for translated website URL: http://cnsnap.cn.FreeBSD.org/zh_CN/
      Latest snapshot for translated documentation URL: http://cnsnap.cn.FreeBSD.org/doc/zh_CN.GB2312/

      Contact: Xin LI <[email protected]>

      In the previous quarter we primarily focused on overall quality of the translation rather than just increasing the number of translations, and we have strived to make sure that these translated stuff are up-to-date with their English revisions. Also, we have merged the translated website into the central repository.

      In the next quarter we will focus on developing documentation that will help to attract more developers.

      Open tasks:

      1. Translate more development related documentation.
      2. Review more of the currently translated documentation.

      Hungarian translation of the webpages

      Links
      Snapshot URL: http://gabor.t-hosting.hu/data/hu/

      Contact: G�bor K�vesd�n <[email protected]>

      Since the last status report, there has been a lot of progress. I investigated a lot of charset issues and found out that HTML tidy breaks some entities when using iso-8859-2, so HTML tidy had to be disabled for Hungarian pages.

      Open tasks:

      1. Translate 4 pages.
      2. Review, fix typos and improve the wording where necessary.

      Userland Programs


      Libelf

      Links
      Wiki page tracking LibELF URL: http://wiki.freebsd.org/LibElf
      Wiki page for PmcTools URL: http://wiki.freebsd.org/PmcTools
      PMC Tools Project URL: http://people.freebsd.org/~jkoshy/projects/perf-measurement/

      Contact: Joseph Koshy <[email protected]>

      Libelf is a BSD-licensed library for ELF parsing & manipulation implementing the SysV/SVR4 (g)ELF[3] API.

      Current status: Implementation of the library is nearly complete. A TET-based test suite for the API is being worked on.

      Open tasks:

      1. Reviewers are needed for the code and the test suite. If you have extensions to the stock SysV/SVR4 ELF(3) API that you would like to see in -lelf, please send Joseph an email.

      OpenBSD dhclient

      Links

      Contact: Brooks Davis <[email protected]>

      Most dhclient changes in HEAD have been merged to 6-STABLE for 6.2-RELEASE. The highlight of these changes is a fix for runaway dhclient processes when packets are not 4 byte aligned. Further changes including always sending client identifiers are scheduled for merge before the release. Work is ongoing to improve dhclient's interaction with alternate methods of setting interface addresses.


      Architectures


      CPU Microcode Update Software

      Contact: Stanislav Sedov <[email protected]>

      Last month I was working on a driver/module to update the microcode of Intel or AMD CPUs that support having their microcode updated. As you might know these processors are microcode-driven and this firmware can be updated. Intel(R) often releases microcode updates, and AMD(R) updates can be found in BIOS programs. The work is almost finished now, I just need to find a bit of time to test it on AMD64 systems and perform some code cleanup. The driver also provide a way for userland programs to access the Machine Specific Registers (MSR) and CPUID info for a certain cpu. This will allow some programs like x86info to provide more accurate information about cpus in SMP systems and make assumptions based on the contents of the MSR.

      Thanks to John Baldwin, Kostik Belousov, John-Mark Gurney and Divacky Roman for helping during development.

      Open tasks:

      1. Perform testing on the AMD64-based systems.
      2. Write manpage.
      3. Code cleanup/checks.

      FreeBSD/arm on Atmel AT91RM9200

      Links

      Contact: Warner Losh <[email protected]>
      Contact: Olivier Houchard <[email protected]>

      The FreeBSD/arm port has grown support for the Atmel AT91RM9200. Boards based on this machine are booting to multiuser off either NFS or an SD card. The onboard serial ports, PIO, ethernet and SD/MMC card controllers are well supported. Support for the SSC, IIC and SPI flash parts in the kernel will be forthcoming shortly.

      In addition to normal kernel support, the port includes a boot loader that can initialize memory and boot off IIC eeprom, SPI DataFlash, BOOTP/TFTP and SD memory cards.

      The port will be included in forthcoming commercial products.

      Open tasks:

      1. Add support for other members of the AT91 family of arm9 processors.
      2. Finish support for AT45D* flash parts.
      3. Finish support for USB ports
      4. Write support for USB Device functionality

      Sun Niagara port

      Contact: Kip Macy <[email protected]>

      Support for the UltraSparc T1 (Niagara) continues to improve. The code has recently been checked into public CVS under sys/sun4v.

      It isn't clear whether or not I will have time to implement full logical domaining support before the APIs become publicly available. Testing indicates that substantial work will be needed before FreeBSD can take full advantage of all 32 threads.

      Open tasks:

      1. Random testing and bug fixes.
      2. Import and extend improved mutex profiling support.
      3. Virtual network and virtual disk device drivers for logical domains.

      Xen Port

      Contact: Kip Macy <[email protected]>

      Work on Xen support has slowly been continuing in perforce. The SOC student fixed several bugs and is continuing to work on it. Someone is needed who has the time to complete dom0 support and shepherd it production level stability.

      Sufficient interest has been expressed in it that it probably makes sense to check it in to public CVS so that more people can try it out. Time permitting, I will bring it up to date and check it in the next month.

      Open tasks:

      1. dom0 support.
      2. General testing and bug fixing.

      Ports


      Enlightenment DR17 support in the ports tree

      Contact: Stanislav Sedov <[email protected]>

      Integration of the new innovative e17 window manager into the ports tree is almost completed. A lot of new e17-related applications was ported, all old ports were updated to the latest stable cvs snapshot. The special framework (bsd.efl.mk) was created to support the whole thing and simplify the creation of dependent ports. I'll commit the changes in the days before the ports freeze.

      Thanks to Sergey Matveychuk (sem@) for providing a machine to place CVS snapshots on. Without his help it will be impossible.

      Open tasks:

      1. Port Entrance (xdm-like app, but very appealing).
      2. Port Net and Wlan e17 module.
      3. Develop FreeBSD-specific e17 apps/modules to use The Ports Collection, system configs, etc.

      FreshPorts

      Links
      FreshPorts - The Place For Ports URL: http://www.freshports.org/

      Contact: Dan Langille <[email protected]>

      The new 2U server mentioned in the last report now has a collection of Raptor drives in a RAID-10 configuration. Thanks to very generous donations from the community, I purchased eight of these drives at very good prices. The server will be deployed in the next few weeks.

      There has been quite a bit of work since the last report in June. Some highlights include:

      • New news feed formats, including newsfeeds for your watch list.
      • Better pages caching for faster response.
      • Sanity Test Failures now available online.
      • Ability to search for all commits (ports, doc, src, etc) under a given point in the tree.

      For more detail, please review the FreshPorts Blog .

      Improving FreeBSD Ports Collection Infrastructure

      Links
      G�bors wiki page. URL: http://wiki.freebsd.org/G%C3%A1borK%C3%B6vesd%C3%A1n

      Contact: G�bor K�vesd�n <[email protected]>

      Contact: Erwin Lansing <[email protected]>

      During the Google Summer of Code 2006, G�bor worked on several ideas to improve the ports infrastructure:

      1. New handling for i386 binary ports.
      2. Cleanup: use ECHO_CMD and ECHO_MSG in bsd.port.mk properly.
      3. Add basic infrastructure support for debugging.
      4. Installing ports with different destination (DESTDIR macro).
      5. Cleanup: Move fetch shell scripts out of bsd.port.mk.
      6. Make ports respect CC and CFLAGS.
      7. Cross-compiling Ports.
      8. Plist generator tool.

      The first three items have been completed and the next two items are being worked on. The DESTDIR support was more complicated than presumed and took more time than expected to complete. G�bor will continue working to finish these tasks and other ports related tasks. FreeBSD is happy to have interested him to keep working on ports and ports infrastructure.

      OCaml language support in ports

      Links
      Framework include file URL: http://www.freebsd.org/cgi/cvsweb.cgi/~checkout~/ports/lang/ocaml/bsd.ocaml.mk?rev=1.3&content-type=text/plain

      Contact: Stanislav Sedov <[email protected]>

      There were a number of OCaml ports in our tree, and each of them was doing the same work by maintaining OCaml ld.conf in the correct state, installing/removing their files/entries etc. To simplify the task of OCaml-language ports creation, the special framework (bsd.ocamk.mk) was developed and most of the ports were converted to use this framework. This allowed a lot of duplicate code to be removed. This new framework handles all the things required to install an OCaml-language library and properly register it. bsd.ocaml.mk also contains knobs to deal with findlib-powered libraries, modify ld.conf in the proper way, etc. Also, a lot of new Ocaml-related ports were added.


      Miscellaneous


      AsiaBSDCon 2007

      Links
      Conference Web Site URL: http://www.asiabsdcon.org/

      Contact: Hiroki Sato <[email protected]>
      Contact: George Neville-Neil <[email protected]>
      Contact: <[email protected]>

      Web site is up and we're soliciting papers and presentations. Some tutorials are already scheduled. Email [email protected] if you have questions or submissions.

      Open tasks:

      1. Send in more papers!

      BSDCan 2007

      Links
      URL: http://www.bsdcan.org/

      Contact: Dan Langille <[email protected]>

      The dates for BSDCan 2007 have been set: 11-12 May 2007. As is usual, BSDCan will be held at University of Ottawa, with two days of tutorials prior to the conference starting.

      The call for papers will go out in mid December. Start thinking about your submissions now!

      EuroBSDCon 2006

      Links
      EuroBSDCon Home Page URL: http://www.eurobsdcon.org/
      Registration Page URL: http://www.eurobsdcon.org/register/

      Contact: EuroBSDCon Organizing Committee <[email protected]>

      EuroBSDCon 2006 is taking place in Milan (Italy), from the 10th to the 12th of November.

      EuroBSDCon represents the biggest gathering for BSD developers from the old continent, as well as users and passionates from around the World. It is also a chance to share experiences, know-how, and cultures.

      The program is rich in talks about FreeBSD, with topics ranging from "How the FreeBSD ports collection works" to "Interrupt Filtering in FreeBSD". This means that both the novice and the hacker can enjoy the conference.

      Registration is open. The EuroBSDCon Organizing Committee hopes to see you in Milan.

      FreeBSD Multimedia Resources List

      Links
      URL: http://www.mavetju.org/unix/multimedia.php
      RSS version URL: http://www.mavetju.org/unix/multimedia-rss.php

      Contact: Edwin Groothuis <[email protected]>

      I have setup the FreeBSD Multimedia Resources List, a one-stop-shop for FreeBSD related podcasts, vodcasts and audio/video resources. Hopefully this list will make it easier for people to find and keep up to date with these recordings. The overview is available as a normal HTML page and as an XML/RSS feed.

      The ultimate goal is to have this list to reside under the www.FreeBSD.org umbrella.

      News Home | Status Home