Introduction
This report is about the rather quiet last quarter of 2005, with the release of FreeBSD 6.0 and the holiday season things evolved in the background. Nonetheless, most exciting projects hit the tree (or are going to very soon).
Upcoming events, such as the release of FreeBSD 6.1/5.5 and the third BSDCan conference with a big developer summit promise to provide a busier start in 2006. The foundation for upcoming development, however, are the projects that are described herein.
We hope that you find interesting projects to look at or work on. The next status report collection will be April 7 2006. We are looking forward to your report then.
Thanks again to everyone who submitted reports, and thanks to Brad Davis who stepped up for an extensive spelling and grammar review. Enjoy reading!
Projects
Documentation
- FreeBSD list of projects and ideas for volunteers (TODO list for volunteers)
- Problem Report Database
- The FreeBSD Dutch Documentation Project
FreeBSD team reports
Kernel
- Bt878 Audio Driver (aka FusionHDTV 5 Lite)
- E1000 driver improvements
- LSI MegaRAID improvements
- Sound subsystem improvements
Network infrastructure
- Early Binding Updates and Credit-Based Authorization for the Kame-Shisa Mobile IPv6 Software
- FAST_IPSEC Upgrade
- KAME Project Status Report
- New Networking Features in FreeBSD 6.0
- Optimizing the FreeBSD IP and TCP Stack
Userland programs
Architectures
Ports
Vendor / 3rd Party Software
Miscellaneous
- A Comprehensive Delay Analysis for Reactive and Proactive Handoffs with Mobile IPv6 Route Optimization
- BSDCan 2006
- TCP/IP Optimization Fundraiser Status
Projects
FreeSBIE
| Links | |
| URL: http://www.freesbie.org | |
| URL: http://torrent.freesbie.org | |
| URL: [email protected] |
Contact: FreeSBIE staff <[email protected]>
Development is going on after the complete rewrite of the toolkit. There are many plugins available and we're testing a new implementation of unionfs for 6.x. Since it's a bit unstable, it won't be included in the release anyway. Developers hope to enter the BETA state on February 1st, to release an -RC image around February 15th and the RELEASE around March 1st. We need more people to test the images we provide. Torrents for them are available at torrent.freesbie.org .
Open tasks:
- A new BETA Release, based on 6-STABLE, is available for testing.
jemalloc
| Links |
Contact: Jason Evans <[email protected]>
libc's malloc implementation has been replaced with an implementation that is designed to scale well for multi-threaded applications running on multi-processor systems. This is accomplished by creating multiple allocation arenas that are independent of each other, and permanently assigning threads to these arenas. In the common case, threads do not access the same allocator arena at the same time, which reduces contention and cache sloshing.
Single-threaded application performance is approximately equivalent to what it was with phkmalloc, but for multi-threaded applications that make heavy use of malloc, the performance difference can be huge (orders of magnitude).
As with phkmalloc, the new malloc implementation supports runtime configuration via the MALLOC_OPTIONS environment variable. See the malloc(3) manpage for details on supported options, as well as more information about the allocator's architecture.
variant symlinks
| Links | |
| URL: http://butcher.heavennet.ru/patches/kernel/varsym/ |
Contact: Andrey Elsukov <[email protected]>
The port of DragonFly's variant symlinks ( project ideas ) to FreeBSD. Variant symlinks is a dynamic symbolic link implementation. Source file of a variant symlink may contain one or more variable names. Each of these variable names is enclosed in braces and preceded by a dollar sign in the style of variable references in sh(1). Whenever a variant symlink is followed, each variable found in source file is replaced by its associated value. In this manner, a variant symlink may resolve to different paths based on context.
Open tasks:
- Document a new system calls.
- More testing.
- Write the rc.d script for the variant symlinks initialization.
Documentation
FreeBSD list of projects and ideas for volunteers (TODO list for volunteers)
| Links | |
| URL: http://www.FreeBSD.org/projects/ideas/ |
Contact:
Joel
Dahl
<[email protected]>
Contact:
Alexander
Leidinger
<[email protected]>
The "TODO list for volunteers" is now committed as the "FreeBSD list of projects and ideas for volunteers". So far the interest in the list is high and some volunteers already took the opportunity to start tackling some of the entries.
Unfortunately the FreeBSD project does not have enough human resources to provide a technical contact for every entry. Interested volunteers should not be afraid to try to come up with a solution for an entry without a technical contact. The people on the hackers and current mailing list are typically very helpful regarding answering specific questions (as long as they know the answer...).
We are looking forward to hear about new ideas, people willing to be technical contacts for generic topics (e.g. USB) or specific entries (already existing or newly created), suggestions for existing entries or completion reports for (parts of) an entry.
Open tasks:
- Add more ideas.
- Find more technical contacts.
Problem Report Database
| Links | |
| GNATS | URL: http://www.freebsd.org/support.html#gnats |
Contact: Mark Linimon <bugmeister_at_freebsd_dot_org>
The experiment to add 'tags' to many of the kern and related PRs, including such things as '[nfs]', '[fxp]', and so forth, continues. In addition, PRs with patches have been more consistently tagged with '[patch]'. Two new periodic reports based on both functional tags and PRs with patches have been added, with the goal of making these PRs more visible.
The FreeBSD Dutch Documentation Project
| Links | |
| FreeBSD released handbook | URL: http://www.freebsd.org/doc/nl/books/handbook |
| Preview documentation | URL: http://www.freebsd-nl.org/doc/nl |
| Preview website | URL: http://www.freebsd-nl.org/www/nl/ |
Contact:
Remko
Lodder
<[email protected]>
Contact:
Siebrand
Mazeland
<[email protected]>
The FreeBSD Dutch Documentation Project is an ongoing project, focussed on translating the English documentation and website to the Dutch language. Currently we are almost done with the FreeBSD Handbook and started the initial translation of the FreeBSD Website. We are always looking for people to help out, if you can help, please contact Siebrand or me so that we can divide the work amongst us.
Recent publications:
Recently the Printing and the Serial Communications chapters were
added to the FreeBSD Dutch Handbook.
Recently started items:
We started with the translation of the PPP and SLIP chapter and the
translation of the website.
Open tasks:
- Translate the final parts of the FreeBSD handbook.
- Translate the FreeBSD Website
FreeBSD team reports
FreeBSD Security Officer and Security Team
| Links | |
| URL: http://www.freebsd.org/security/ | |
| URL: http://www.freebsd.org/administration.html#t-secteam | |
| URL: http://vuxml.freebsd.org/ |
Contact:
Security
Officer
<[email protected]>
Contact:
Security
Team
<[email protected]>
This report covers the period July 2005 - January 2006, since the FreeBSD Security Team did not submit a status report for July - October 2005.
In August 2005, the long-time Security Officer, Jacques Vidrine, stepped down and was replaced by Colin Percival. Jacques remains with the team as Security Officer Emeritus, and the team thanks him for all his work over the past four years.
Also in August 2005, Dag-Erling C. Sm�rgrav was replaced by Simon L. Nielsen as Deputy Security Officer. In addition, Tom Rhodes and Guido van Rooij retired from the team in September 2005 and January 2006 respectively in order to devote their time to other parts of the FreeBSD project. The current Security Team membership is published on the web site.
In the time since the last status report, ten security advisories have been issued (five in 2005, five in 2006) concerning problems in the base system of FreeBSD; of these, four problems were in "contributed" code, while six were in code maintained within FreeBSD. The Vulnerabilities and Exposures Markup Language (VuXML) document has continued to be updated by the Security Team and the Ports Committers documenting new vulnerabilities in the FreeBSD Ports Collection; since the last status report, 117 new entries have been added, bringing the total up to 636.
The following FreeBSD releases are supported by the FreeBSD Security Team: FreeBSD 4.10, FreeBSD 4.11, FreeBSD 5.3, FreeBSD 5.4, and FreeBSD 6.0. Their respective End of Life dates are listed on the web site.
Ports Collection
Contact: Mark Linimon <[email protected]>
During this time, the number of ports PRs briefly dipped below 500 -- a number not seen since late 2000, when there were 4000 ports instead of our new total of over 14,000 ports. This is due to the hard work of a large number of individuals, including pav, edwin, mnag, garga, and many others. Congratulations folks! Some of this was due to more aggressively committing PRs where the maintainer had not responded within the timeout period. Although controversial, this new policy seems to be succeeding in its goal of improving the Ports Collection.
A new file, ports/KNOBS, was added by ahze to help bring some order in the chaos that had been the OPTIONS namespace.
dougb has changed the way that rc.d works in -HEAD to work more like the base rc.d scripts. We are hoping that this change will make ports maintenance easier in the future. However, in the meantime a few bugs have been introduced (which we intend to have fixed by the time 6.1 is released). While this regression is unfortunate, it was decided that now was the best time to try to make this change rather than waiting for 7.0. We hope our users can be patient with us in the interim.
Work continues to improve the marcuscom ports tinderbox, with new features added by marcus, aDe, and edwin in particular. Several ports committers are now running their own copies to test ports changes.
The www.FreeBSD.org/ports page, and the portmgr web pages, were reworked as well.
We have added 4 new committers since the last report.
Open tasks:
- Progress has been made in cracking down on ports that do not correctly install when LOCALBASE is not /usr/local, but some ports remain.
- portmgr would like to remind committers that PRs for their ports should be handled (either committed or marked 'suspended' or 'analyzed') within the two week timeout period. In this way other committers do not have to invoke the maintainer timeout and things will work more smoothly.
Release Engineering Status Report
| Links | |
| URL: http://www.freebsd.org/releng | |
| URL: http://www.freebsd.org/releases |
Contact: RE Team <[email protected]>
Another very busy year for the FreeBSD Release Engineering Team. Recognizing the problems, both technical and emotional, surrounding the FreeBSD 5.x releases, our primary focus was in getting the bugs out of FreeBSD 6.0 and getting it released. We succeeded at that quite well, and the 6.0 release on Nov 18 was a huge success for the project. Many thanks to all of the developers who put in countless hours fixing bugs and improving performance, and to the users who helped find, fix, and verify bugs.
Moving forward to 2006, we plan on doing a joint release of FreeBSD 5.5 and 6.1 in late March. The 5.5 release will mark the end of active FreeBSD 5.x development and releases, and is intended to help users who have not yet switched to FreeBSD 6. It consists primarily of bug fixes and minor improvements. FreeBSD 6.1 will be an upgrade to 6.0 and will include new drivers, better performance in certain areas, as well as bug fixes. We expect to release FreeBSD 6.2 and 6.3 later in 2006.
Kernel
Bt878 Audio Driver (aka FusionHDTV 5 Lite)
| Links | |
| Perforce source repository | URL: http://perforce.freebsd.org/fileSearch.cgi?FSPC=%2F%2Fdepot%2Fuser%2Fjmg%2Fbktrau%2F...&ignore=GO%21 |
Contact: John-Mark Gurney <[email protected]>
Basic audio capture is working. All of the parameters are set by userland, while the RISC program generation is by kernel. No real audio has been captured as there are no drivers for the tuner yet. Someone with a real Bt878 NTSC card that is supported by bktr(4) could use this to capture audio w/o using the sound card.
The real goal of this driver is to make HD capture possible with the DViCO FusionHDTV5 Lite card that I have. I have some of the documentation that I need, but I'm still missing two key docs. The docs for the LGDT3303 ATSC/8VSB/QAM demodulator chip and a block diagram of the board showing which GPIO lines go where and how the chips are interconnected. DViCO has been responsive in acknowledging my emails, but they have yet to produced any data besides pointing me to the Linux driver (which is difficult to figure out stuff by).
Open tasks:
- Complete basic capture driver.
- Make the bktr(4) drive cleanly attach to the card, and possibly add support for analog capture.
E1000 driver improvements
| Links |
Contact:
Scott
Long
<[email protected]>
Contact:
Andre
Opperman
<[email protected]>
In an effort to solve the 'interrupt aliasing' problem that plagues many motherboards under FreeBSD, I modified the Intel e1000 network driver (if_em) to use a combination of fast interrupts and taskqueues. This technique avoids interrupt threads entirely, which in turn avoids triggering the aliasing problem in the Intel APIC. The result is that the driver now handles and masks interrupts immediately, and a private taskqueue is then scheduled to run to process the link events and rx/tx events. A side effect of this asynchronous processing is that it acts much as traditional polling does, in that the amount of work done in the taskqueue can be controlled, and the taskqueue rescheduled to process work at a later time. This leads to the driver having the low-latency benefits of interrupts and the workload segmentation of polling, all without complicated heuristics. Several users have reported that the driver can handle higher loads than traditional polling without deadlocks.
Along with this work, I modified the SMPng locking in the driver so that no lock is required for the RX path. Since this path is already implicitly serialized by the interrupt and/or taskqueue and/or polling handler (all of which are exclusive to each other), there was no need for extra synchronization. This has two benefits. The first is reduction in processing overhead to unlock and lock the driver for every RX packet, and significant reduction in contention of the driver lock when transmitting and receiving packets at the same time. I believe that it is further possible to run the TX-complete path without a lock, further reducing overhead and contention for high transmit loads. The reduced contention also greatly benefited the fast-forward bridging code in FreeBSD, with up to 25% performance improvement seen, as well as lower CPU utilization.
The work can be found in FreeBSD 7-CURRENT for now. There are still some rough edges relating to falling back to traditional ithread and polling behavior, and I do not intend to merge the changes back to FreeBSD 6.x until these are resolved. I also hope to extend the INTR_FAST+taskqueue model into a general framework for doing Mac OSX style filter interrupts. The work in the if_em driver can also be extended to other high-performance network drivers such as if_bge and if_ti. Any help with investigating these topics is welcomed.
LSI MegaRAID improvements
| Links |
Contact:
Scott
Long
<[email protected]>
Contact:
Doug
Ambrisko
<[email protected]>
Major work has gone into improving both the performance of the LSI MegaRAID (amr) driver, and in adding Linux compatibility support. SMPng locking was added in Oct 2005 as well as a number of performance improvements. The result is 138% performance improvement in some local transaction tests.
Throughout 2005 a lot of work has gone into adding Linux compatibility to the driver. It is now possible to run many of the LSI-provided management apps for Linux under FreeBSD. Both this feature and the performance improvements are in the 7-CURRENT development branch of FreeBSD and are scheduled to be backported in time for the FreeBSD 6.1 release.
Sound subsystem improvements
| Links | |
| Patches for RELENG_5. | URL: http://people.FreeBSD.org/~ariff/ |
| The FreeBSD Project Ideas List. | URL: http://www.FreeBSD.org/projects/ideas/ |
Contact:
Ariff
Abdullah
<[email protected]>
Contact:
Alexander
Leidinger
<[email protected]>
Contact:
Multimedia
Mailinglist
<[email protected]>
A lot of changes have taken place in the sound system since the last status report. They range from less hiccups and distortion by disk accesses and/or driver bugs to new and improved features (software volume control implemented for soundcards which do not have hardware volume control). Additionally a new driver (snd_atiixp) has seen the light and a lot of problem reports were fixed.
Most of those changes and the changes mentioned in the previous status report are already merged to RELENG_6 and will be part of 6.1-RELEASE.
Open tasks:
- Have a look at the sound related entries on the ideas list.
- Rewrite some parts (e.g. a new mixer subsystem with OSS compatibility).
- sndctl(1): tool to control non-mixer parts of the sound system (e.g. spdif switching, virtual-3D effects) by an user (instead of the sysctl approach in -current); pcmplay(1), pcmrec(1), pcmutil(1).
- Plugable FEEDER infrastructure. For ease of debugging various feeder stuff and/or as userland library and test suite.
- Support for new hardware (envy24, Intel HDA).
- Performance enhancement (via 'slave'-channels).
- Closer compatibility with OSS, especially for the upcoming OSS v4.
Network infrastructure
Early Binding Updates and Credit-Based Authorization for the Kame-Shisa Mobile IPv6 Software
Contact: Christian Vogt <[email protected]>
Based on the Kame-Shisa Mobile IPv6 Software for FreeBSD 5.4, we implemented the performance optimization "Early Binding Updates" and "Credit-Based Authorization". The combined optimizations facilitate significant reductions in handoff delay without compromising protocol security [1][2].
FAST_IPSEC Upgrade
| Links |
Contact:
George
Neville-Neil
<[email protected]>
Contact:
Bjoern A.
Zeeb
<[email protected]>
Currently splitting out the rest of the PF_KEY data-structures from the key database. This will mean the user level applications and the kernel will not share datastructures and that they can, hopefully, advance on their own without being in lockstep.
Open tasks:
- Calculate diffs between Kame IPv4 version of IPSec and FAST_IPSEC and upgrade FAST to the latest standards.
- Add IPv6 support to FAST_IPSEC.
KAME Project Status Report
Contact: SUZUKI Shinsuke <[email protected]>
Most of the latest KAME code has been merged to 7-current and 6-stable, to prepare for the project conclusion in March 2006. For the same reason, we moved some ports applications (security/racoon, net/pim6sd, net/pim6dd, net/dhcp6) from KAME to sourceforge.net.
Some of the items (e.g. IGMPv3/MLDv2, Mobile-IPv6/NEMO, SCTP, DCCP, ISATAP) are not merged yet from the latest KAME code for several reasons. Other projects will continue to merge their work.
Open tasks:
- remove __P() macros
- set net.inet6.ip6.kame_version to a more appropriate date :-)
- update src/sys/netinet6/README
New Networking Features in FreeBSD 6.0
Contact: Andre Oppermann <[email protected]>
FreeBSD 6 has evolved drastically in the development branch since FreeBSD 5.3 and especially so in the network area. The presentation and paper give an in-depth overview of all network stack related enhancements, changes and new code with a narrative on their rationale.
Optimizing the FreeBSD IP and TCP Stack
Contact: Andre Oppermann <[email protected]>
FreeBSD has gained fine grained locking in the network stack throughout the 5.x-RELEASE series cumulating in 6.0-RELEASE. Hardware architecture and performance characteristics have evolved significantly since various BSD networking subsystems have been designed and implemented. This paper gives a detailed look into the implementation and design changes in FreeBSD 7-CURRENT to extract the maximum network performance from the underlying hardware.
Sponsored by: TCP/IP Optimization Fundraiser 2005
Userland programs
OpenBSD dhclient
| Links |
Contact:
Brooks
Davis
<[email protected]>
Contact:
Sam
Leffler
<[email protected]>
The OpenBSD rewrite of dhclient has been imported, replacing the ISC dhclient. The OpenBSD client provides better support for roaming on wireless networks and a simpler model of operation. Instead of a single dhclient process per system, there is one per network interface. This instance automatically goes away in the even of link loss and is restarted via devd when link is reacquired. To support this change, many aspects of the network interface configuration process were overhauled.
Support for adding aliases to DHCP configured interfaces has been committed to CURRENT and will be merged before 6.1-RELEASE. Soon work will begin to merge changes from OpenBSD that have taken place since the initial import.
Work on further interface configuration enhancements is underway for FreeBSD 7.0.
Architectures
FreeBSD on Xen 3.0
| Links | |
| current status | URL: http://www.fsmware.com/xenofreebsd/7.0/STATUS |
Contact: Kip Macy <[email protected]>
Full domU support in p4 branch of -CURRENT, except suspend / restore. Dom0 work is in progress. Scott Long is working on xenbus integration with newbus. After newbus integration it will go into CVS. I hope to see it MFCed to RELENG_6 so it will be available for 6.1.
Open tasks:
- Port the backend drivers from Linux.
- Port the domain management tools from Linux.
- Add multiboot support to loader(8) to support it booting xen.
- SMP, x86_64, and PAE support.
FreeBSD/xbox
| Links | |
| FreeBSD/xbox project page. | URL: http://xbox-bsd.nl |
Contact: Rink Springer <[email protected]>
FreeBSD/xbox support is nearing completion. Patches are available for nve(4) ethernet support, as well as a syscons(4)-capable console. I am working to integrate these in CURRENT, a backport to 6.x is planned too.
Work is under way to support X.Org as well; people with more detailed knowledge of X.Org are welcome to assist.
Open tasks:
- Enable framebuffer support in X.Org
- Figure out a way to use mfsroots without using loader(8)
Ports
FreshPorts
| Links | |
| URL: http://www.freshports.org/ |
Contact: Dan Langille <[email protected]>
FreshPorts recently moved to a new webserver. This should speed things up considerably.
You can read all about the new hardware on the recently introduced FreshPorts Blog . This blog will include technical discussions about ports and the problems they present with respect to FreshPorts. Site announcements will be posted there. As bugs are found, they will be listed, as well as their fixes.
Supporting multiple platforms and architectures is still in the development stage. Lack of time is affecting progress.
A fix for virtual ports is in the works. I'm also going to implement more caching to speed things up. If interested in discussing the options there, please get involved in the blog.
Vendor / 3rd Party Software
SysKonnect/Marvell Yukon device driver
| Links | |
| URL: http://www.marvell.com | |
| URL: http://www.syskonnect.de |
Contact: Karim Jamal <[email protected]>
This project provides support for SysKonnect's SK-98xx, SK-95xx,SK-9Exx and SK-9Sxx PCI/PCI-Express Gigabit Ethernet adapters via the yk(4) driver, as well as Marvell's Yukon LOM Gigabit Ethernet controllers via the myk(4) driver. Driver source has been made available to selected members of the FreeBSD project.
Miscellaneous
A Comprehensive Delay Analysis for Reactive and Proactive Handoffs with Mobile IPv6 Route Optimization
| Links | |
| Download document here. | URL: http://doc.tm.uka.de/2006/vogt-2006-delay-analysis-for-reactive-and-proactive-handoffs.pdf |
Contact: Christian Vogt <[email protected]>
Optimizations to reduce handoff delays inherent in Mobile IPv6 Route Optimization as well as IPv6 router discovery, address configuration, and movement detection have so far been mostly considered on an individual basis. This document evaluates three integrated solutions for improved handoff experience in surroundings with different preconditions: reactive handoffs with unmodified routers, reactive handoffs with router support, and movement anticipation and proactive handoff management.
BSDCan 2006
| Links | |
| URL: http://www.bsdcan.org/ |
Contact: Dan Langille <[email protected]>
We are well into the process of selecting the talks for BSDCan 2006. Our new program committee has a hard selection task over the new few weeks. The deadline for the Call For Papers has passed, but it's not too late to submit a talk. Please see the above URL for details. After the success of the Work in Progress last year , we are going to do it again this year. If you are working on something you'd like to tell the world about, considering giving a 5 minute talk at BSDCan. The registration prices for BSDCan 2006 will be the same as they were for 2005 . We will be again in the SITE building at University of Ottawa and you'll have lots of opportunity to meet with people from all over the world. Be sure to make your travel plans now and don't miss out on the biggest BSD event this year: BSDCan 2006.
Open tasks:
- We're looking for volunteers to help out just before and during the conference. Contact Dan at the above address.
- If you have a talk you'd like to present, contact Dan at the above address.
TCP/IP Optimization Fundraiser Status
Contact: Andre Oppermann <[email protected]>
The fundraiser has been very successful and I want to thank everyone who has pledged their support and tipped the jar. The full amount plus a little bit more has been raised in a very short timeframe. More information on the exact amounts and their sponsors can be found at the first link.
After the delays on this project caused by the FreeBSD 6.0 Release cycle code freeze work has picked up and a paper was written and a presentation held on "Optimizing the FreeBSD IP and TCP Stack" for EuroBSDCon 05 on November 27th. See related status report under that title.
From December 21st to January 11th I received access to a calibrated Agilent N2X gigabit tester and traffic generator. Stock FreeBSD 7-current was tested and profiled extensively in this timeframe. A first proof of concept optimization was developed in cooperation with Scott Long. It involved converting the Intel Gigabit ethernet em(4) driver to make use of fast interrupt handlers, taskqueues and lockless RX ring handling. This improved the performance from 570kpps to 750kpps, a 25% improvement, with IP fastforwarding enabled.
Open tasks:
- A large number of profiles and measurements was taken and a detailed report on the performance characteristics and remaining bottlenecks is under preparation.
- Further optimizations and new features described on the Optimization Fundraiser page.
News Home | Status Home