Introduction
The third quarter of 2015, from July to September, was again a period of busy activity for FreeBSD: for the second quarter in a row we have the largest report yet published.
The Foundation continues to play a strong role, bringing both a developer and evangelist presence to conferences, funding much of the hardware that the cluster administration team uses to keep things running, and sponsoring many development projects for FreeBSD. This quarter we also hear from some of the student projects funded by Google Summer of Code 2015, ranging a wide gamut from the bootloader to additional ARM support, but also at a range of completion status. Some of the GSoC output is in the tree already, but others could benefit from additional attention to help out our budding new contributors as their schedules fill with the return to classes.
ZFS and the network stack continue to be strong areas for FreeBSD, with both receiving active maintenance and feature improvements during this quarter. Substantial work continues on arm64, potentially putting it on the path toward a promotion to Tier-1 status, and a new port to the RISC-V architecture has made great headway in a short period of time. But it is not just our strengths and exciting new areas that have seen attention this cycle; there are also some parts of the system that are frequently perceived as unchanging infrastructure that have received attention and improvements, with truss and (k)gdb receiving significant overhauls, new implementations for the man page tools being brought in, the website receiving a new skin, and a brand new system for translating documentation that greatly lowers the barrier to entry.
Nonetheless, despite its record length, this report does not and cannot cover all of the work being done on FreeBSD throughout the reporting period — there are many bug fixes too minor to mention here, and developers too busy working on the next project to write up an entry for the previous project. It is not just the developers committing to Subversion that comprise the ongoing activities of FreeBSD, but also the users testing unreleased code or reporting bugs in released code, and participants on the mailing lists and forums helping each other solve their problems. Even the chats on IRC that wander far from the stated topic of a channel contribute to the community around FreeBSD; it is that community whose effectiveness and helpfulness is a key component of the effectiveness and usefulness of FreeBSD itself. Not just to the developers listed in this report, but to everyone in the community, thank you for making FreeBSD a great operating system.
—Ben Kaduk
Please submit status reports for the fourth quarter of 2015 (from October to December) by January 7, 2016.
FreeBSD Team Reports
Projects
- automtud: Better Jumbo Frame Support
- bhyve
- Clang, llvm, lldb, compiler-rt and libc++ Updated to 3.7.0
- DTrace and TCP
- FreeBSD on the Acer C720 Chromebook
- High Availability Clustering in CTL
- Multipath TCP for FreeBSD
- Porting bhyve to ARM-based Platforms
- Root Remount
- The Graphics Stack on FreeBSD
- The nosh Project
- UEFI Boot and Framebuffer Support
- ZFS Code Sync with Latest Illumos
- ZFS Support for UEFI Boot/Loader
Kernel
- Adding PCIe Hot-plug Support
- Cavium LiquidIO Smart NIC Driver
- CloudABI: Pure Capabilities Runtime Environment
- FreeBSD Xen
- ioat(4) Driver Import
- IPsec Upgrades
Architectures
- Atomics
- FreeBSD on Cavium ThunderX (arm64)
- FreeBSD on the HiKey ARMv8 Board
- FreeBSD/arm64
- FreeBSD/RISC-V Port
Userland Programs
Ports
- Bringing GitLab into the Ports Collection
- GNOME on FreeBSD
- KDE on FreeBSD
- Node.js Modules
- Ports Collection
- Ports on PowerPC
- Xfce on FreeBSD
Documentation
Google Summer of Code
- Allwinner A10/A20 Support
- mtree Parsing and Manipulation Library
- Multiqueue Testing
- Update Ficl in Bootloader
Miscellaneous
FreeBSD Team Reports
FreeBSD Cluster Administration Team
Contact: FreeBSD Cluster Administration Team <clusteradm@>
The FreeBSD Cluster Administration Team consists of the people responsible for administering the machines that the project relies on for its distributed work and communications to be synchronised.
Our primary cluster has been hosted as a guest in California for many years. Our ongoing project is relocating the core functionality to a location in New Jersey with a formal hosting arrangement. This is an equipment refresh, consolidation for better use of resources, and for better continuity of service.
There is a significant amount of behind-the-scenes work to make this happen. The original cluster was implemented with a common, shared, assumed-to-be secure network with ubiquitous NFS everywhere. This structure does not lend itself well to being distributed across geographically diverse locations, particularly when Internet transit is required. The bulk of the work is rebuilding services to be portable, stand-alone components that do not depend on shared-network access and are safe enough to use across the insecure Internet.
Highlights this quarter:
- Many internal distribution systems switched from rsync to a distribution mesh using "syncthing".
- We have implemented more code/data signing infrastructure with out-of-band verification.
- New 32-core reference build hosts are online.
- Internal admbugs switched from bugzilla 4.4 to 5.0 and packages were made available for the bugmeister team.
- Finally switched from varnish3 to varnish4.
- We exorcised hub.FreeBSD.org, the last survivor of the 2012 security incident.
- vuxml and the legacy portaudit build system were converted to components and integrated.
- https://download.FreeBSD.org/ is nearing completion (please do not use until officially announced).
- A Taiwan node was brought into service for pkg, ftp, svn, and vuxml mirroring.
- One of the freebsd-update mirrors was converted from lighttpd to nginx due to a data corruption bug.
- We completed detachment of the svn repository from the old cluster and moved it to its new location.
Ongoing:
- The cluster runs a mixture of 11-current and 10-stable as part of our "eat our own dogfood" project. For this to be useful, we do monthly cluster refreshes to keep up with current code.
- We build internal base system snapshots every few days and packages every day.
- We also provide support for non-clusteradm-operated services including jenkins, reviews, portsnap, freebsd-update, bugzilla, package builders, git, and mercurial. This varies from as little as maintaining SSL front-ends through operating servers, distributing data or building packages/binaries to run.
FreeBSD Release Engineering Team
Contact: FreeBSD�Release Engineering Team <[email protected]>
The FreeBSD Release Engineering Team is responsible for setting and publishing release schedules for official project releases of FreeBSD, announcing code freezes, and maintaining the respective branches, among other things.
In mid-August, the FreeBSD�Release Engineering Team released FreeBSD�10.2-RELEASE, two weeks earlier than the original schedule anticipated.
The FreeBSD�Release Engineering Team would like to thank all that have tested the BETA and RC builds and reported issues during the release cycle.
The FreeBSD�Release Engineering Team, with approval from the FreeBSD�Core Team, appointed Marius Strobl as the Deputy Lead.
This project was sponsored by The FreeBSD Foundation.
The FreeBSD Core Team
Contact: FreeBSD Core Team <[email protected]>
The biggest task handled by the Core Team during this quarter was developing and publishing the new Code of Conduct. The Code of Conduct describes how people are expected to behave on all FreeBSD official communication channels, as well as how developers and other people involved with the project are to behave when representing the project in public.
The Code of Conduct was generally well received and elicited numerous comments and suggestions for improvements from the community, many of which have been integrated.
The next task handled by Core was the restoration of Babak Farrokhi's ports commit bit. Babak resides in Iran. A few years ago, legal advice suggested that allowing contributions from Iranian residents might violate US trade sanctions. After several years, Core was asked to revisit the issue. On the advice of counsel, Core decided that it could restore commit privileges to commmitters residing in Iran.
The CTM service came under security review. Given that the lack of use of routine authenticity checking made the injection of trivial exploit code relatively easy, the service was held to be too risky to continue as an official part of the FreeBSD base system. CTM has very few remaining users but they should be able to install CTM from the Ports Collection in order to continue doing so.
Core learned that ISC was ceasing its hosting service, which has entailed a rapid rework of plans on the movement of significant portions of the FreeBSD cluster to that data center. Cluster administration has taken ownership of the situation and is making progress.
Core fielded an enquiry about NextBSD and whether this should be the future direction for the whole FreeBSD project. Core's position is that NextBSD is an interesting project, and we regard it, like the other BSD projects, as a potential source of good ideas. However, we currently have no plans to adopt NextBSD as the official FreeBSD distribution.
Beyond these issues, Core also spent time in various routine activities. During this quarter we issued three new src commit bits, and took none in for safekeeping. Welcome to Allan Jude, Marcelo Araujo, and Andriy Voskoboinyk.
Projects
automtud: Better Jumbo Frame Support
Links | |
jmgurney/automtud on github | URL: https://github.com/jmgurney/automtud |
Contact: John-Mark Gurney <[email protected]>
The automtud script will allow a FreeBSD machine to send jumbo frames to machines that support them, while using normal-sized frames for other machines.
There are various advantages to using jumbo frames, such as reduced protocol overhead. It also means that TCP streams will not be segmented as much, although TSO helps mitigate the disadvantages of such segmentation. In cases where LRO does not work well, fewer packets will be received.
The script currently does not restore the system to its original state when it exits. This means that you must manually change the interface MTU and delete host routes after stopping the script.
Open tasks:
-
Fix up various Ethernet drivers to better support jumbo frames. Most Ethernet drivers, though they support scatter/gather, use a physically contiguous zone to do so, which can cause resource shortages.
-
More testing is needed to ensure that things behave as expected. This means that when running the script, communication to all machines functions normally, without slowdown or connectivity issues. Check vmstat -z | grep mbuf to ensure that such issues are not due to running out of jumbo_9k or jumbo_16k buffers due to Ethernet driver issues.
bhyve
Contact:
Peter
Grehan
<[email protected]>
Contact:
Neel
Natu
<[email protected]>
Contact:
Tycho
Nightingale
<[email protected]>
Contact:
Allan
Jude
<[email protected]>
Contact:
Michael
Dexter
<[email protected]>
bhyve is a hypervisor that runs on the FreeBSD/amd64 platform. At present, it runs FreeBSD (8.x or later), Linux i386/x64, OpenBSD i386/amd64, NetBSD/amd64, Illumos, and Windows Vista/7/8/10/2008r2/2012r2/2016 x64 guests. Current development is focused on enabling additional guest operating systems and implementing features found in other hypervisors.
A combined bhyve and ZFS BoF was held during vBSDCon 2015, hosted by Michael Dexter and Allan Jude. Questions asked about bhyve included live migration and suspend/resume support, and configurations using ZFS.
Three bhyve-related projects were selected for GSoC 2015: NE2000 device emulation, porting bhyve to ARM, and ptnetmap support.
The major enhancement for bhyve this quarter was support for external firmware, along with a port of the Intel edk2 UEFI firmware. This allows bhyve to run Windows in headless mode, and also Illumos.
Open tasks:
-
Improve the documentation.
-
bhyveucl is a work-in-progress script for starting bhyve instances based on a libUCL config file. More information at https://github.com/allanjude/bhyveucl.
-
Add support for virtio-scsi.
-
Flexible networking backends: wanproxy, vhost-net.
-
Support running bhyve as non-root.
-
Add filters for popular VM file formats (VMDK, VHD, QCOW2).
-
Implement an abstraction layer for video (no X11 or SDL in base system).
-
Suspend/resume support.
-
Live migration.
-
Nested VT-x support (bhyve in bhyve).
-
Support for other architectures (ARM, MIPS, PPC).
Clang, llvm, lldb, compiler-rt and libc++ Updated to 3.7.0
Contact:
Dimitry
Andric
<[email protected]>
Contact:
Ed
Maste
<[email protected]>
Contact:
Roman
Divacky
<[email protected]>
Contact:
Davide
Italiano
<[email protected]>
We have updated clang, llvm, lldb, compiler-rt, and libc++ in base to the 3.7.0 release. These all contain numerous improvements. Please see the linked release notes for more detailed information. This brings us completely up-to-date with the latest upstream versions of these projects. Meanwhile, Ed Maste is working on importing the llvm.org version of libunwind.
Like the 3.5.x and 3.6.x releases, these components require C++11 support to build. At this point, FreeBSD 10.0 and later provide that support, at least on x86. Currently, there are no solid plans to MFC these versions to any stable branches, due to the difficulties this would introduce for the usual upgrade scenarios.
Thanks to Ed Maste and Andrew Turner for their help with this import, and thanks to Antoine Brodin for several ports exp-runs.
During the first ports exp-run, some major problems were found, one introduced by a clang bug which caused pow() to generate floating point exceptions in some cases. This in turn caused libpng to fail to build, and one bug in libjpeg-turbo, which was caused by undefined behavior. These two problems took some time to fix, after which another exp-run was done, and this resulted in about a dozen newly failed ports. For almost all of these new failures, fixes were submitted and linked to the original PR 201377 for the exp-run.
Open tasks:
- Commit ports fixes for dependencies of PR 201377.
- Test and report issues with the new tool chain.
DTrace and TCP
Links | |
Commit adding trace points replacing TCPDEBUG | URL: https://svnweb.freebsd.org/changeset/base/287759 |
Contact: George Neville-Neil <[email protected]>
With the advent of DTrace we are able to replace many of the internal kernel debugging options, such as TCPDEBUG, with statically defined tracepoints (SDTs). Tracepoints have now been added to the system that replicate the functionality of the TCPDEBUG kernel option. No new kernel options need to be added — they are standard with any kernel that has DTrace, which is included in the default GENERIC kernels in 10.X and HEAD.
This project was sponsored by Limelight Networks.
FreeBSD on the Acer C720 Chromebook
Contact: Michael Gmelin <[email protected]>
The Acer C720 Chromebook is an affordable (under $200) and powerful little laptop that provides a battery life of up to six hours running FreeBSD. It is a great machine for travelling and coding in general. The machine is fully functional, meaning that all essential devices work: keyboard, trackpad, light sensor, backlight control, display in VESA mode (fast), external Display on HDMI (only VESA mirror mode), sound, USB ports, SD card slot, camera, and Atheros wireless.
This quarter, this project extended previous work on the boot process and keyboard driver as well as the smbus(4) driver. It added three new drivers: ig4(4) for the I2C bus, cyapa(4) for the trackpad, and isl(4), for the ambient light sensor.
Much of the development was originally done in late 2014. Since then, the patches have been massively improved and merged into HEAD, so that all relevant devices work without manual patching.
For those who are unable to run HEAD, there is a backported patch to 10.2-RELEASE.
Thanks to everyone who helped in the process. I couldn't have done it without you (you know who you are).
High Availability Clustering in CTL
Contact: Alexander Motin <[email protected]>
CAM Target Layer (CTL), when originally developed by Copan/SGI, had support for High Availability clustering. Unfortunately, significant portions of the HA code were never published with the main body of the source code. Now, the missing part has been reimplemented from scratch, fixed, and improved.
This code supports dual-node HA with Asynchronous LUN Unit Access (ALUA) in four modes:
- Active/Unavailable without interlink between nodes, where the secondary node can report nothing except its presence.
- Active/Standby with the secondary node handling only basic LUN discovery and reservation, synchronizing state and command execution with the primary node through the interlink.
- Active/Active with both nodes processing commands and accessing the backing storage, synchronizing state and command execution with the primary node through the interlink.
- Active/Active with the secondary node having no backing storage access, but instead working as a proxy, transferring all commands to the first node for execution through the interlink.
In the case of lost interlink connectivity to primary node, the secondary node falls into the Transitioning state, which is like Unavailable and cannot answer most requests, but makes the initiator wait for recovery or cluster failover.
CTL also got a large number of other improvements, including support for emulation of CD/DVD drives and removable disks, live LUN reconfiguration, and so on.
The code is committed to FreeBSD head and was recently merged to the stable/10 branch.
This project was sponsored by iXsystems, Inc..
Multipath TCP for FreeBSD
Links | |
MPTCP for FreeBSD Project Website | URL: http://caia.swin.edu.au/urp/newtcp/mptcp/ |
MPTCP for FreeBSD Source Repository | URL: https://bitbucket.org/nw-swin/caia-mptcp-freebsd/ |
Contact: Nigel Williams <[email protected]>
Multipath TCP (MPTCP) is an extension to TCP that allows for the use of multiple network interfaces on a standard TCP session. The addition of new addresses and scheduling of data across them occurs transparently from the perspective of the TCP application.
The goal of this project is to deliver an MPTCP kernel patch that interoperates with the reference MPTCP implementation, along with additional enhancements to aid network research.
The v0.5 patch was released, which is the first patch of the re-written implementation. We are in the process of documenting the new design and addressing some feedback as provided from the community.
Work has commenced on improved input handling, as the current method of receiving and reassembling segments has been the cause of some instability and stalls during connection shutdown. This will involve re-using the subflow receive buffers and an upcall to enqueue a MP-layer reassembly task without the need to take a lock on the MP control block. The improvements should also allow bypassing mptcp_usrreq for standard TCP connections.
The MPTCP commit history was synchronized with hg-beta.FreeBSD.org, and we have made the repository available on BitBucket (see links). Future patch releases will be tagged there. The tree is now merged with FreeBSD head weekly. An updated v0.51 patch is ready for release.
This project was sponsored by The FreeBSD Foundation.
Open tasks:
-
Release the v0.51 patch.
-
Populate documentation and the issue tracker on the BitBucket repository.
-
Improvements to receive-side code before further testing.
-
Prepare a technical report detailing the design of the current patch.
Porting bhyve to ARM-based Platforms
Links | |
Project Wiki page | URL: https://wiki.FreeBSD.org/SummerOfCode2015/PortingBhyveToArm |
Contact:
Mihai
Carabas
<[email protected]>
Contact:
Peter
Grehan
<[email protected]>
This summer we have started porting bhyve onto ARMv7 platforms. The low-level routines for ARM processors were rewritten while trying to preserve the hypervisor API originally created for the x86 architectures. We managed to bring up a FreeBSD guest up to the point of initializing interrupts. There is still work to be done in order to virtualize the interrupts and the timer. Our short-term plan after finishing the interrupts and the timer is porting to a real hardware platform (Cubie2).
Open tasks:
-
Virtualize interrupts and timer.
-
Port to a real hardware platform.
-
Create SMP support for bhyve-on-arm.
-
Port to ARMv8.
Root Remount
Links | |
Userland code review | URL: https://reviews.freebsd.org/D3693 |
Contact: Edward Tomasz Napierala <[email protected]>
A feature long missing from FreeBSD was the ability to boot up with a temporary rootfs, configure the kernel to be able to access the real rootfs, and then replace the temporary root with the real one. In Linux, this functionality is known as pivot_root. The reroot project aims to provide similar functionality in a different, slightly more user-friendly, way. Simply put, from the user's point of view it is as simple as running reboot -r. The system performs a partial shutdown, killing all processes and unmounting the rootfs, and then partial bringup, mounting the new rootfs, running init, and running the startup scripts as usual.
The kernel part of the project has been committed to 11-CURRENT. The userland part is at the "finishing touches" stage, and is expected to be committed soon. A merge to stable/10 is planned and reroot support is planned be included in FreeBSD 10.3.
This project was sponsored by The FreeBSD Foundation.
The Graphics Stack on FreeBSD
Contact: FreeBSD Graphics team <[email protected]>
The Mesa ports were updated to 10.6.8. At the same time, the ports received a major overhaul to make sure all ports are correctly configured. Dual version support was removed. There is only one Mesa version for all supported FreeBSD versions. The libosmesa port, which provided the off-screen version of Mesa, was merged into the Mesa framework.
Another big item that was included in the Mesa port is OpenCL. There are two GPU-based OpenCL implementations: lang/clover for supported Radeon cards, and lang/beignet for supported Intel cards (currently only Ivybridge). Thanks go to Johannes Dieterich, O. Hartmann, and Koop Mast for making this happen.
Now that Mesa is up-to-date, we can apply the same update procedure to the X.Org server. It is currently at 1.14, and an update to 1.17 is ready. It will be committed shortly.
On the kernel side, progress has been made with the i915 update. The driver is able to attach. There are some reports that the X.Org server starts but Mesa is unhappy, so acceleration does not work yet. If you want to test, instructions will be posted on the wiki in the i915 update article (see links). At this stage, we can only accept patches, though — we will not be able to provide support.
We attended two conferences: XDC 2015 in Toronto and EuroBSDcon 2015 in Stockholm. Reports will be posted on the blog.
Open tasks:
-
See the Graphics wiki page for up-to-date information.
The nosh Project
Contact: Jonathan de Boyne Pollard <[email protected]>
The nosh project is a suite of system-level utilities for initializing, running, and shutting down BSD systems, and for managing daemons, terminals, and logging. It supersedes BSD init and the NetBSD rc.d system, drawing inspiration from Solaris SMF for named milestones, daemontools-encore for service control/status mechanisms, UCSPI, and IBM AIX for separated service and system management. It comprises a range of compatibility mechanisms, including shims for familiar commands from other systems, and an automatic import mechanism that takes existing configuration data from /etc/fstab, /etc/rc.conf{,.local}, /etc/ttys, and elsewhere, applying them to its native service definitions and creating additional native services. It is portable (including to Linux) and composable, it provides a migration path from the world of systemd Linux, and it does not require new kernel APIs. It provides clean service environments, orderings and dependencies between services, parallelized startup and shutdown (including fsck), strictly size-capped and autorotated logging, the service manager as a "subreaper", and uses kevent(2) for event-driven parallelism.
The past few months have seen a growth in the import mechanism, with full import of /etc/fstab and /etc/ttys available in version 1.18 in July, and importing PC-BSD Warden and FreeBSD�9 jails, and full import of gbde and geli mount/unmount mechanisms in version 1.21 in October. It has also gained the ability to automatically re-generate host.conf and sysctl.conf whenever their source files change.
Other developments in the past few months include fully independent shutdown support, no longer relying upon an externally provided shutdown command from another toolset, and a full suite of binary packages. As of version 1.20, it became possible to have a fully-nosh-managed system, on both FreeBSD and Linux, using just precompiled binary packages.
The biggest task remaining is one that was set a while ago: the creation of enough native service bundles and ancillary utilities to entirely supplant the rc.d system. A lot of this has been achieved, with the original target list of 157 items now down to just 39 remaining. These are the tricky ones, of course, where help is most needed.
Open tasks:
-
There are still a few rc scripts left that should be easy to convert, such as /etc/rc.d/gptboot and /etc/rc.d/growfs as oneshot services, /etc/rc.d/routing, and /etc/rc.d/kldxref.
-
FreeBSD's /etc/rc.d/bluetooth is over 360 lines long. In 2011, Iain Hibbert wrote a "simpler" bluetooth for NetBSD. This can perhaps be used as a simpler basis for a nosh translation.
-
Add kernel support for passing a -b option to pid 1, and support for a boot_bare variable in the loader, to allow "emergency" (where even no shell dotfiles are loaded) and "rescue" mode bootstraps, akin to Linux. (History: The -b mechanism and idea date back to version 2.57d of Miquel van Smoorenburg's System 5 init clone, dated 1995-12-03, and was already known as "emergency boot" by 1997.)
-
Add support to FreeBSD's fsck(8) for outputting machine-readable progress reports to a designated file descriptor, so that nosh can provide progress bars for multiple fscks running in parallel. nosh already provides this functionality on Linux, where fsck(8) does provide machine-readable output.
-
Identify when the configuration import system needs to be triggered, such as when bsdconfig alters configuration files, and create the necessary hooks to import external configuration changes into nosh.
-
Investigate how FreeBSD/PC-BSD could be improved by taking advantage of some available nosh package mechanisms.
UEFI Boot and Framebuffer Support
Contact:
Ed
Maste
<[email protected]>
Contact:
Marcel
Moolenaar
<[email protected]>
A number of UEFI bug fixes were committed over the last quarter, improving compatibility with different UEFI implementations. This includes improvements to EFI's vt(4) framebuffer driver, efifb, to handle systems with high resolution displays and unusual framebuffer stride values. In particular, this improves compatibility with a large number of recent Apple MacBook Pros and other Macs.
Open tasks:
- Test FreeBSD head and FreeBSD-STABLE snapshots on a variety of UEFI implementations.
ZFS Code Sync with Latest Illumos
Contact: Alexander Motin <[email protected]>
The ZFS codebase received a significant batch of merges, and is now in sync with the latest Illumos. Among other things, this update includes:
- LZ4 is now the default compression algorithm.
- Improved prefetch for faster send/receive.
- Reduced RAM usage by almost 50% for L2ARC.
- Improved I/O aggregation.
- Fine-grained checksumming in send/receive stream.
- Reduced import time for pools with many datasets.
- Reworked and simplified predictive prefetcher.
The code is committed to FreeBSD head and was recently merged to the stable/10 branch.
ZFS Support for UEFI Boot/Loader
Contact: Eric McCorkle <[email protected]>
UEFI-enabled boot1.efi and loader.efi have been modified to support loading and booting from a ZFS filesystem, as described in the previous report. The ZFS-enabled loader.efi can be treated as a chainloader when using ZFS-enabled GRUB.
During this quarter, several successful tests have been reported on simple ZFS setups, using both boot1.efi with loader.efi as well as GRUB and loader.efi.
Successful tests have been reported for UFS with the patched boot1.efi and loader.efi as well.
Open tasks:
-
Test reports are needed for more complex ZFS setups, such as with log/l2arc vdevs, mirroring, striping, and raidz.
-
Pending successful reports, the patch needs to be reviewed and committed.
Kernel
Adding PCIe Hot-plug Support
Contact: John-Mark Gurney <[email protected]>
PCI Express (PCIe) hot-plug is used on both laptops and servers to allow peripheral devices to be added or removed while the system is running. Laptops commonly include hot-pluggable PCIe as either an ExpressCard slot or a Thunderbolt interface. ExpressCard has built-in USB support that is already supported by FreeBSD, but ExpressCard PCIe devices like Gigabit Ethernet adapters and eSATA cards are only supported when they are present at boot, and their removal may cause FreeBSD to crash.
The goal of this project is to allow these devices to be inserted and removed while FreeBSD is running. This work will provide the basic infrastructure to support adding and removing devices, though it is expected that additional work will be needed to update individual drivers to support hot-plug.
Current testing is focused on getting a simple UART device functional. Basic hot swap is currently functional.
A set of the patches for the work done in this project is now available on github.com.
This project was sponsored by The FreeBSD Foundation.
Open tasks:
-
Get suspend/resume functional by saving and restoring the necessary registers. This should be addressed by r281874.
-
Make sure that upon suspend, devices are removed so that we are not fooled if they are replaced with different devices while the machine is suspended.
-
Improve how state transitions are handled, possibly by using a proper state machine.
Cavium LiquidIO Smart NIC Driver
Links | |
LiquidIO product page | URL: http://www.cavium.com/LiquidIO_Application_Acceleration_Adapters.html |
Contact:
Stanislaw
Kardach
<[email protected]>
Contact:
Zyta
Racia
<[email protected]>
This project aims to add support for the LiquidIO family of high-performance programmable accelerator 10/40-gigabit Ethernet network adapters. The currently developed kernel driver supports CN6640- and CN6880-based PCIe cards, enabling these features:
- A CNNIC API for controlling/interacting with the smart NIC
from user and kernel space including:
- Handling multiple concurrent applications running on the same device
- A request/reply mechanism for (a)synchronous ordered/unordered communication
- Remote memory operations
- Device shutdown/reset
- A basic NIC module utilizing the CNNIC API and a
Cavium-provided NIC firmware. This module provides:
- Single/multi-queue TX
- Hardware TCP/UDP checksum offloading
- Large Receive Offload
- Promiscous mode
- Sysctl-based device statistics and configuration view
- Custom firmware loading via user-built modules and FreeBSD's firmware(9) mechanism.
The project is currently being developed in house and is being prepared for upstream. We plan on making it available in FreeBSD 11.
This project was sponsored by Cavium, and Semihalf.
Open tasks:
- Upstream the code to FreeBSD head.
CloudABI: Pure Capabilities Runtime Environment
Contact: Ed Schouten <[email protected]>
CloudABI is a POSIX-like runtime environment that uses Capsicum as its sole access control mechanism. CloudABI allows you to develop software that is better hardened against security vulnerabilities, is easier to test, and is easier to migrate across systems.
As of August, all of the kernel modifications that are needed to run CloudABI programs have been integrated into FreeBSD head. After loading the cloudabi64 kernel module, you can either run CloudABI programs directly from the shell or by using the cloudabi-run tool (sysutils/cloudabi-utils). cloudabi-run allows you to inject sockets, files, and directories into the launched program in a more structured way.
In the meantime, work has started on developing a Ports Collection that contains cross-compiled utilities and libraries for CloudABI. The intent is that this framework generates native packages for a number of operating systems, making it possible to develop CloudABI applications on any operating system, regardless of whether that operating system actually supports CloudABI.
If you are interested in CloudABI, be sure to go to the project page on GitHub, watch recordings of talks at conferences, or wait for the upcoming edition of the FreeBSD Journal, which will feature an article on CloudABI.
This project was sponsored by Nuxi, the Netherlands.
Open tasks:
-
CloudABI is currently only available for amd64. It would make sense to port CloudABI to additional architectures like aarch64.
-
Support for CloudABI has only been integrated into FreeBSD. If we manage to upstream support for CloudABI into other operating systems, it should be possible to run the same binary on multiple operating systems without recompilation.
-
The CloudABI Ports Collection currently has only 60 packages. Although these packages already the building blocks of some interesting software, we are always interested in expanding.
FreeBSD Xen
Contact:
Roger
Pau Monn�
<[email protected]>
Contact:
Julien
Grall
<[email protected]>
Xen is a hypervisor using a microkernel design, providing services that allow multiple computer operating systems to execute on the same computer hardware concurrently. Xen support for FreeBSD on x86 as a guest was introduced in version 8 and ARM support is currently being worked on. Support for running FreeBSD as an amd64 Xen host (Dom0) is available in HEAD.
On the x86 front, most of the work during this quarter focused on the implementation of PVH inside Xen. Consequently, most of the activity happened inside of the hypervisor. Patches for a clean PVH implementation have been posted, with the aim of having them merged in the next Xen release (4.7). Once that is done, work will continue adding new features to both FreeBSD and Xen to have feature parity with traditional PV guests/hosts.
Apart from this, work is ongoing to import a new netfront from Linux in order to support new features, like split event channel and multiple queue support.
On the ARM front, this quarter's work focused on getting FreeBSD/arm64 booting as a Xen guest. The current activity is to upstream patches preparing Xen drivers to support arm64. This includes a rework of the console driver.
This project was sponsored by Citrix Systems R&D.
Open tasks:
-
Generalize the event channel code so it can be used on ARM.
-
Improve backend (netback, blkback) performance.
-
Work with upstream Xen to improve PVH and make it stable.
-
Improve the generic bounce buffer code for unmapped bios in order to support the alignment requirements of the blkfront driver.
ioat(4) Driver Import
Links | |
Wikipedia article on IOAT | URL: https://en.wikipedia.org/wiki/I/O_Acceleration_Technology |
Commit importing ioat(4) | URL: https://svnweb.FreeBSD.org/base?view=revision&revision=r287117 |
Contact:
Jim
Harris
<[email protected]>
Contact:
Conrad
Meyer
<[email protected]>
A new driver, ioat(4), was added to the tree. ioat(4) supports Intel's I/O Acceleration Technology devices which are found on some Intel server systems.
These devices are DMA offload engines, which can accelerate some I/O-heavy applications by offloading memory copies from the main CPU to the I/OAT unit. This acceleration is not transparent; applications must be adapted to take advantage of the hardware.
Some I/OAT models support more advanced copying modes, such as XOR; these modes are not yet supported in the ioat(4) driver.
This project was sponsored by Intel Corporation, and EMC / Isilon Storage Division.
Open tasks:
-
Further testing, especially on a range of device models other than BDXDE (looking for volunteers here).
-
Support for the more advanced copy modes.
IPsec Upgrades
Contact:
George
Neville-Neil
<[email protected]>
Contact:
John-Mark
Gurney
<[email protected]>
Contact:
Ermal
Lu�i
<[email protected]>
IPsec is now enabled by default in the GENERIC kernel configuration, and work is proceeding to speed things up in various ways. The latest changes are the addition, by John-Mark Gurney, Ermal Lu�i, and George V. Neville-Neil, of AES modes both in hardware and in software. Part of this work also includes more benchmarks undertaken using Conductor in the netperf project. Results have been reported at BSDCan and vBSDcon with more to come at EuroBSDcon and BSDCon Brasil.
This project was sponsored by Netgate, and The FreeBSD Foundation.
Open tasks:
-
Performance improvements and other tweaks are ongoing.
Architectures
Atomics
Contact:
Konstantin
Belousov
<[email protected]>
Contact:
Alan
Cox
<[email protected]>
Contact:
Bruce
Evans
<[email protected]>
Atomic operations serve two fundamental purposes. First, they are the building blocks for expressing synchronization algorithms in a single, machine-independent way using high-level languages. In essense, atomics abstract the different building blocks supported by the various architectures on which FreeBSD runs, making it easier to develop and reason about lock-less code by hiding hardware-level details.
Atomics also provide the barrier operations that allow software to control the effects on memory of out-of-order and speculative execution in modern processors as well as optimizations by compilers. This capability is especially important to multithreaded software, such as the FreeBSD kernel, when running on systems where multiple processors communicate through a shared main memory.
Each machine architecture defines a memory model, which specifies the possible effects on memory of out-of-order and speculative execution. More precisely, it specifies the extent to which the machine may visibly reorder memory accesses to optimize performance. Unfortunately, there are almost as many models as architectures. Some architectures, for example IA32 or Sparcv9 TSO, are relatively strongly ordered. In contrast, others, like PowerPC or ARM, are very relaxed. In effect, atomics define a very relaxed abstract memory model for FreeBSD's machine-independent code that can be efficiently realized on any of these architectures.
Most FreeBSD development and testing still happens on x86 machines, which, when combined with x86's strongly ordered memory model, leads to errors in the use of atomics, specifically, barriers. In other words, the code is not properly written to FreeBSD's abstract memory model, but the strong ordering of the x86 architecture hides this fact. The architectures impacted by the code that incorrectly uses atomics are less popular or have limited availability, and the resulting bugs from the misuse of atomics are hard to diagnose.
The goal of this project is to audit and upgrade the usage of lockless facilities, hopefully fixing bugs before they are observed in the wild.
FreeBSD defines its own set of atomics operations, like many other operating systems. But unlike other operating systems, FreeBSD models its atomics and barriers on the release consistency model, which is also known as acquire/release model. This is the same model which is used by the C11 and C++11 language standards as well as the new 64-bit ARM architecture. Despite having syntactical differences, C11 and FreeBSD atomics share essentially the same semantics. Consequently, ample tutorials about the C11 memory model and algorithms expressed with C11 atomics can be trivially reused under FreeBSD.
One facility of C11 that was missing from FreeBSD atomics was fences. Fences are bidirectional barrier operations which could not be expressed by the existing atomic+barrier accesses. They were added in r285283.
Due to the strong memory model implemented by x86 processors, atomic_load_acq() and atomic_store_rel() can be implemented by plain load and store instructions with only a compiler barrier. No additional ordering constraints are required. This simplification of atomic_store_rel() was done some time ago in r236456. The atomic_load_acq() change was done in r285934, after careful review of all its uses in the kernel and user-space to ensure that no hidden dependency on a stronger implementation was left.
The only reordering in memory accesses which is allowed on x86 is that loads may be reordered with older stores to different locations. This results from the use of store buffers at the micro-architecural level. So, to ensure sequentially consistent behavior on x86, a store/load barrier needs to be issued, which can be done with an MFENCE instruction or by any locked read-modify-write operation. The latter approach is recommended by the optimization guides from Intel and AMD. It was noted that careful selection of the scratch memory location, which is modified by the locked RMW operation, can reduce the cost of the barrier by avoiding false data dependencies. The corresponding optimization was committed in r284901.
The atomic(9) man page was often a cause of confusion due to both erroneous and ambiguous statements. The most significant of these issues were addressed in changes r286513 and r286784.
Some examples of our preemptive fixes to the misuse of atomics that would only become evident on weakly ordered machines are:
- A very important lockless algorithm, used in both the kernel and libc, is the timekeeping functionality implemented in kern/kern_tc.c and the userspace __vdso_gettimeofday. This algorithm relied on x86 total store order (TSO) behavior. It was fixed in r284178 and r285286.
- The kern/kern_intr.c lockless updates to the it_need indicator were corrected in r285607.
- An issue with kern/subr_smp.c:smp_rendezvous_cpus() not guaranteeing the visibility of updates done on other CPUs to the caller was fixed in r285771.
- The pthread_once() implementation was fixed to include missed barriers in r287556.
This project was sponsored by The FreeBSD Foundation (Konstantin Belousov's work).
FreeBSD on Cavium ThunderX (arm64)
Contact:
Dominik
Ermel
<[email protected]>
Contact:
Wojciech
Macek
<[email protected]>
Contact:
Zbigniew
Bodek
<[email protected]>
Cavium’s ThunderX is a high-performance 64-bit ARMv8 CPU, available in configurations with up to 48 cores per package. ThunderX is the initial reference platform for the FreeBSD/arm64 porting effort.
Additional Semihalf-sponsored work on ThunderX support brought brand new features such as:
- Multi-socket operation: FreeBSD now runs on a two-node ThunderX server board with a total of 96 CPU cores!
- Virtual Networking Interface Card driver: The VNIC driver consists of 4 elements (BGX, MDIO, and Physical and Virtual Functions) and is the second driver in FreeBSD to utilize SR-IOV capabilities. ThunderX is now able to use built-in networking interfaces at 1–40 Gbps.
Moreover, previously introduced functionalities have been improved and committed to HEAD. This includes:
- PCIe drivers for both internal and external controllers
- ITS (Interrupt Translation Services) fixes
- Platform-specific changes for ThunderX
- Various other fixes to the kernel (PCI, UMA, etc.)
The remaining features are being reviewed and will be integrated into HEAD soon. However, the GENERIC kernel already supports and runs on ThunderX.
This project was sponsored by The FreeBSD Foundation, ARM Ltd., Cavium, and Semihalf.
Open tasks:
-
Upstream the remaining features: 2-socket support, VNIC driver, and PCIe fixes
FreeBSD on the HiKey ARMv8 Board
Links | |
HiKey wiki entry | URL: https://wiki.FreeBSD.org/arm64/HiKey |
Hardware description | URL: https://www.96boards.org/products/ce/hikey/ |
Contact: Andrew Turner <[email protected]>
The HiKey is a low-cost ARMv8 development board from the Linaro 96boards initiative. It contains a HiSilicon Kirin 6220 with eight ARMv8 cores and 1GB of ram.
FreeBSD has been ported to run on the HiKey with a minimal set of drivers. As of this report, FreeBSD supports the micro-SD slot and USB host, and will boot off the SD card to multi-user mode using a recent arm64 snapshot.
The kernel is missing a number of device drivers. However, it is at a usable state for people interested in testing FreeBSD on ARMv8 hardware.
This project was sponsored by ABT Systems Ltd, and ARM Ltd.
Open tasks:
-
A driver for SDIO and the onboard WiFi.
-
Fix the MMC driver to access the eMMC.
-
Support the USB in OTG mode.
-
Support a display via HDMI.
-
Add thermal management.
FreeBSD/arm64
Links | |
FreeBSD arm64 wiki page | URL: https://wiki.FreeBSD.org/arm64 |
Contact:
Andrew
Turner
<[email protected]>
Contact:
Ed
Maste
<[email protected]>
Numerous cleanups and fixes have been applied to the arm64 kernel. This includes fixes to exception handling, asynchronous signals, ddb, and pmap. ddb has been updated to better handle accessing memory that may be unmapped. The pmap code was made more complete by implementing more functions as needed.
Further work on SMP means that FreeBSD now boots on all 48 cores on the Cavium ThunderX platform. This includes adding support for the ARM GICv3 interrupt controllers and fixing the memory mapping to be shareable between CPUs.
The test suite has been run on both qemu and hardware. Most of the test cases are passing, with around 30 tests either broken or failing. Work on diagnosing the issues with the remaining test cases is ongoing.
This project was sponsored by The FreeBSD Foundation, and ABT Systems Ltd.
Open tasks:
-
Port to more SoCs.
FreeBSD/RISC-V Port
Links | |
FreeBSD wiki RISC-V | URL: https://wiki.freebsd.org/riscv |
Single user boot log | URL: https://people.freebsd.org/~br/riscv-singleuser.txt |
Contact:
Ruslan
Bukin
<[email protected]>
Contact:
Arun
Thomas
<[email protected]>
Contact:
Ed
Maste
<[email protected]>
RISC-V is an open source Instruction Set Architecture (ISA) designed at UC Berkeley. It is freely available for all uses without requiring fees or license agreements. The RISC-V team intends to provide freely available BSD licensed CPU designs.
Ruslan Bukin (University of Cambridge) now has FreeBSD booting to a single user shell on a RISC-V simulator. The porting effort started only two months ago and is very much a work in progress, requiring significant refactoring and clean up before it reaches a committable state. Nonetheless, this is exceptional progress in a short time. The porting effort also identified a number of proposed ISA improvements.
The port currently uses the GNU tool chain (GCC and binutils), and runs on the Spike simulator. Improved RISC-V support in Clang/LLVM and related tools is highly desired.
This project was sponsored by DARPA, AFRL.
Userland Programs
mandoc and roff Toolchain
Links | |
Heirloom doctools | URL: https://github.com/n-t-roff/heirloom-doctools |
mandoc | URL: http://mdocml.bsd.lv/ |
Contact: Baptiste Daroussin <[email protected]>
mandoc is a suite of tools for compiling mdoc, the roff macro language of choice for BSD manual pages.
mandoc is the default renderer for manpages on FreeBSD head. This quarter, the apropos(1) utility was switched to use mandoc's version, which offers a new database format (in SQLite) bringing more powerful, fine-grained ways to search man pages.
While mandoc is very good for man pages, we also provide lots of other documentation in plain roff format. The Heirloom toolchain is being studied to replace groff in base. The Heirloom nroff toolchain has multiple benefits: it has very good unicode support and very good compatibility with groff.
A great deal of work as been done testing the Heirloom nroff toolchain with all the roff documents in the base system (including man pages), and upstream has been very proactive in fixing reported bugs.
The soelim(1) utility has been replaced with a BSD-licensed version which is good enough to work with all available roff toolchains to ease the transition. This version of the soelim(1) utility, originally written solely for FreeBSD, is now part of the mandoc tool suite.
In coordination with Ingo Schwarze from OpenBSD, the col(1) utility has been cleaned up and updated to recognize both SUSv2-style escape-digit and BSD-style escape-control-char sequences in the input stream.
The checknr(1) utility has been cleaned up and extended to support modern roff(7) macros, including synchronizing code from NetBSD and the Heirloom doctools version.
Many roff fixes were made to documentation and man pages, having been discovered while testing the new toolchain.
pkg 1.6
Contact: FreeBSD pkg Team <[email protected]>
pkg 1.6.0 has been released. Many changes have been made since pkg 1.5:
- The dependency solver is greatly improved
- Lots of fixes in the three-way merge code
- pkg add can now work without a version specified in the dependency line
- pkg check -d now also checks the required libraries
- Improved support for partial upgrades
- Improved zsh completion support
- Improved Linux support: all regression tests now pass on Linux
- Messages can now be context-aware, showing a given message always, or only during installation, upgrade (conditional on the previous version), or removal
- @keywords now accept new entries to add context-aware messages
- Added the ability to generate graphiz's dot format representation of the solver's problem
- pkg search now defaults to showing the pkg-comments of the matched packages
- Lots of bug fixes and code cleanup
- Improvements in cross-installation support
Open tasks:
-
Add a notion of priority to the list of files to ensure that certain files are the first to be replaced. This was a blocker for packaging base.
-
Investigate replacing openssl by mbedtls.
sesutil(8)
Links | |
Wikipedia: SCSI Enclosure Services (SES) | URL: https://en.wikipedia.org/wiki/SCSI_Enclosure_Services |
Contact:
Baptiste
Daroussin
<[email protected]>
Contact:
Allan
Jude
<[email protected]>
sesutil(8) was originally created as a more universal way to blink the "locate" LEDs on most hot-swappable drive enclosures.
This work is based on the original SES tools created by Matthew Jacob in 2000, which have been available in the share/examples section of the source tree, but were not built by default.
The new utility extends the original code with a number of very useful features:
- Print a map of all objects connected to the SES controller
- Map device names (/dev/da5) to SES slot number
- Blink the Locate and/or Fault LED of a drive by its SES slot number or device name
- Check the status of the entire SES controller
This project was sponsored by Gandi, and ScaleEngine Inc..
Open tasks:
-
Test sesutil(8) against more hardware.
-
Diagnose an issue where the locate command sometimes needs to be sent twice to activate the LED.
-
Add support for libxo output types.
truss(1)
Contact:
John
Baldwin
<[email protected]>
Contact:
Bryan
Drewery
<[email protected]>
The interface between the ABI-specific backends and the truss core was refactored, reducing duplicated code. This prompted additional follow-on work to add support for more ABIs, including aarch64 and CloudABI.
ptrace(2) was extended to return more information about the currently executing system call. This restored behavior that had been present in a previous version of truss: knowing the correct number of arguments for all system calls.
The fork-following support in truss was reworked to use native fork following in ptrace(2) rather than forking a new truss process for each child of a traced process.
Support for decoding more arguments has been added in the last quarter as well.
Open tasks:
-
Create a new libsysdecode library to hold shared code between truss(1) and kdump(1).
-
Decode more system call arguments.
-
Add appropriate system call decoding specifications for freebsd32 system calls.
-
Implement an ABI for 64-bit Linux binaries under FreeBSD/amd64.
Updates to GDB
Links | |
Extend libkvm to support cross-debugging of vmcores | URL: https://reviews.FreeBSD.org/D3341 |
Contact: John Baldwin <[email protected]>
Support for following children after forks for FreeBSD was implemented and merged upstream to GDB's master branch, and was included in GDB 7.10.
Work has continued on porting kgdb to newer gdb. The amd64, i386, powerpc, powerpc64, and sparc64 backends have all been ported and are now available via a new KGDB option in the devel/gdb port.
The MD backends for libkvm have been rewritten to support cross-debugging crashdumps, and the kgdb targets for amd64 and i386 have been reworked to support cross-debugging. Both i386 and amd64 kgdb binaries have been able to cross-debug the other architecture's vmcores with these changes. This changeset for libkvm is not yet in the tree, but is awaiting more testing.
Open tasks:
-
Test the libkvm changes on platforms other than amd64, i386, and powerpc64.
-
Figure out why the powerpc kgdb targets are not able to unwind the stack past the initial frame.
-
Add support for more platforms (arm, mips, aarch64) to upstream gdb for both userland and kgdb.
-
Write a new 1:1-only thread target for FreeBSD that can be sent upstream.
-
Add support for debugging powerpc vector registers.
Ports
Bringing GitLab into the Ports Collection
Contact:
Torsten
Z�hlsdorff
<[email protected]>
Contact:
Michael
Fausten
<[email protected]>
GitLab is a web-based Git repository manager with many features, used by more than 100.000 organizations, including NASA and Alibaba. It also is a very long-standing entry on the "Wanted Ports" list on the FreeBSD Wiki.
In the last month there was steady progress, finally resulting in the PR for adding the new port. In addition to the many dependencies Philip M. Gollucci is working on, there was already a large amount of work done. Along with many new or updated rubygems, Rails 4.1 was resurrected. A large group of committers were involved in the process and guided us through the various problems and pitfalls.
Because of the number of dependencies — we nearly hit 100 — making progress takes some time. In the meantime, a new major version of GitLab has already been released, requiring even more dependencies and updates. Work on this version is in progress, but the first goal is to get the latest stable version from the 7.14 branch into the ports tree.
This project was sponsored by anyMOTION GRAPHICS GmbH, D�sseldorf, Germany.
Open tasks:
-
Closing all the PRs of the dependencies
-
Committing the GitLab port itself
-
Updating the port to the latest version of the 8.x branch
GNOME on FreeBSD
Contact: FreeBSD GNOME Team <[email protected]>
The FreeBSD GNOME Team maintains the GNOME, MATE, and CINNAMON desktop environments and graphical user interfaces for FreeBSD. GNOME 3 is part of the GNU Project. MATE is a fork of the GNOME 2 desktop. CINNAMON is a desktop environment using GNOME 3 technologies but with a GNOME 2 look and feel.
This quarter, GNOME 3.16 and MATE 1.10 were committed to the ports tree, followed up by some incremental improvements. A chapter covering the use of USE_GNOME within individual ports' Makefiles was written and committed to the Porter's Handbook.
GNOME 3.18 has been ported. There are, however, some issues that need to be resolved before it can be committed to the ports tree.
Open tasks:
-
The FreeBSD GNOME website is stale. Work is under way to improve it.
-
Please give feedback on and suggest improvements to the chapter in the Porter's Handbook on the USE_GNOME functionality.
-
Continue working on investigating the issues blocking GNOME 3.18.
KDE on FreeBSD
Contact: KDE on FreeBSD team <[email protected]>
Overall, we have updated the following ports this quarter:
- CMake 3.3.1 (r396266)
- Qt 4.8.7 (r397043)
- QtCreator 3.5.0 (r395935)
- Fixed some dependencies, typos and plists in Qt5-ports (r396044-r396047), spotted by Ralf Nolden
In our development repository, we have done the following work:
- Updated PyQt-bindings for qt4 to 4.11.4 and added qt5 bindings 5.5, contributed by Guido Falsi, and modified by Tobias Berner (area51)
- Updated qt5 to 5.5.0. Ralf Nolden has contributed a handful of useful new ports, for example lang/qt5-l10n (area51/qt-5.5)
- The plasma5 branch has been kept up to date with KDE's upstream and contains ports for Frameworks 5.14.0, Plasma Desktop 5.4.2 and Applications 15.08.1 (area51/plasma5)
Open tasks:
-
Work on getting the stuff from plasma5 branch into ports. (This is a major update to nearly all KDE applications, so testers are very welcome.)
-
Finalize the work on PyQt5.
-
Port qt5-webengine. Qt-5.5 will probably be the last release shipping a www/webkit-qt5 port.
Node.js Modules
Links | |
Node.js modules | URL: https://www.assembla.com/spaces/cozycloud/subversion/source |
Pre-draft documentation | URL: https://people.FreeBSD.org/~olivierd/porters-handbook/using-nodejs.html |
Contact: Olivier Duchateau <[email protected]>
Node.js is a platform built on Chrome's JavaScript runtime for easily building fast, scalable network applications. It uses an event-driven, non-blocking I/O model that makes it lightweight and efficient, perfect for data-intensive real-time applications that run across distributed devices.
The goal of this project is to make it easy to install the modules available in the npm package registry.
Currently, the repository contains more than 100 new ports, in particular:
- CoffeeScript (a programming language that transcompiles to JavaScript)
- node-gyp (allows building Node.js addons, often written in C or C++)
- Request (a simplified HTTP client)
We have also written several helpers for the porting, available in our experimental repository.
Open tasks:
-
Bring in grunt.js (and modules), the JavaScript task runner.
-
Put more effort into support for node-gyp in the USES framework
Ports Collection
Contact:
Frederic
Culot
<[email protected]>
Contact: FreeBSD Ports Management Team <[email protected]>
As of the end of Q3 the ports tree holds just over 25,000 ports, and the PR count is above 2,000. The summer period saw less activity on the ports tree than during the previous quarter, with fewer than 7,000 commits performed by 120 active committers. Unfortunately, the number of problem reports closed also decreased significantly, with fewer than 1,500 problem reports fixed during Q3.
In Q3 several commit bits were taken in for safekeeping,
following an inactivity period of more than 18 months
(fluffy), or on committer's request (xmj, stefan, brix). One
new developer was granted a ports commit bit
(Jason Unovitch
On the management side, no changes were made to the portmgr team during Q3.
On the QA side, 25 exp-runs were performed to validate sensitive updates or cleanups. Amongst those, the noticeable changes are the update to pkg 1.6, the automake14 removal, and several important port updates such as doxygen to 1.8.10, gnome3 to 3.16, cmake to 3.3.1, and the Qt4 ports to 4.8.7. The default jdk was also set to openjdk8. Some infrastructure changes included the addition of new options helpers: opt_VARS, opt_VARS_OFF, opt_IMPLIES, and opt_PREVENTS. Some macros were also removed, such as UNIQUENAME and LATEST_LINK.
Open tasks:
-
We would like to remind everyone that the ports tree is built and run by volunteers, and any help is greatly appreciated. This is more important than ever, since the number of problem reports cannot seem to stop increasing. So if you use ports or packages, please consider jumping in and helping! This is also true for existing porters: it would be great if you would consider the next step, which is to share your knowledge and mentor someone more junior with the ports tree internals. And if you already do these tasks, many thanks to you!
Ports on PowerPC
Contact: Alexey Dokuchaev <[email protected]>
The Ports Collection typically receives less attention on Tier-2 architectures than on Tier-1 architectures, although several build-runs were performed at various points in the past, and broken ports were marked as such at those times.
Some of the Tier-2 platforms, such as PowerPC and ARM, have improved considerably recently, both on FreeBSD's and the compilers' sides, but as the tree is not rebuilt on the cluster very often, it was suspected that many ports are marked BROKEN while they in fact now build and run correctly.
Over the past several weeks, 26 ports that were indeed broken on at least PowerPC had been fixed, 58 ports that were incorrectly marked as broken (leftovers from the old times) were marked as working, and fewer than 40 ports still have issues requiring further work.
Open tasks:
-
The Ports Collection could benefit a lot from more frequent sweeps targeting Tier-2 systems.
-
Recent work on QEMU-backed emulators and the much-anticipated cross-building of ports are essential pieces to bring FreeBSD packages on par with the base system's support, architecture-wise.
Xfce on FreeBSD
Links | |
FreeBSD Xfce Project | URL: https://wiki.FreeBSD.org/Xfce |
FreeBSD Xfce Repository | URL: https://www.assembla.com/spaces/xfce4/subversion/source |
Contact: FreeBSD Xfce Team <[email protected]>
Xfce is a free software desktop environment for Unix and Unix-like platforms, such as FreeBSD. It aims to be fast and lightweight, while still being visually appealing and easy to use.
During this quarter, the team has kept these applications up-to-date:
- science/xfce4-equake-plugin 1.3.8
- sysutils/xfce4-power-manager 1.5.2
- x11/libexo 0.10.7
- x11/xfce4-embed-plugin 1.6.0
- x11/xfce4-verve-plugin 1.1.0
- x11/xfce4-whiskermenu-plugin 1.5.1
- x11-wm/xfce4-desktop 4.12.3
- www/midori 0.5.11
We also follow the unstable releases (available in our experimental repository) of:
- sysutils/xfce4-panel-switch 1.0.2 (utility to backup panel layouts)
- x11/xfce4-dashboard 0.5.1
In the trunk branch, x11-wm/xfce4-panel contains a patch to support sysutils/xfce4-panel-switch (available through the panel preferences).
Open tasks:
-
Test the new stable release of GLib 2.46.x with the kqueue/kevent backend enabled (it was disabled with revision r393663). Currently several features are broken, especially in Thunar, xfce4-panel, and Xfdashboard.
Documentation
PO Translation Project
Contact: FreeBSD Documentation Team <[email protected]>
Contact: FreeBSD Translators <[email protected]>
Contact:
Warren
Block
<[email protected]>
The FreeBSD documentation translation process has been in need of modernization for quite some time. The existing process was just too difficult for translators to keep translations up to date.
With help from Benedict Reuschling, Shaun McCance, Ryan Lortie, Hiroki Sato, and many others, the availability of a new PO translation system was announced in August.
PO translations handle most of the overhead of the translation process. Translators do not have to keep track of commits to the upstream English version. The actual work of translating is quicker and easier. PO editors show how much of the document has been translated. If a translation is already available for a given string, it can be easily reused.
Early testing has been very successful. Most issues involve discovering and documenting the new processes rather than fixing bugs. New translations of English documents have already been committed.
There will certainly be additional changes and improvements, but the system works. We will continue to discover how to share work between translation teams and the project as a whole. This work will be much easier now that the initial hurdle of being able to use PO software has been passed.
Open tasks:
-
Continue testing. The system is new to us and there are bound to be bugs and situations with unexpected results.
-
Improve documentation on using the new PO translations. Much of this involves things that rarely happened with the old system, like adding a completely new language directory.
-
Add new translations for existing documents. There is much less work to create and update a translated document now. Existing and new translators are working on adding and updating translations of the English documentation.
-
Figure out how to generate and share translation memory with other members of a language team or translators outside the team.
-
Test new PO editors like Pootle and Virtaal.
-
Determine a method to allow translators commit access for translations.
-
Develop and test code to translate manual pages.
Website CSS Update
Links | |
FreeBSD Main Site | URL: https://www.FreeBSD.org/ |
Contact: Warren Block <[email protected]>
The FreeBSD website has remained essentially unchanged in appearance for many years. Like other legacy systems, it is difficult to change. It is heavily used and therefore subject to non-trivial bikeshedding.
The CSS shrunk the reader's font from the size they had requested. It specified hardcoded font and object sizes in pixels. On wide monitors, only the middle third of the screen was used. Hardware has changed from what existed when this version of the site was created. Screens have become larger and wider, and increased in resolution at the same time.
It was time for a change. Font sizes were set to percentages, with none smaller than 100%. The width of the main box was changed to 90%. Other small adjustments were added. These limited changes produced a rendered site that better respects the reader's settings, is much easier to read, and shows more information.
Although no content changed, the appearance was so different that some viewers thought we had redesigned the site. It is gratifying to know that so many people are using it. We would also like to thank people for the response, which was overwhelmingly positive and hardly bikesheddy at all.
Open tasks:
-
Fix other outdated assumptions in the CSS. Alternately, rework the entire site. However, that is a much more complex and ambitious project than it might seem.
Google Summer of Code
Allwinner A10/A20 Support
Links | |
Wiki page | URL: https://wiki.FreeBSD.org/FreeBSD/arm/Allwinner |
Contact:
Luiz Otavio
O Souza
<[email protected]>
Contact:
Pratik
Singhal
<[email protected]>
The Allwinner A10 and A20 chips are ARM CPUs found in increasingly common development boards and other devices, such as the Cubieboard/Cubieboard 2 and the Banana Pi.
With the end of a GSoC project by Pratik Singhal, our A10 and A20 support has improved. Pratik helped with the implementation and testing of the SD card and SATA support for the Allwinner chips.
Luiz Otavio O Souza added support for the dwc network interface on the A20, which is capable of gigabit speeds.
Glen Barber kindly added support for official FreeBSD images for Cubieboard 2 and the Banana Pi.
This project was sponsored by Google Summer of Code 2015 (partly).
Open tasks:
-
Some drivers are still missing: audio, video/HDMI/framebuffer, IR, I2C, SPI, PWM.
-
Fix if_dwc for better performance.
mtree Parsing and Manipulation Library
Links | |
Wiki page | URL: https://wiki.FreeBSD.org/SummerOfCode2015/mtreeParsingLibrary |
Contact:
Michal
Ratajsky
<[email protected]>
Contact:
Brooks
Davis
<[email protected]>
FreeBSD includes several programs that work with file system hierarchy descriptions in the mtree(5) format. These descriptions, also called specifications, have a broad range of uses, from automatically creating directory structures to security auditing.
Each of the programs, namely mtree, bsdtar, install, and makefs, has its own implementation of the mtree format. This not only adds maintenance overhead, but also makes interoperability difficult, as each of the implementations only supports a limited subset of the format.
The goal of this project was to create a new libmtree library, implementing everything the mtree format has to offer, and wrapping it with an expressive API which all the listed programs can use. We also wanted libmtree to be portable, as one of the major users of the mtree format is libarchive, the library implementing most of bsdtar.
Currently, the library is functionally complete, ready to be downloaded and receive everyone's attention. We have also decided to bundle the mtree program along with it. The bundled mtree has also been modified for better portability.
The project included modifying libarchive, install and makefs to use libmtree. These modified versions are also available.
Please see the Wiki page for more information, download locations, and an example of using the libmtree API.
This project was sponsored by Google Summer of Code 2015.
Open tasks:
-
Test and review the library code and API, and the modifications made to the programs.
-
Fix the known problems that are mentioned on the Wiki page.
Multiqueue Testing
Links | |
Project Wiki Page | URL: https://wiki.FreeBSD.org/SummerOfCode2015/MultiqueueTestingProject |
Contact:
Tiwei
Bie
<[email protected]>
Contact:
Hiren
Panchasara
<[email protected]>
Contact:
George
Neville-Neil
<[email protected]>
Contact:
Robert
Watson
<[email protected]>
The aim of this project is to design and implement infrastructure to validate that a number of the network stack's multiqueue behaviours are functioning as expected.
At present, most of this project has been implemented. It mainly consists of two parts:
- A general mechanism to collect the per-ring per-cpu statistics that can be used by all NIC drivers, and extensions to netstat(1) to report these statistics.
- A suite of network stack behavior testing programs that
consists of:
- a virtual multiqueue ethernet interface (vme)
- a UDP packet generator based on vme
- a UDP server based on socket(2)
- a TCP client based on lwip and vme
- a TCP server based on socket(2)
However, it still needs further refinements to make it suitable for committing to FreeBSD head.
This project was sponsored by Google Summer of Code 2015.
Update Ficl in Bootloader
Links | |
Wiki Page | URL: https://wiki.FreeBSD.org/SummerOfCode2015/UpdateFiclInBootloader |
Contact: Colin Lord <[email protected]>
The FreeBSD bootloader has used Ficl 3 for quite some time. This project was intended to update the version of Ficl in use to Ficl 4. Ficl 4 is not only faster but also has a smaller memory footprint, both being important advantages for a bootloader.
As part of the Google Summer of Code program, I worked on importing the Ficl 4 sources to get a bootloader running Ficl 4. The first half of the summer consisted of setting up my test environment, as well as arranging the sources in the tree properly and modifying the build files to point to the new locations. Once that was complete, the sources had to be modified to build correctly and to add back in some of the FreeBSD-specific parts from Ficl 3. Unfortunately, after all those tasks were completed, a few bugs in the Ficl project were discovered that delayed the bootloader update, so it is not finished. The Illumos project has faced similar issues with Ficl 4 so I received some good tips from them, but since school has started back up I have not been able to put much work into fixing the bugs.
This project was sponsored by Google Summer of Code 2015.
Miscellaneous
The FreeBSD Foundation
Links | |
Foundation website | URL: http://www.FreeBSDFoundation.org/ |
FreeBSD Journal | URL: http://freebsdjournal.com/ |
Contact: Deb Goodkin <[email protected]>
The FreeBSD Foundation is a 501(c)(3) non-profit organization dedicated to supporting and promoting the FreeBSD Project and community worldwide. Funding comes from individual and corporate donations and is used to fund and manage development projects, conferences and developer summits, and provide travel grants to FreeBSD developers. The Foundation purchases hardware to improve and maintain FreeBSD infrastructure and publishes FreeBSD white papers and marketing material to promote, educate, and advocate for the FreeBSD Project. The Foundation also represents the FreeBSD Project in executing contracts, license agreements, and other legal arrangements that require a recognized legal entity.
Here are some highlights of what we did to help FreeBSD last quarter:
Anne Dickison and Deb Goodkin attended OSCON to promote FreeBSD.
Robert Watson organized and ran the Cambridge FreeBSD Developer Summit 2015 ("BSDCam"). We provided travel grants to two FreeBSD developers to attend the summit. Three Foundation board/staff members attended too.
George V. Neville-Neil attended the ARM Partner Meeting where he met with 15 silicon and systems vendors to present the unique traits and qualities of FreeBSD and work on setting up partnerships with the companies building and deploying ARM hardware.
George and Robert Watson collaborated in Cambridge on developing further FreeBSD-based teaching material at undergraduate and masters levels. Part of this project was funded by the Foundation.
George planned and ran the DevSummit at vBSDCon 2015.
We were proud to be a sponsor of
Cheryl Blain and John Baldwin promoted the Foundation and FreeBSD at the SNIA 2015 Storage Developer Conference, in Santa Clara, California, Sept 21-24. The Foundation was also a sponsor.
We sponsored Andy Turner to attend Linaro Connect in San Francisco, Sept 21-25.
Ed Maste, our project development director, attended the X.Org Developer's Conference (XDC) in Toronto, Ontario.
We sponsored the 2015 nginx Conference and sent FreeBSD community member John Baldwin.
George Neville-Neil continued planning the
Benedict Reuschling and Erwin Lansing helped plan and organize the EuroBSDCon FreeBSD Developer Summit. This included setting up the working groups, securing the venue, and getting the T-shirts made.
Benedict helped organize, and he and Dru Lavigne participated
in the
Dru Lavigne taught a FreeBSD class in Berlin, Germany July 29-31.
We were a sponsor of
We continued to publish our monthly newsletters, keeping the community informed on what we are doing, including event recaps, testimonials, project updates, and upcoming events. We received testimonials from Microsoft, NYCBus, and ScaleEngine. We also continued to approach companies to provide us with testimonials to help promote their use of FreeBSD.
Anne Dickison rebooted the Faces of FreeBSD series and is working with FreeBSD contributors on writing their stories. She continued to produce more FreeBSD Swag and literature to promote FreeBSD, as well as advocating for FreeBSD over our social channels and with new partnerships.
We reached our 2015 goal of 10,000 FreeBSD Journal subscribers, and we published a new Open Journal article on our website, to help promote the Journal. We also started offering a new subscription bundle, where you can buy all the 2014 issues. The July/August issue was published.
Justin T. Gibbs began teaching a semester-long FreeBSD class at a middle school in Boulder, Colorado. We are using the BeagleBone Black (BBB) to run FreeBSD connected to Macs and PCs. We have received a lot of support, both internally, and from the Project, to get the FreeBSD images to work on the BBB with the Macs and PCs. It has been a great collaborative effort with community members, and this will help future classes in being able to support inexpensive platforms for teaching FreeBSD.
Work continued on creating a FreeBSD curriculum for a half day workshop. Hopefully this will be available in late Spring.
We provided legal support for the Project including granting trademark permission for some users and companies who requested permission to put the FreeBSD logo on their websites and marketing literature.
We met with commercial users to get their input on what they would like to see supported in FreeBSD. We also do this to help connect FreeBSD developers with commercial users to help facilitate collaboration.
FreeBSD Foundation employee and Release Engineer Glen Barber was extremely busy during this quarter, working on a number of exciting areas of the FreeBSD Project. Some of the highlights include:
- Code cleanup and bug fixes to several parts of the release build code, and finishing adding support for automatically uploading cloud provider images, which was merged to the stable/10 branch before the code freeze. The 10.2-RELEASE cycle spanned a 9-week timeframe overall, starting from the code slush.
- With the FreeBSD Release Engineering Team, released two BETA builds and three RC builds for the 10.2-RELEASE cycle, with the final release announced mid-August, two weeks ahead of the original schedule.
- With the FreeBSD Cluster Administrators Team, assisted with a number of general updates and enhancements to the FreeBSD infrastructure.
ZFSguru
Links | |
Home page | URL: http://zfsguru.com |
Forum post on Gnome�3 debugging | URL: http://zfsguru.com/forum/zfsgurudevelopment/1038 |
Contact: Jason Edwards <[email protected]>
ZFSguru started as a front-end to ZFS but has since grown into a multifunctional server appliance with its own unique features. While the project is still in early development, it already offers multiple unique features not found in other projects. Unlike similar projects, nothing is stripped away from the base operating system, meaning ZFSguru behaves as a normal FreeBSD installation and thus is very versatile. The web-interface is designed to unite both novice and advanced users, providing both very easy to use basic functionality as well as features to be appreciated by more experienced users. The modular nature of the project combats the danger of bloat, whilst still allowing extended functionality to be easily deployed.
On the 15th of August, version 0.3 of ZFSguru was released. Some highlights of the new version:
- New build infrastructure allows for frequent releases of system images and services in a semi-automated way.
- A new GuruDB database allows for a growing number of system images and servers, and provides good caching to accelerate pages.
- The installation procedure was given a major overhaul.
- In addition to the LiveCD, USB images are now available. The USB image supports both legacy MBR bootcode and UEFI boot.
- Many libraries in the web-interface have been overhauled, in addition to many other additions to the web interface.
- Many improvements were made to optional add-on services, such as the new Gnome 3 graphical environment.
Other progress made in the months July, August, and September:
- System image builds 001, 002, 003, and 004 have been released for all supported branches: 10.0, 10.1, 10.2, 10.3 (-STABLE), and 11.0 (-CURRENT).
- Work on the 0.4 web-interface has started, which focuses on improving network support in the web-interface.
- Work on a new visual theme for the web-interface has started. The new interface is likely to be included in the upcoming 0.4 release.
- A new master server is being prepared, which is likely to be operational in December.
- A new website is being worked on, to be launched the first of January, 2016.
Open tasks:
-
The new Gnome�3 desktop does not work for everyone and still has issues. Anyone capable of diagnosing these issues can give the Gnome�3 LiveCD a try. Please see the linked forum thread for more information.
-
Several ports fail to compile with our own build infrastructure, and require bug reports in order to get them fixed upstream.
-
A 'State of the Project 2015' is due in Q4, providing an overview for future development of the project.
News Home | Status Home