Introduction
As spring leads into summer, we reflect back on what the FreeBSD project has accomplished in the first quarter of 2019. Events included FOSDEM and AsiaBSDCon, the FreeBSD Journal is now free to everyone, ASLR is available in -CURRENT and KPTI can be controlled per-process. The run up to 11.3-RELEASE has begun, and a team is applying syzkaller guided fuzzing to the kernel, plus so much more. Catch up on many new and ongoing efforts throughout the project, and find where you can pitch in.
FreeBSD Team Reports
- Continuous Integration
- FreeBSD Core Team
- FreeBSD Foundation
- FreeBSD Release Engineering Team
- Ports Collection
Projects
- AXP803 PMIC driver update
- Broadcom ARM64 SoC support
- C Runtime changes
- Capsicum
- CFT - Package Base
- ENA FreeBSD Driver Update
- FreeBSD boot security improvements
- FUSE
- Kernel ZLIB Update
- LLVM's lld as the FreeBSD system linker
- mlx5 Drivers Update
- PCI Express Resets
- Security-Related changes
Architectures
Ports
Third-Party Projects
- FreeBSD Wiki Apple Intel Mac mini update
- Fuzzing FreeBSD with syzkaller
- sysctlmibinfo API 1.0
- sysctlview 1.0
- University of Waterloo Co-operative Education Students
FreeBSD Team Reports
Entries from the various official and semi-official teams, as found in the Administration Page.
Continuous Integration
Contact: Jenkins Admin <[email protected]>
Contact: Li-Wen Hsu <[email protected]>
The FreeBSD CI team maintains continuous integration system and related tasks for the FreeBSD project. The CI system regularly checks the changes committed to the project's Subversion repository can be successfully built, and performs various tests and analysis of the results. The results from build jobs are archived in an artifact server, for the further testing and debugging needs. The CI team members examine the failing builds and unstable tests, and work with the experts in that area to fix the code or adjust test infrastructure.
Starting from this quarter, we started to publish CI weekly report at freebsd-testing@ mailing list. The archive is available at https://hackfoldr.org/freebsd-ci-report/
We also worked on extending test executing environment to improve the code coverage, temporarily disabling flakey test cases, and opening tickets to work with domain experts. The details are of these efforts are available in the weekly CI reports.
We published the draft FCP for CI policy and are ready to accept comments.
Please see freebsd-testing@ related tickets for more information.
Work in progress:
- Fixing the failing test cases and builds
- Adding drm ports building test against -CURRENT
- Implementing automatic tests on bare metal hardware
- Implementing the embedded testbed
- Planning for running ztest and network stack tests
- Help more 3rd software get CI on FreeBSD through a hosted CI solution
FreeBSD Core Team
Contact: FreeBSD Core Team <[email protected]>
The FreeBSD Core Team is the governing body of FreeBSD.
Core initiated a Release Engineering Charter Modernization working group. The purpose of the working group is to present (to Core) a modernized version of the Release Engineering Charter and a first version of a new Release Engineering Team Operations Plan. The group hopes to complete its goals and dissolve by 2019-06-30.
The Core Team invites all members of the FreeBSD community to complete the 2019 FreeBSD Community Survey.
https://www.research.net/r/freebsd2019
The purpose of the survey is to collect quantitative data from the public in order to help guide the project's priorities and efforts. It will remain open for 17 days and close at midnight May 13 UTC (Monday 5pm PDT). (Editor's note: Survey has finished)
Core voted to approve source commit bits for Johannes Lundberg (johalun@) and Mitchell Horne (mhorne@) and associate membership for Philip Jocks. Core also voted to revoke Michael Dexter's documentation bit.
After a long lapse of not closing idle source commit bits, core has taken in the commit bit for these developers. We thank each for contributing to the project as a source committer.
- Alfred Perlstein (alfred@)
- Eric Badger (badger@)
- Daniel Eischen (deischen@)
- Ermal Lu�i (eri@)
- Tony Finch (fanf@)
- Justin T. Gibbs (gibbs@)
- Imre Vad�sz (ivadasz@)
- Julio Merino (jmmv@)
- John W. De Boskey (jwd@)
- Kai Wang (kaiw@)
- Luigi Rizzo (luigi@)
- Neel Natu (neel@)
- Craig Rodrigues (rodrigc@)
- Stanislav Sedov (stas@)
- Thomas Quinot (thomas@)
- Andrew Thompson (thompsa@)
- Pyun YongHyeon (yongari@)
- Zbigniew Bodek (zbb@)
FreeBSD Foundation
Contact: Deb Goodkin <[email protected]>
The FreeBSD Foundation is a 501(c)(3) non-profit organization dedicated to supporting and promoting the FreeBSD Project and community worldwide. Funding comes from individual and corporate donations and is used to fund and manage software development projects, conferences and developer summits, and provide travel grants to FreeBSD contributors.
The Foundation purchases and supports hardware to improve and maintain FreeBSD infrastructure and provides resources to improve security, quality assurance, and release engineering efforts; publishes marketing material to promote, educate, and advocate for the FreeBSD Project; facilitates collaboration between commercial vendors and FreeBSD developers; and finally, represents the FreeBSD Project in executing contracts, license agreements, and other legal arrangements that require a recognized legal entity.
Here are some highlights of what we did to help FreeBSD last quarter:
We kicked off the year with an all-day board meeting in Berkeley, where FreeBSD began, to put together high-level plans for 2019. This included prioritizing technologies and features we should support, long-term planning for the next 2-5 years, and philosophical discussions on our purpose and goals.
Partnerships and Commercial User Support
We began the year by meeting with a few commercial users, to help them navigate working with the Project, and understanding how they are using FreeBSD. We're also in the process of setting up meetings for Q2 and throughout the rest of 2019. Because we're a 501(c)(3) non-profit, we don't directly support commercial users. However, these meetings allow us to focus on facilitating collaboration with the community.
Fundraising Efforts
Our work is 100% funded by your donations. We kicked off the year with many individual and corporate donations, including donations and commitments from NetApp, Netflix, Intel, Tarsnap, Beckhoff Automation, E-Card, VMware, and Stormshield. We are working hard to get more commercial users to give back to help us continue our work supporting FreeBSD. Please consider making a donation to help us continue and increase our support for FreeBSD at: www.FreeBSDfoundation.org/donate/.
We also have the Partnership Program, to provide more benefits for our larger commercial donors. Find out more information at https://www.FreeBSDfoundation.org/FreeBSD-foundation-partnership-program/ and share with your companies!
OS Improvements
The Foundation improves the FreeBSD operating system by employing our technical staff to maintain and improve critical kernel subsystems, add features and functionality, and fix problems. This also includes funding separate project grants like the arm64 port, porting the blacklistd access control daemon, and the integration of VIMAGE support, to make sure that FreeBSD remains a viable solution for research, education, computing, products and more.
Over the quarter there were 241 commits from nine Foundation-sponsored staff members and grant recipients.
We kicked off or continued the following projects last quarter:
- FUSE file system kernel support (update and bug fixes)
- Linuxulator testing and diagnostics improvements
- SDIO and WiFi infrastructure improvements
- x86-64 scalability and performance improvements
- OpenZFS Online RAID-Z Expansion
Having software developers on staff has allowed us to jump in and work directly on projects to improve FreeBSD like:
- amd64 and i386 pmap improvements and bugfixes
- address userland threading library issues
- improve i386 support to keep the platform viable
- improve FreeBSD on RISC-V
- application of the Capsicum sandboxing framework
- build system improvements and bug fixes
- respond to reports of security issues
- implement vulnerability mitigations
- tool chain updates and improvements
- adding kernel code coverage support for the Syzkaller coverage-guided system call fuzzer
- improved Syzkaller support for FreeBSD
- improve the usability of freebsd-update
- improve network stack stability and address race conditions
- ensure FreeBSD provides userland interfaces required by contemporary applications
- implement support for machine-dependent optimized subroutines
- update and correct documentation and manpages
- DTrace bug fixes
- update the FreeBSD Valgrind port and try to upstream the changes
Continuous Integration and Quality Assurance
The Foundation provides a full-time staff member who is working on improving our automated testing, continuous integration, and overall quality assurance efforts.
During the first quarter of 2019, Foundation staff continued improving the project's CI infrastructure, working with contributors to fix failing build and test cases, and working with other teams in the project for their testing needs. In this quarter, we started publishing the CI weekly report on the freebsd-testing@ mailing list.
See the FreeBSD CI section of this report for more information.
Release Engineering
The Foundation provides a full-time staff member to oversee the release engineering efforts. This has provided timely and reliable releases over the last five years.
During the first quarter of 2019, the FreeBSD Release Engineering team continued providing weekly development snapshots for 13-CURRENT, 12-STABLE, and 11-STABLE.
In addition, the Release Engineering team published the schedule for the upcoming 11.3-RELEASE cycle, the fourth release from the stable/11 branch, which builds on the stability and reliability of 11.2-RELEASE.
The upcoming 11.3-RELEASE schedule can be found at: https://www.freebsd.org/releases/11.3R/schedule.html
FreeBSD 11.3 is currently targeted for final release in early July 2019.
Please see the FreeBSD Release Engineering Team section of this quarterly status report for additional details surrounding the above mentioned work.
Supporting FreeBSD Infrastructure
The Foundation provides hardware and support to improve FreeBSD infrastructure. Last quarter, we continued supporting FreeBSD hardware located around the world.
FreeBSD Advocacy and Education
A large part of our efforts are dedicated to advocating for the Project. This includes promoting work being done by others with FreeBSD; producing advocacy literature to teach people about FreeBSD and help make the path to starting using FreeBSD or contributing to the Project easier; and attending and getting other FreeBSD contributors to volunteer to run FreeBSD events, staff FreeBSD tables, and give FreeBSD presentations.
The FreeBSD Foundation sponsors many conferences, events, and summits around the globe. These events can be BSD-related, open source, or technology events geared towards underrepresented groups. We support the FreeBSD-focused events to help provide a venue for sharing knowledge, to work together on projects, and to facilitate collaboration between developers and commercial users. This all helps provide a healthy ecosystem. We support the non-FreeBSD events to promote and raise awareness of FreeBSD, to increase the use of FreeBSD in different applications, and to recruit more contributors to the Project.
Check out some of the advocacy and education work we did last quarter:
- Attended FOSDEM 2019 where we: staffed the FreeBSD Stand, sponsored the co-located FreeBSD Developer Summit, and gave the 25 Years of FreeBSD presentation in the BSD Dev room.
- Sponsored and presented at SANOG33 in Thimphu, Bhutan
- Represented FreeBSD at APRICOT 2019 in Yuseong-gu, Daejeon South Korea
- Sponsored the USENIX FAST conference in Boston, MA as an Industry Partner
- Ran our first ever FreeBSD track at SCALE 17x, which included an all-day Getting Started with FreeBSD workshop. We were thrilled with the turnout of almost 30 participants and received a lot of positive feedback. Thanks to Roller Angel who taught the class with the help of Deb Goodkin and Gordon Tetlow. We also promoted FreeBSD at the FreeBSD table in the Expo Hall.
- Sponsored, presented, and exhibited at FOSSASIA in Singapore
- Sponsored AsiaBSDCon 2019
- Committed to sponsoring Rootconf, BSDCan, and EuroBSDcon
- Created registration systems for the Aberdeen Hackathon and the upcoming 2019 Vienna FreeBSD Security Hackathon
- Provided FreeBSD advocacy material
- Provided 3 travel grants to FreeBSD contributors to attend many of the above events.
We continued producing FreeBSD advocacy material to help people promote FreeBSD around the world.
Read more about our conference adventures in the conference recaps and trip reports in our monthly newsletters.
We help educate the world about FreeBSD by publishing the professionally produced FreeBSD Journal. We're excited to announce that with the release of the January/February 2019 issue, the FreeBSD Journal is now a free publication. Find out more and access the latest issues at www.FreeBSDfoundation.org/journal/.
You can find out more about events we attended and upcoming events at www.FreeBSDfoundation.org/news-and-events/.
We also engaged with a new website developer to help us improve our website to make it easier for community members to find information more easily and to make the site more efficient.
Legal/FreeBSD IP
The Foundation owns the FreeBSD trademarks, and it is our responsibility to protect them. We also provide legal support for the core team to investigate questions that arise.
Go to www.FreeBSDfoundation.org to find out how we support FreeBSD and how we can help you!
FreeBSD Release Engineering Team
| Links | |
| FreeBSD 11.3-RELEASE schedule | URL: https://www.freebsd.org/releases/11.3R/schedule.html |
| FreeBSD development snapshots | URL: https://download.freebsd.org/ftp/snapshots/ISO-IMAGES/ |
Contact: FreeBSD Release Engineering Team <[email protected]>
The FreeBSD Release Engineering Team is responsible for setting and publishing release schedules for official project releases of FreeBSD, announcing code freezes and maintaining the respective branches, among other things.
During the first quarter of 2019, the FreeBSD Release Engineering team published the initial schedule for the upcoming the 11.3-RELEASE.
FreeBSD 11.3-RELEASE will be the fourth release from the stable/11 branch, building on the stability and reliability of 11.2-RELEASE. FreeBSD 11.3-RELEASE is currently targed for release in early July, 2019.
Additionally throughout the quarter, several development snapshots builds were released for the head, stable/12, and stable/11 branches.
Much of this work was sponsored by the FreeBSD Foundation.
Ports Collection
Contact: Ren� Ladan <[email protected]>
Contact: FreeBSD Ports Management Team <[email protected]>
As always, below is a summary of what happened in the Ports Tree during the last quarter.
During 2019q1, the number of ports dropped slightly to just over 32,500. At the end of the quarter, we had 2092 open port PRs. The last quarter saw 8205 commits from 167 committers. So more PRs were closed and more commits were made than in 2018q4.
During the last quarter, we welcomed Kai Knoblich (kai@) and said goodbye to Matthew Rezny (rezny@).
On the infrastructure side, two new USES were introduced (azurepy and sdl) and USES=gecko was removed. The default versions of Lazarus and LLVM were bumped to 2.0.0 and 8.0 respectively. Some big port frameworks that were end-of-life were removed: PHP 5.6, Postgresql 9.3, Qt4, WebKit-Gtk and XPI. Firefox was updated to 66.0.2, Firefox-ESR to 60.6.1, and Chromium was updated to 72.0.3626.121.
During the last quarter, antoine@ ran 30 exp-runs for package updates, moving from GNU ld to LLVM ld, and switching clang to DWARF4.
Projects
Projects that span multiple categories, from the kernel and userspace to the Ports Collection or external projects.
AXP803 PMIC driver update
Contact: Ganbold Tsagaankhuu <[email protected]>
The AXP803 is a highly integrated PMIC that targets Li-battery (Li-ion or Li-polymer) applications. It provides flexible power management solution for processors such as the Allwinner A64 SoC. This SoC is used by Pinebook.
The following updates were performed on the AXP803 driver:
- Enabled necessary bits when activating interrupts. This allows reading some events from the interrupt status registers. These events are reported to devd via system "PMU" and subsystem "Battery", "AC" and "USB" such as plugged/unplugged, battery absent, charged and charging.
- Added sensors support for AXP803/AXP813. Sensor values such as battery charging, charge state, voltage, charging current, discharging current, battery capacity can be obtained via sysctl.
- Added sysctl for setting battery charging current. The charging current can be set using steps from 0 to 13. These steps correspond to 200mA to 2800mA, with a granularity of 200mA/step.
Broadcom ARM64 SoC support
Contact: Michal Stanek <[email protected]>
Contact: Marcin Wojtas <[email protected]>
The Semihalf team continued working on FreeBSD support for the Broadcom BCM5871X SoC series
BCM5871X are quad-core 64-bit ARMv8 Cortex-A57 communication processors targeted for networking applications such as 10G routers, gateways, control plane processing and NAS.
Completed since the last update:
- iProc PCIe root complex (internal and external buses)
- OTP (One Time Programmable memory) driver
In progress:
- BNXT Ethernet support
- Crypto engine acceleration for IPsec offloading.
Todo:
- Upstreaming of work. This work is expected to be submitted/merged to HEAD in the second half of 2019.
This project was sponsored by Juniper Networks, Inc.
C Runtime changes
Contact: Konstantin Belousov <[email protected]>
Several changes where made to the C runtime which generally improves the environment provided to an application.
Fix for libraries with initial exec TLS mode
Some libraries, most prominent of which is NVidia-provided and thus binary-only libGL.so.1, use so called initial exec mode for TLS variables access. This is the fastest mode of TLS access, but its drawback is that it only reliably work when the main binary is linked against the library, i.e. dlopen-ing the library to load it at runtime is not guaranteed to work.
This mode works by placing the TLS variables for objects in one area allocated during the executable initialization, which somewhat explains the name of the mode. An obvious consequence is that if such library is loaded later, there is no space in the TLS area for an application to put its TLS variables.
The FreeBSD dynamic linker is aware of misbehaviour of the app builders, and provides some amount of slack in the TLS area to give space for such libraries. But it appeared that the initial content of the TLS segment from libraries was not distributed among the threads' TLS areas, still breaking libraries which use initial exec mode for TLS.
Another issue that somewhat mitigates mis-use of the mode is the DF_STATIC_TLS flag in the dynamic section. This flag allows the linker to check for the space earlier and avoid loading dependencies if there is no total required space. This linker flag was implemented by the BFD ld linker, but not by the LLVM lld linker.
The FreeBSD dynamic linker was fixed to properly distribute TLS initialization data to all threads' initial segments, which required reasonably extensive per-architecture changes to libc and libthr. Simultaneously, LLD was improved to mark libraries using initial exec TLS mode with the appropriate flag.
These measures should make FreeBSD more resilent to improperly linked libraries. The most interesting fix is to users of the nvidia libgl library, because it cannot be fixed by relinking.
Use rtld malloc in libthr
The FreeBSD implementation of mutexes in libthr allocates some memory to keep the mutex data needed for mutex initialization. In contrast, the malloc implementation used by FreeBSD, jemalloc(3), requires working pthread mutexes for operation.
This creates a chicken-and-egg problem during executable startup, and requires jemalloc to provide fragile hacks to make it possible to initialize mutexes. This has been a constant source of mismatches on imports of new versions of jemalloc.
The FreeBSD rtld implementation already contained a very light-weight malloc implementation, suitable for limited use in pre-C-runtime environments. This seemed to be the ideal fit for an allocator for the pthread private mutexes memory. By using this allocator, a method to address the cyclic dependencies between jemalloc and libthr could finally be implemented.
The entry points in the rtld malloc.c were renamed to avoid a clash with the libc exported symbols, and now the file is linked statically into libthr, providing an allocator for private mutexes and pthread key storage. The later was already switched to direct use of mmap(2) for similar reasons. Now less memory is wasted when key storage requires less than a page.
Destructors order bug
Alexander Kabaev (kan@) noted that C++ destructors for the static objects from the linked shared libraries are executed before C++ destructors of the static objects from the main binary. This was verified both for clang++ and g++, but amusingly not for __attribute__(((destructor))).
The bug was introduced when init functions and init arrays for main binary startup are called from the rtld instead of csu (C startup code linked to the binary, typically from crt1.o). The cause is due to the somewhat complicated way of how destructors are called both by fini/fini arrays and rtld-registered atexit(3) handler.
Solution is to register rtld atexit(3) handler before main binary init functions are called, using new internal ABI __libc_atexit() function.
It is amusing that the bug was not noticed for so many years.
This project was sponsored by The FreeBSD Foundation.
Capsicum
| Links | |
| Capsicum Wiki Page | URL: https://wiki.FreeBSD.org/Capsicum |
Contact: Enji Cooper <[email protected]>
Contact: Mark Johnston <[email protected]>
Contact: Ed Maste <[email protected]>
Contact: Mariusz Zaborski <[email protected]>
Contact: Bora �zarslan <[email protected]>
Three themes for Capsicum work were:
- Importing Google's Capsicum test suite into FreeBSD
- Porting and sandboxing openrsync for FreeBSD
- Applying capsicum to additional base system utilities
The Googletest-based Capsicum test cases are now integrated into FreeBSD. After some discussion with David Drysdale - the main maintainer and developer for the Capsicum port on Linux - we decided that from now the FreeBSD will be upstream for Capsicum test cases.
The next major step was sandboxing openrsync. In the course of that work we extended our fileargs service with two new functionalities. We modified the fileargs service to allow limiting the operations which can be performed, and can now delegate lstat to the Casper service.
Furthermore, openrsync highly depends on the fts API. We spend some time in optimizing fts and making it sandbox friendly by introducing fts_openat function and removing the need to change the working directory to traverse the paths. The changes to the fts API are now in the tests phase.
Moreover, we improved bootstrapping for non-FreeBSD machines. Thanks to this work we can now build tools needed to bootstrap FreeBSD which use Casper services. In the base system strings is now sandboxed as a result.
We also sandboxed rtsol, rtsold, and savecore.
We host biweekly Capsicum calls. The notes from the meetings are published in FreeBSD's Capsium meeting repository on GitHub. If you would like to join the call do not hesitate to send us an email.
CFT - Package Base
| Links | |
| Package Base CFT - FAQ | URL: https://trueos.github.io/pkgbase-docs/ |
Contact: Kris Moore <[email protected]>
The TrueOS project has been working on a Package Base implementation, and is pleased to issue its first CFT to the FreeBSD community.
The TrueOS packaging work has been in development for close to 6 months, and differs from the original FreeBSD package base effort, in that it is an "out of tree" implementation. It allows any version of FreeBSD to be packaged, and only requires a patch to poudriere, as well as some minor ports enhancements, the first which is currently in review. For more information on the current status, please refer to the FAQ page.
Additionally there will be a working-group at BSDCan 2019, and we encourage porters to attend and join the discussion.
This project was sponsored by iXsystems Inc.
ENA FreeBSD Driver Update
| Links | |
| ENA README | URL: https://github.com/amzn/amzn-drivers/blob/master/kernel/fbsd/ena/README |
Contact: Michal Krawczyk <[email protected]>
Contact: Marcin Wojtas <[email protected]>
ENA (Elastic Network Adapter) is the smart NIC available in the virtualized environment of Amazon Web Services (AWS). The ENA driver supports multiple transmit and receive queues and can handle up to 100 Gb/s of network traffic, depending on the instance type on which it is used.
ENAv2 has been under development for FreeBSD, similar to Linux and DPDK. Since the last update internal review and improvements of the patches were done, followed by validation on various AWS instances.
To do:
- Upstream of the ENAv2 patches
Recently, AWS released the A1 instances which are arm64 instances. The FreeBSD kernel was fixed, so the ENA can be used on those instances with no issues. There were changes required in resource activation in the ENA driver r345371 and the addition of a missing bus release method to the nexus module for aarch64 r345373. With these changes, the ENA driver can run on A1 instances without any known issues.
This project was sponsored by Amazon.com Inc.
FreeBSD boot security improvements
Contact: Michal Stanek <[email protected]>
Contact: Marcin Wojtas <[email protected]>
Contact: Kornel Duleba <[email protected]>
FreeBSD gained TPM 2.0 (Trusted Platform Module) support at the end of 2018. A kernel configuration option, TPM_HARVEST, was also added to use the TPM RNG as system entropy source. When used this way, the TPM can be harvested every ten seconds for entropy which is mixed into the OS entropy pool. The kernel option is currently disabled by default in amd64 GENERIC kernel configuration.
UEFI Secure Boot support, developed by Semihalf, has been merged with sjg's Veriexec support, resulting in a unified library named libsecureboot. This library is used for verification of kernel and modules by the loader. The library uses BearSSL as the cryptographic backend. The library supports loading trusted and blacklisted certificates from UEFI (DB/DBx databases) and can use them as trust anchors for the verification.
The library is also used by Veriexec to verify and parse the authentication database (called 'manifest') in the kernel. Previously the manifest was verified and parsed by a userspace application, then sent to the kernel via /dev/veriexec, which was a significant limitation and a security weakness.
To do:
- Backport to stable branches.
Special thanks to sjg and Juniper for fruitful cooperation around Veriexec and the libsecureboot development.
This project was sponsored by Stormshield.
FUSE
Contact: Alan Somers <[email protected]>
FUSE (File system in USErspace) allows a userspace program to implement a file system. It is widely used to support out-of-tree file systems like NTFS, as well as for exotic pseudo file systems like sshfs. FreeBSD's fuse driver was added as a GSoC project in 2012. Since that time, it has been largely neglected. The FUSE software is buggy and out-of-date. Our implementation is about 11 years behind.
The FreeBSD Foundation has agreed to fund a project to improve the state of the FreeBSD FUSE driver. So far I've written a test suite for the fusefs(5) module, fixed 1 previously reported bug, discovered and fixed 6 new bugs, fixed all of fusefs's Coverity CIDs, made some minor performance enhancements and done some general cleanup. During the next quarter I plan to continue fixing bugs, and I'll also raise the driver's API level as high as I can before the quarter runs out. We're currently at 7.8; the highest defined level is 7.28.
This project was sponsored by The FreeBSD Foundation.
Kernel ZLIB Update
| Links | |
| Review D19706 | URL: https://reviews.freebsd.org/D19706 |
Contact: Yoshihiro Ota <[email protected]>
The FreeBSD system still uses an ancient (over 20 year-old) version of zlib (version 1.0.4). The FreeBSD kernel zlib implementation has special enhancements only used by netgraph. There is a separate version of code derived from unzip 5.12 used to inflate gzip files in the kernel which could be replaced with a more modern zlib. More detailed information is written in sys/modules/zlib/README in the review.
In order to use the latest zlib, version 1.2.11, work has been done to revisit all existing zlib uses in the system. Most of the code works with the newer version of zlib as is. The unzip code will need some conversion work to use the newer zlib. A few callers will be made simplier by using some newer APIs available in the updated zlib. There are some zombie programs that have been broken and I would like to delete.
This will clean up zombie programs and duplicated zlib code. This will also make future zlib version updates easier.
These changes touch some very sensitive areas of the system, such as kernel loading, or are architecture specific like armv6/armv7, and also touch some legacy code like kgzip+kgzldr on i386. Testers and active users of these legacy zlib code are welcomed.
- armv elf_trampoline Arm up to v5 can boot from gzipped kernel. This code is modified to use newer API for simplicity. Please verify gzipped kernel still boots with new code (Current code has fall back to legacy zlib in case of failure). Please also elaborate how to link such kernel, too. I'm still trying to figure that out.
- netgraph compression/decompression Please help testing and/or teach how to test. Netgraph compiles in the FreeBSD zlib version inside.
- gzipped a.out Does anyone use gzipped a.out executables, still? If so, does someone have an easy and safe program to run? Is a.out format i386 only?
- zfs boot Can we boot from gzipped file system today?
- CTF Checking how I can test.
LLVM's lld as the FreeBSD system linker
| Links | |
| LLD on the FreeBSD Wiki | URL: https://wiki.freebsd.org/LLD |
| lld exp-run | URL: https://bugs.freebsd.org/214864 |
Contact: Ed Maste <[email protected]>
In FreeBSD-HEAD and 12.0 the default FreeBSD system linker (i.e., /usr/bin/ld) is LLVM's lld, on amd64, arm64, and armv7. For i386 in 12.0 lld is used as the bootstrap linker (i.e., to build the kernel and base system) but it is not enabled as the system linker because of multiple issues building FreeBSD ports with it enabled.
The primary issue affecting i386 with lld is that many ports build position-dependent code (i.e., non-PIC) for use in shared libraries. This either comes from omitting the -fPIC compiler flag, or using hand-written position-dependent assembly. Compared with other CPU architectures i386 position-independent code is rather inefficient, which may be responsible for port authors making an explicit decision to avoid PIC.
By default lld does not allow position-dependent code in shared objects (in particular, it does not permit relocations against read-only segments - typically containing the`.text` section).
Over the last quarter many commits were made to the ports tree to fix the build when the system linker is lld - either building PIC code, or adding the -znotext linker flag to permit relocations against read-only segments, or just switching the port to link with GNU ld if it is incompatible with lld in some other way.
At this point there are only a few dozen open bug reports for issues linking ports with lld as the system linker, and I expect FreeBSD 12.1 to use lld as the system linker on i386 as well.
Tasks:
- Fix freepascal/Lazarus ports with lld
- Triage and address remaining port failures
- Holistic review of lld workarounds in the ports tree, to identify changes that are no longer needed, should be addressed in lld, or should be sent upstream
This project was sponsored by The FreeBSD Foundation.
mlx5 Drivers Update
| Links | |
| Mellanox OFED for FreeBSD Documentation | URL: http://www.mellanox.com/page/products_dyn?product_family=193&mtag=freebsd_driver |
Contact: Slava Shwartsman, Hans Petter Selasky, Konstantin Belousov <[email protected]>
The mlx5 driver provides support for PCI Express adapters based on ConnectX-4(LX), ConnectX-5(EX) and ConnectX-6(DX). The mlx5en driver provides support for Ethernet and the mlx5ib driver provides support for InfiniBand and RDMA over Converged Ethernet, RoCE.
Following updates done in mlx5 drivers:
- Added support for ConnectX-6 and ConnectX-6dx devices, which support of up to 200Gb/s interface speeds!
- Added TLS hardware offload support for ConnectX-6dx devices. TLS Tx crypto offload is a new feature for network devices. It enables the kernel TLS socket to skip encryption and authentication operations on the transmit side of the data path, delegating those to the NIC. In turn, the network adapter encrypts packets that belong to an offloaded TLS socket on the fly. The Mellanox network adapter does not modify any packet headers. It expects to receive fully framed TCP packets with TLS records as payload. The NIC replaces plaintext with ciphertext and fills the authentication tag. The adapter does not hold any state beyond the context needed to encrypt the next expected packet, i.e. expected TCP sequence number and crypto state.
- Add support for Dynamic Receive Queue Interrupt Moderation. Dynamic Interrupt Moderation (DIM) refers to any action made by hardware and/or software on run time to control interrupt rate on the system. The moderation action itself should not interfere with the system's operation and should not require any human interaction. In networking, dynamic interrupt moderation is used for controlling the rate of interrupts generated by the hardware for multiple traffic scenarios.
- Enhanced support for self-healing mechanism: In a rare occasion when Mellanox network adapters fail, due to a firmware bug for example, the driver will sense the catastrophic error. As a result of this failure detection, the device driver can trigger a firmware reset for the device so it can recover - without the need to reboot the entire host.
- Added support for in-driver firmware updating using mlx5tool.
This project was sponsored by Mellanox Technologies.
PCI Express Resets
Contact: Konstantin Belousov <[email protected]>
Sometimes the need to reset a device attached to the system presents itself. Preferrably this device reset can be accomplished without causing the whole machine to reboot. It is easy to do with USB devices if the physical access is available -- you can just re-plug the device. For in-chassis devices, built-in, or on add-on cards, it is not possible to reset the device with physical action, unless the device is hot-plugged. Nonetheless, for typical modern PCIe devices, and most built-in PCI-emulation devices, the reset can be initiated using software actions.
If device is a real plugged-in PCIe device, then reset can be initiated by disabling and then re-training PCIe-link by the upstream port controls. For most PCI devices, which support the PCI power management specification, the proven way to accomplish the reset is to put the device into state D3 (off) and then return to the previous power state.
FreeBSD was missing a way to conveniently request user- or driver-initiated reset of devices. While it was possible to manually fiddle with registers using pciconf, this is impractical for users, and requires a lot of boilerplate code from drivers.
A new BUS_RESET_CHILD() method was added to the newbus bus interface, and implementations added for PCIe bridges and PCI devices. The libdevctl(3) library call and devctl(8) command provide convenient userspace accessors for applications and administrators.
During the reset, the device driver must stop its operations with the device. One way to achieve this is to detach drivers before reset, and re-attach after the device afterwards. This is mostly fine for network interfaces, but other devices require more coordination to handle properly. For example, an NVMe disk device being detached it means that all mounted volumes abruptly disapper from VFS view. Due to this, the BUS_RESET_CHILD() method allows the caller to select either detach/re-attach or suspend/resume driver actions around the reset.
Mellanox uses the infrastructure to perform reset of the mlx(5) card after firmware reset without server reboot. It is believed that 'devctl reset' will be more widely useful.
This project was sponsored by Mellanox Technologies.
Security-Related changes
Contact: Konstantin Belousov <[email protected]>
ASLR
The ASLR (Address Space Layout Randomization) patch from review D5603 was committed into svn. While debate continues about the current and forward-looking value ASLR provides, having an implementation in the FreeBSD source tree makes it easily available to those who wish to use it. This also moves the conversation past the relative merits to more comprehensive security controls.
KPTI per-process control
The KPTI (Kernel Page Table Isolation) implementation was structured so that most selections of page isolation mode were local to the current address space. In other words, the global control variable pti was almost unused in the code paths, instead the user/kernel %cr3 values were directly loaded into registers or compared to see if the user page table was trimmed. Some missed bits of code were provided by Isilon, and then bugs were fixed and last places of direct use of pti were removed.
Now when the system starts in the pti-enabled mode, proccontrol(1) can be used by root to selectively disable KPTI mode for children of a process. The motivation is that if you trust the program that you run, you can get the speed of non-pti syscalls back, but still run your normal user session in PTI mode. E.g., firefox would be properly isolated.
Feature-control bits
Every FreeBSD executable now contains a bit mask intended for enabling/disabling security-related features which makes sense for the binary. This mask is part of the executable segments loaded on image activation, and thus is part of any reasonable way to authenticate the binary content.
For instance, the ASLR compatibility is de-facto the property of the image and not of the process executing the image. The first (zero) bit in the mask controls ASLR opt-out. Other OSes (e.g. Solaris) used an OS-specific dynamic flag, which has the same runtime properties but leaves less bits to consume in the feature-control mask.
The feature-control mask is read both by kernel and by rtld during image activation. It is expected that more features will be added to FreeBSD and the mask can be used for enabling/disabling those features..
It is expected that a tool to manipulate the mask will be provided shortly, see review D19290.
This project was sponsored by The FreeBSD Foundation.
Architectures
Updating platform-specific features and bringing in support for new hardware platforms.
FreeBSD/RISC-V Update
Contact: Ruslan Bukin <[email protected]>
Contact: Mitchell Horne <[email protected]>
Contact: Mark Johnston <[email protected]>
Work has continued on RISC-V port in the past quarter.
Support for transparent superpage promotion was added to the RISC-V port, meaning that applications will now automatically use large page mappings when possible. Per-CPU pmap activation tracking was added, reducing the overhead of various pmap operations. This noticeably improves the responsiveness of FreeBSD when running in a multi-CPU virtual machine.
A RISC-V implementation of minidumps was completed. Support for debugging RISC-V kernel dumps will land in devel/gdb after the next GDB release.
It is now possible to compile the in-tree LLVM's RISC-V target by setting WITH_LLVM_TARGET_RISCV=YES in /etc/src.conf. The use of LLVM to compile the RISC-V port is currently experimental and further investigation is ongoing.
Work is ongoing to bring up FreeBSD on SiFive's HiFive Unleashed development board now that one has been obtained by a FreeBSD developer. We also expect to work on support for a new version of the SBI specification.
This project was sponsored by The FreeBSD Foundation, DARPA, AFRL.
Ports
Changes affecting the Ports Collection, whether sweeping changes that touch most of the tree, or individual ports themselves.
FreeBSD GNOME status report
| Links | |
| GNOME FreeBSD | URL: https://freebsd.org/gnome/ |
| GNOME development Repo | URL: https://github.com/freebsd/freebsd-ports-gnome |
Contact: Koop Mast <[email protected]>
Contact: Eric Turgeon <[email protected]>
Ports activity in this quarter were:
- The x11-toolkits/gtk30 port updated to 3.24.5 and later to 3.24.7.
- The www/webkit2-gtk3 port was updated to 2.24.0.
- And the old insecure webkit-gtk2 and webkit-gtk3 where finally removed.
Work in progress, the branches are available in the GNOME development repo, see the link above.
- Eric Turgeon is working on MATE 1.22 in the mate-1.22 branch. And is almost complete.
- Charlie Li (IRC: vishwin) is working on a long overdue update of the cinnamon desktop. This update is almost complete. The only real blocker is that the screensaver can't be unlocked after it activates. The work is in the cinnamon branch.
- Koop Mast works on GNOME 3.32. The desktop is usable apart from gdm which is currently non-functional. Due to lack of free time the work is going slowly. This work is available in the gnome-3.32 branch.
People who are willing to contribute can find us on #freebsd-gnome on freenode.
FreeBSD KDE status report
| Links | |
| KDE FreeBSD | URL: https://freebsd.kde.org/ |
Contact: Adriaan de Groot <[email protected]>
Contact: Tobias C. Berner <[email protected]>
The two biggest accomplishements this quarter were:
- Qt4 and all its consumers have been removed from the ports tree.
- www/qt5-webengine has been updated from the ancient 5.9.4 to 5.12.x by kai@
Further we have kept the KDE Frameworks, Plasma and Applications ports up to date with upstreams releases, which thanks to upstreams' FreeBSD-CI uses less and less patches.
All the kde@ maintained ports (including cmake) have been kept up to date with their releases.
The plans for the next quarter are in no particular order
- Cleanup PyQt ports and pyqt.mk
- Improve qt.mk components
- Update sddm to 0.18.x
- Implement user management functionality in system settings (write non-logind backend)
People who are willing to contribute can find us on #kde-freebsd on freenode, and the [email protected] mailing list. Further we accept pull-requests and contributions on github.com/freebsd/freebsd-ports-kde.
Third-Party Projects
Many projects build upon FreeBSD or incorporate components of FreeBSD into their project. As these projects may be of interest to the broader FreeBSD community, we sometimes include brief updates submitted by these projects in our quarterly report. The FreeBSD project makes no representation as to the accuracy or veracity of any claims in these submissions.
FreeBSD Wiki Apple Intel Mac mini update
| Links | |
| FreeBSD Wiki | URL: https://wiki.freebsd.org/IntelMacMini |
Contact: Trevor Roydhouse <[email protected]>
The FreeBSD Wiki page for the Apple Intel Mac minis has been comprehensively updated over the last quarter to drag it from 2009 into 2019.
There are now detailed instructions for installing FreeBSD as the only operating system on models from 2007 through 2014 and itemised model specific information detailing FreeBSD support.
If anyone is interested, help is needed to provide more specific information for the macmini 1,1 and 6,1 through 8,1 models and to test patches for the asmc(4) driver for temperature sensor feedback and for setting fan speed. If you would like to help and have access to these Mac minis, please contact me.
Future tasks:
- Create and test more patches for asmc(4) to cover all Intel Mac minis
- Provide more information for 2006, 2012, 2014 and 2018 Mac minis
- Instructions for dual boot (macOS/FreeBSD) installations
Fuzzing FreeBSD with syzkaller
| Links | |
| syzkaller | URL: https://github.com/google/syzkaller |
Contact: Mark Johnston <[email protected]>
Contact: Andrew Turner <[email protected]>
Contact: Michael Tuexen <[email protected]>
Contact: Ed Maste <[email protected]>
Syzkaller is a coverage-guided system call fuzzer. It was originally developed for Linux. It programmatically creates programs consisting of sequences of random system calls and executes them in a VM (virtual machine). Using feedback from a kernel code coverage facility called kcov, syskaller mutates the generated test programs in an attempt to expand the executed coverage of code paths within the kernel. Sometimes exercising a seldom or infrequently used code path will crash the kernel. When syzkaller manages to crash the running kernel in the VM, it attempts to generate a minimal test case which reproduces the crash, simplifying debugging. Syzkaller is very effective at finding kernel bugs and has uncovered hundreds of issues in Linux. Over the past couple of years, syzkaller's author, Dmitry Vyukov, has added support for other operating systems, including FreeBSD.
Recently, a number of FreeBSD developers have been using syzkaller to find and fix bugs in the FreeBSD kernel. If interested, one can search the commit logs for "syzkaller" to find examples. Syzkaller can be run on a FreeBSD or Linux host to fuzz FreeBSD running in QEMU instances. It can also fuzz FreeBSD instances running on GCE (Google Compute Engine). Additionally, Google maintains a dedicated cluster of GCE hosts to continuously fuzz the latest builds of several different OS kernels. A FreeBSD target was recently added. Subscribe to the syzkaller-freebsd-bugs Google Group to receive notifications for newly discovered bugs.
Work is ongoing to improve syzkaller's coverage of FreeBSD's system calls. In particular, syzkaller needs to be taught about all of the target kernel's entry points and argument types in order to be useful. Many of the standard POSIX system calls are already covered, but most FreeBSD-specific system calls are not. Similarly, many ioctl(2) definitions are missing.
Some in-progress work aims to add support for bhyve as a VM backend for syzkaller, making it easier to fuzz FreeBSD VMs hosted on FreeBSD. Currently that can be done using QEMU, but QEMU on FreeBSD lacks support for hardware acceleration. See the PR for the implementation.
Finally, a number of bugs identified by syzkaller have yet to be fixed. If you are interested in helping out with any of the above, please mail the contacts listed above.
This project was sponsored by The FreeBSD Foundation.
sysctlmibinfo API 1.0
| Links | |
| gitlab.com/alfix/sysctlmibinfo | URL: https://gitlab.com/alfix/sysctlmibinfo |
Contact: Alfonso Sabato Siciliano <[email protected]>
Port: devel/libsysctlmibinfo
The sysctl() system call can get or set the value of a 'property' of the system. A 'property' has others info (description, type, label, etc.), they are necessary to build an utility like /sbin/sysctl, example:
kern.ostype: Operating system type
% sysctl -t kern.ostype
kern.ostype: string
Primarily sysctlmibinfo wraps the undocumented kernel interface and provides an easy C API: sysctlmif_name(), sysctlmif_description(), sysctlmif_info(), sysctlmif_label(), sysctlmif_nextnode() and sysctlmif_nextleaf(), to retrieve the info of a 'property'.
Moreover sysctlmibinfo provides a high level API: defines a struct sysctlmif_object and has some function: sysctlmif_filterlist(), sysctlmif_grouplist() and sysctlmif_tree(), to build lists and trees of objects.
You can use this library to quickly build a custom sysctl utility. For example, the core of deskutils/sysctlview (a graphical explorer for the sysctl MIB Tree) is just a call to sysctlmif_tree() and a visit to the resulting tree to show its sysctlmif_object nodes.
Note, actually a 'property' is an OID of the sysctl MIB, it is implemented by a struct sysctl_oid defined in sys/sysctl.h.
sysctlview 1.0
| Links | |
| gitlab.com/alfix/sysctlview | URL: https://www.gitlab.com/alfix/sysctlview |
Contact: Alfonso Sabato Siciliano <[email protected]>
Port: deskutils/sysctlview
The FreeBSD's kernel maintains a Management Information Base where the objects are properties to tuning the system using the sysctl() syscall and the /sbin/sysctl utility. The sysctlview utility is a "graphical sysctl MIB explorer", it depends on gtkmm (to build a GUI) and sysctlmibinfo (to retrieve the info from the kernel).
The version 1.0 provides two "TreeView":
- "Main" to show 'name', 'description', 'type', 'format' and 'value'
- "Flags" to show 'name' and a column for each 'flag' defined in sys/sysctl.h
The rows are "clickable" to display others info (e.g., 'label'). Currently sysctlview can show numeric and string values, the support for some opaque value will be added in the future.
University of Waterloo Co-operative Education Students
Contact: Ed Maste <[email protected]>
For the January-April 2019 term the FreeBSD Foundation has again brought on two co-operative education (co-op) students from the University of Waterloo.
Gerald Aryeetey is a 2nd year Computer Engineering
student. Gerald
started looking at a FreeBSD tool chain issue - our static
library
archiver (
Gerald later looked at a number of freebsd-update issues in FreeBSD's bugzilla database, and submitted many fixes. Around a dozen have been committed to FreeBSD, and more are in review.
Gerald also worked on the FreeBSD Foundation's hardware continuous integration effort. The prototype installation is building FreeBSD on a commit-by-commit basis and testing on a BeagleBone Black and a Pine64 LTS. The prototype will be converted to a permanent, public installation in the near future, after which additional test devices will be added.
For his final project Gerald intends to write a device driver for the Microchip LAN743x PCIe NIC.
Bora �zarslan is a 3rd year student in Computing and Financial Management. Bora's initial focus was also on tool chain issues in FreeBSD, starting with improvements or bug fixes in FreeBSD's readelf (from the ELF Tool Chain project).
Bora developed a tool to modify feature control bits in ELF binaries - for example, allowing binaries incompatible with ASLR to request to opt-out. As part of his readelf work Bora also added support to report the status of the feature control bits.
Bora continued investigating security topics, looking at applying Capsicum sandboxing to Kristaps' BSD licensed rsync implementation, openrsync. This work required first implementing fileargs_lstat support in cap_fileargs (which as now been committed) as well as changes to the fts directory hierarchy routines (which have not yet been committed to FreeBSD).
For the rest of the work term Bora will investigate and test unmodified Linux Docker containers on FreeBSD, to evaluate the state of Linuxulator support.
This project was sponsored by The FreeBSD Foundation.
News Home | Status Home