Introduction
Here is the last quarterly status report for 2019. As you might remember from last report, we changed our timeline: now we collect reports the last month of each quarter and we edit and publish the full document the next month. Thus, we cover here the period October 2019 - December 2019.
If you thought that the FreeBSD community was less active in the Christmas' quarter you will be glad to be proven wrong: a quick glance at the summary will be sufficient to see that much work has been done in the last months.
Have a nice read!
-- Lorenzo Salvadore
FreeBSD Team Reports
- FreeBSD Core Team
- FreeBSD Foundation
- FreeBSD Release Engineering Team
- Cluster Administration Team
- Continuous Integration
Projects
- IPSec Extended Sequence Number (ESN) support
- NFS Version 4.2 implementation
- DTS Update
- RockChip Support
- Creating virtual FreeBSD appliances from RE VMDK images
Kernel
- SoC audio framework and RK3399 audio drivers
- FreeBSD on Microsoft HyperV and Azure
- FreeBSD on EC2 ARM64
- ENA FreeBSD Driver Update
Architectures
Userland Programs
Ports
- Ports Collection
- KDE on FreeBSD
- Java on FreeBSD
- Electron and VSCode
- Bastille
- Universal Packaging Tool (upt)
- Wine on FreeBSD
Third-Party Projects
FreeBSD Team Reports
Entries from the various official and semi-official teams, as found in the Administration Page.
FreeBSD Core Team
Contact: FreeBSD Core Team <[email protected]>
The FreeBSD Core Team is the governing body of FreeBSD.
- Julie Saravanos, the sister of Bruce D. Evans (bde),
mailed core with the sad
news that Bruce passed away on 2019-12-18 at the age of 68
years. Bruce was a
deeply respected member of the community, served on the
Core team, and made
over 5,000 commits. Bruce's impact on our culture was so
profound that new
terminology was spawned. This is an excerpt of a message
from Poul-Henning
Kamp to Julie.
I don't know precisely when I first communicated with Bruce, it was in the late 1980'ies via "UseNET", but I can say with certainty that few people have inspired me more, or improved my programming more, than Bruce he did over the next half of my life.
All large projects invent its own vocabulary and in FreeBSD two of the neologisms are "Brucification", and "Brucified".
A "brucification" meant receiving a short, courteous note pointing out a sometimes subtle deficiency, or an overlooked detail in a source code change. Not necessarily a serious problem, possibly not even a problem to anybody at all, but nonetheless something which was wrong and ought to be fixed. It was not uncommon for the critique to be considerably longer than the change in question.
If one ignored brucifications one ran the risk of being "brucified", which meant receiving a long and painstakingly detailed list of every single one of the errors, mistakes, typos, shortcomings, bad decisions, questionable choices, style transgressions and general sloppiness of thinking, often expressed with deadpan humor sharpened to a near-fatal point.
The most frustrating thing was that Bruce would be perfectly justified and correct. I can only recall one or two cases where I were able to respond "Sorry Bruce, but you're wrong there..." - and I confess that on those rare days I felt like I should cut a notch in my keyboard.
The last email we received from Bruce is a good example of the depth of knowledge and insight he provided for the project:
https://docs.freebsd.org/cgi/getmsg.cgi?fetch=1163414+0+archive/2019/svn-src-all/20191027.svn-src-all
- The 12.1 release was dedicated to another FreeBSD
developer who passed away in
the fourth quarter of 2019, Kurt Lidl. The FreeBSD
Foundation has a memorial
page to Kurt.
https://www.freebsdfoundation.org/blog/in-memory-of-kurt-lidl/
We send our condolences to both the families of Bruce and Kurt.
- Core has documented The Project's policy on support tiers.
https://www.freebsd.org/doc/en_US.ISO8859-1/articles/committers-guide/archs.html
- Core approved a source commit bit for James Clarke. Brooks Davis (brooks) will mentor James and John Baldwin (jhb) will co-mentor.
- The Project's first Season of Docs ended with a negative result. The work was not completed and contact could not be established with the writer. No payment was made and the financing was set aside for future work.
- Google Summer of Code completed. Information about the
seven accepted
projects can be found on the wiki page.
https://wiki.freebsd.org/SummerOfCode2019Projects
- Adam Weinberger (adamw) was added to conduct@. Adam has demonstrated competence, understanding, and fairness in personal matters.
- Li-Wen Hsu (lwhsu) contacted Core after receiving a report
from concerned
local community members about past updates to The
Project's
internationalization policy. Lengthy discussions took
place to determine how
to reaffirm that The Project maintains a neutral position
in political
disputes. Updates were made to the document and it was
decided that any
future changes would require explicit Core approval.
https://www.freebsd.org/internal/i18n.html
- After nomination by Edward Napierała (trasz), core voted to grant Daniel Ebdrup (debdrup) and Lorenzo Salvadore (salvadore) membership in The Project. Both Daniel and Lorenzo have been working on the quarterly reports for the past few quarters.
- The Core-initiated Git Transition Working Group continued to meet over the last quarter of 2019. Their report is still forthcoming.
FreeBSD Foundation
Contact: Deb Goodkin <[email protected]>
The FreeBSD Foundation is a 501(c)(3) non-profit organization dedicated to supporting and promoting the FreeBSD Project and community worldwide. Funding comes from individual and corporate donations and is used to fund and manage software development projects, conferences and developer summits, and provide travel grants to FreeBSD contributors. The Foundation purchases and supports hardware to improve and maintain FreeBSD infrastructure and provides resources to improve security, quality assurance, and release engineering efforts; publishes marketing material to promote, educate, and advocate for the FreeBSD Project; facilitates collaboration between commercial vendors and FreeBSD developers; and finally, represents the FreeBSD Project in executing contracts, license agreements, and other legal arrangements that require a recognized legal entity.
Here are some highlights of what we did to help FreeBSD last quarter:
Partnerships and Commercial User Support
We help facilitate collaboration between commercial users and FreeBSD developers. We also meet with companies to discuss their needs and bring that information back to the Project. In Q4, Ed Maste and Deb Goodkin met with a few commercial users in the US. It's not only beneficial for the above, but it also helps us understand some of the applications where FreeBSD is used. We were also able to meet with a good number of commercial users at the Bay Area Vendor/Developer Summit and Open Source Summit Europe. These venues provide an excellent opportunity to meet with commercial and individual users and contributors to FreeBSD.
Fundraising Efforts
In 2019, we focused on supporting a few key areas where the Project needed the most help. The first area was software development. Whether it was contracting FreeBSD developers to work on projects like wifi support, to providing internal staff to quickly implement hardware workarounds, we've stepped in to help keep FreeBSD innovative, secure, and reliable. Software development includes supporting the tools and infrastructure that make the development process go smoothly, and we're on it with team members heading up the Continuous Integration efforts, and actively involved in the clusteradmin and security teams.
Our advocacy efforts focused on recruiting new users and contributors to the Project. We attended and participated in 38 conferences and events in 21 countries. From giving FreeBSD presentations and workshops to staffing tables, we were able to have 1:1 conversations with thousands of attendees.
Our travels also provided opportunities to talk directly with FreeBSD commercial and individual users, contributors, and future FreeBSD users/contributors. We've seen an increase in use and interest in FreeBSD from all of these organizations and individuals. These meetings give us a chance to learn more about what organizations need and what they and other individuals are working on. The information helps inform the work we should fund.
In 2019, your donations helped us continue our efforts of supporting critical areas of FreeBSD such as:
- Operating System Improvements: Providing staff to immediately respond to urgent problems and implement new features and functionality allowing for the innovation and stability you've come to rely on.
- Improving and increasing test coverage, continuous integration, and automated testing with a full-time software engineer to ensure you receive the highest quality, secure, and reliable operating system.
- Security: Providing engineering resources to bolster the capacity and responsiveness of the Security team providing you with peace of mind when security issues arise.
- Growing the number of FreeBSD contributors and users from our global FreeBSD outreach and advocacy efforts, including expanding into regions such as China, India, Africa, and Singapore.
- Offering FreeBSD workshops and presentations at more conferences, meetups, and universities around the world.
- Providing opportunities such as developer and vendor summits and company visits to help facilitate collaboration between commercial users and FreeBSD developers, as well as helping to get changes pushed into the FreeBSD source tree, and creating a bigger and healthier ecosystem.
We've accomplished a lot this year, but we are still only a small 501(c)3 organization focused on supporting FreeBSD and not a trade organization like many other open source Foundations.
Please consider making a donation at https://www.FreeBSDfoundation.org/donate/ to help us continue and increase our support for FreeBSD.
We also have the Partnership Program, to provide more benefits for our larger commercial donors. Find out more information at https://www.FreeBSDfoundation.org/FreeBSD-foundation-partnership-program/ and share with your companies!
OS Improvements
The Foundation supports software development projects to improve FreeBSD through our full time technical staff, contractors, and project grant recipients. They maintain and improve critical kernel subsystems, add new features and functionality, and fix bugs.
Between October and December there were 236 commits to the FreeBSD source repository tagged with FreeBSD Foundation sponsorship. This is about 10% of all commits during this period. Some of these projects have their own entries in the quarterly report, and are not repeated here, while others are briefly described below.
As usual, Foundation staff member Konstantin Belousov committed a large number of UFS, NFS, tmpfs, VM system, and low-level Intel x86 bug fixes and improvements. Kostik also committed improvements to the run-time linker (rtld), and participated in very many code reviews, helping to get changes from other developers integrated into the tree.
Following on from his work to improve debugging tools in the Linuxulator environment, Edward Napierała integrated the Linux Test Project (LTP) with FreeBSD's CI system, and committed a number of small bug fixes to the Linuxulator itself.
Mark Johnston continued working on infrastructure for the Syzkaller system call fuzzing tool, and committed fixes for many issues identified by it. Mark committed improvements to RISC-V infrastructure, the network stack, performance and locking, and x86 pmap.
Mark also added support for newer Intel WiFi chipsets to the iwm driver, enabling WiFi support for the Lenovo X1 Carbon 7th generation, and other contemporary laptops.
Ed Maste committed a number of improvements and cleanups in build infrastructure, vt console fixes including issues with keyboard maps, some blacklistd updates, documentation updates, and other small changes. Ed also committed some work to prepare for the removal of GCC 4.2.1 from the FreeBSD source tree, currently planned for Q1 2020.
Continuous Integration and Quality Assurance
The Foundation provides a full-time staff member who is working on improving our automated testing, continuous integration, and overall quality assurance efforts.
During the fourth quarter of 2019, Foundation staff continued to improve the project's CI infrastructure, worked with contributors to fix the failing build and test cases. We worked with other teams in the project for their testing needs and also worked with many external projects and companies to improve their support of FreeBSD. We added several new CI jobs and brought the FreeBSD Hardware Testing Lab online.
We published 2019 in Review: CI and Testing Advancements on the Foundation's blog.
See the FreeBSD CI section of this report for completed work items and detailed information.
Supporting FreeBSD Infrastructure
The Foundation provides hardware and support to improve the FreeBSD infrastructure. Last quarter, we continued supporting FreeBSD hardware located around the world.
FreeBSD Advocacy and Education
A large part of our efforts are dedicated to advocating for the Project. This includes promoting work being done by others with FreeBSD; producing advocacy literature to teach people about FreeBSD and help make the path to starting using FreeBSD or contributing to the Project easier; and attending and helping other FreeBSD contributors volunteer to run FreeBSD events, staff FreeBSD tables, and give FreeBSD presentations.
The FreeBSD Foundation sponsors many conferences, events, and summits around the globe. These events can be BSD-related, open source, or technology events geared towards underrepresented groups. We support the FreeBSD-focused events to help provide a venue for sharing knowledge, to work together on projects, and to facilitate collaboration between developers and commercial users. This all helps provide a healthy ecosystem. We support the non-FreeBSD events to promote and raise awareness of FreeBSD, to increase the use of FreeBSD in different applications, and to recruit more contributors to the Project.
Check out some of the advocacy and education work we did last quarter:
- Organized the 2019 Bay Area FreeBSD Vendor and Developers Summit in Santa Clara, CA
- Presented at COSCON '19 in Shanghai, China
- Represented FreeBSD at All Things Open 2019, in Raleigh, North Carolina
- Industry Partner Sponsor for LISA '19 in Portland, OR
- Silver Sponsor of OpenZFS in San Francisco, CA
- Gave a technical presentation at School of Mines in Golden, CO
- Presenting and representing FreeBSD at Seagl, in Seattle, WA
- Presented at Open Source Summit Europe in Lyon France
- Committed to sponsoring LinuxConfAu 2020, in Gold Coast, Australia in addition to holding a FreeBSD Mini-Conf
- Accepted to present at the BSD Dev Room at FOSDEM '20, in Brussels, Belgium
- Accepted to have a stand at FOSDEM '20, in Brussels, Belgium
- Committed to sponsoring FOSSASIA 2020, in Singapore
- Committed to hold FreeBSD Day at SCALE 18x, in Pasadena, CA
We continued producing FreeBSD advocacy material to help people promote FreeBSD. Learn more about our efforts in 2019 to advocate for FreeBSD: https://www.freebsdfoundation.org/blog/2019-in-review-advocacy/
Our Faces of FreeBSD series is back. Check out the latest post: Mahdi Mokhtari. https://www.freebsdfoundation.org/blog/faces-of-freebsd-2019-mahdi-mokhtari/
Read more about our conference adventures in the conference recaps and trip reports in our monthly newsletters: https://www.freebsdfoundation.org/news-and-events/newsletter/
We help educate the world about FreeBSD by publishing the professionally produced FreeBSD Journal. As we mentioned previously, the FreeBSD Journal is now a free publication. Find out more and access the latest issues at https://www.FreeBSDfoundation.org/journal/.
You can find out more about events we attended and upcoming events at https://www.FreeBSDfoundation.org/news-and-events/.
We have continued our work with a new website developer to help us improve our website. Work has begun to make it easier for community members to find information more easily and to make the site more efficient.
Legal/FreeBSD IP
The Foundation owns the FreeBSD trademarks, and it is our responsibility to protect them. We also provide legal support for the core team to investigate questions that arise.
Go to http://www.FreeBSDfoundation.org to find out how we support FreeBSD and how we can help you!
FreeBSD Release Engineering Team
Contact: FreeBSD Release Engineering Team <[email protected]>
The FreeBSD Release Engineering Team is responsible for setting and publishing release schedules for official project releases of FreeBSD, announcing code freezes and maintaining the respective branches, among other things.
The FreeBSD Release Engineering Team continued work on the 12.1-RELEASE, which started September 6th. This release cycle was the first "freeze-less" release from the Subversion repository, and the test bed for eliminating the requirement of a hard code freeze on development branches.
The 12.1-RELEASE cycle concluded with the final build beginning November 4th, preceded by three BETA builds and two RC builds. The RC3 build had been included in the original schedule, but had been decided to not be required.
Additionally throughout the quarter, several development snapshots builds were released for the head, stable/12, and stable/11 branches.
Much of this work was sponsored by Rubicon Communications, LLC (netgate.com) and the FreeBSD Foundation.
Cluster Administration Team
| Links | |
| Cluster Administration Team members | URL: https://www.freebsd.org/administration.html#t-clusteradm |
Contact: Cluster Administration Team <[email protected]>
The FreeBSD Cluster Administration Team consists of the people responsible for administering the machines that the Project relies on for its distributed work and communications to be synchronised. In this quarter, the team has worked on the following:
- Upgrade ref11-{amd64,i386}.freebsd.org to 11.3-STABLE r353313
- Ongoing systems administration work:
- Creating accounts for new committers.
- Backups of critical infrastructure.
- Keeping up with security updates in 3rd party software.
Work in progress:
- Review the service jails and service administrators operation.
- South Africa Mirror (JINX) in progress.
- NVME issues on PowerPC64 Power9 blocking dual socket machine from being used as pkg builder.
- Drive upgrade test for pkg builders (SSDs) courtesy of the FreeBSD Foundation.
- Boot issues with Aarch64 reference machines.
- New NYI.net sponsored colocation space in Chicago-land area.
- Setup new host for CI staging environment.
- Plan how to add new semi-official pkg mirrors
Continuous Integration
Contact: Jenkins Admin <[email protected]>
Contact: Li-Wen Hsu <[email protected]>
The FreeBSD CI team maintains continuous integration system and related tasks for the FreeBSD project. The CI system regularly checks the committed changes can be successfully built, then performs various tests and analysis of the results. The results from build jobs are archived in an artifact server, for the further testing and debugging needs. The CI team members examine the failing builds and unstable tests, and work with the experts in that area to fix the code or adjust test infrastructure. The details are of these efforts are available in the weekly CI reports.
During the fourth quarter of 2019, we worked with the contributors and developers in the project for their testing needs and also worked with many external projects and companies to improve their support of FreeBSD. The FreeBSD Hardware Testing Lab is online in this quarter. It's still in work in progress stage and we are merging the different versions and will integrate more tightly to the main CI server. We are also working on make this work more easierly to be reproduced.
Work in progress:
- Collecting and sorting CI tasks and ideas at https://hackmd.io/bWCGgdDFTTK_FG0X7J1Vmg
- Setup the CI stage environment and put the experimental jobs on it
- Implementing automatic tests on bare metal hardware
- Adding drm ports building test against -CURRENT
- Testing and merging pull requests at https://github.com/freebsd/freebsd-ci/pulls
- Planning for running ztest and network stack tests
- Helping more 3rd software get CI on FreeBSD through a hosted CI solution
- Adding LTP test jobs.
- Adding non-x86 test jobs.
- Adding external toolchin related jobs.
Please see freebsd-testing@ related tickets for more WIP information.
This project was sponsored by The FreeBSD Foundation.
Projects
Projects that span multiple categories, from the kernel and userspace to the Ports Collection or external projects.
IPSec Extended Sequence Number (ESN) support
Contact: Patryk Duda <[email protected]>
Contact: Marcin Wojtas <[email protected]>
Extended Sequence Number (ESN) is IPSec extension defined in RFC4303 Section 2.2.1. It makes possible to implement high-speed IPSec implementations where standard, 32-bit sequence number is not sufficent. Key feature of the ESN is that only low order 32 bits of sequence number are transmitted over the wire. High-order 32 bits are maintained by sender and receiver. Additionally high-order bits are included in the computation of Integrity Check Value (ICV) field.
Extended Sequence Number support contains following:
- Modification of existing anti-replay algorithm to fulfil ESN requirements
- Trigger soft lifetime expiration at 80% of UINT32_MAX when ESN is disabled
- Implement support for including ESN into ICV in cryptosoft engine in both encrypt and authenticate mode (eg. AES-CBC and SHA256 HMAC) and combined mode (eg. AES-GCM)
- Implement support for including ESN into ICV in AES-NI engine in both encrypt and authenticate mode and combined mode
Remaining work:
- Upstream patches of the anti-replay algorithm
- Adjust implementation of crypto part after the reworked Open Crypto Framework gets stable
This project was sponsored by Stormshield.
NFS Version 4.2 implementation
Contact: Rick Macklem <[email protected]>
RFC-7862 describes a new minor revision to the NFS Version 4 protocol. This project implements this new minor revision.
The NFS Version 4 Minorversion 2 protocol adds several optional features to NFS, such as support for SEEK_DATA/SEEK_HOLE, file copying done on the server that avoids data transfer over the wire and support for posix_fallocate(), posix_fadvise(). Hopefully these features can improve performance for certain applications.
This project has basically been completed. The code changes have now all been committed to head/current and should be released in FreeBSD 13.
Testing by others would be appreciated. To do testing, an up to date head/current system is required. Client mounts need the "minorversion=2" mount option to enable this protocol. The NFS server will have NFSv4.2 enabled by default.
DTS Update
Contact: Emmanuel Vadot <[email protected]>
DTS files (Device Tree Sources) were updated to be on par with Linux 5.4 for HEAD and 5.2 for the 12-STABLE branch. The DTS for the RISC-V architecture are now imported as well.
RockChip Support
Contact: <[email protected]>
Contact: Emmanuel Vadot <[email protected]>
Contact: Michal Meloun <[email protected]>
RockChip RK3399 now has USB3 support, some configuration such as device mode are still not supported however host mode should work on any board.
Support for SPI has been committed which enables ability to interact with SPI flash if present.
All regulators for the RK808 PMIC (Power Management IC) have been added.
All clocks are now supported which completes clock and reset implementation, previously only clocks from devices with drivers were supported.
The TS-ADC (Temperature Sensor ADC) is now supported, this adds the ability to read temperature of the CPU and GPU via sysctl hw.temperature .
Initial PCIe support has been committed and verified working on several different boards. Known working devices are NVMe devices and PCIe cards that doesn't utilize PCIe switching or bridge functionality.
Card Detection for SDCard on RK3328 and RK3399 is now supported. There is still some problems if the board is using a GPIO for CD instead of the internal detection mechanism.
Creating virtual FreeBSD appliances from RE VMDK images
| Links | |
| freebsd-mkova | URL: https://github.com/gonzoua/freebsd-mkova |
Contact: Oleksandr Tymoshenko <[email protected]>
OVA is a file format for packaging and distributing virtual appliances: pre-configured virtual machine images. Virtual appliance file contains full VM information like the number of CPUs, amount of memory, list of virtual devices, it also includes disk images. Applications like VirtualBox or VMWare can import OVA files; this process can be easily automated.
freebsd-mkova is a CLI tool to create OVA files using VMDK images provided by FreeBSD RE. For now, only a limited set of attributes can be specified: VM name, number of CPU, amount of memory, and disk size. The tool also does only cursory sanity checks on the VMDK file format, assuming it's a monolithic sparse file and that it has to be converted to the stream-optimized format. The script can be extended to make hardware configuration more flexible and VMDK parser more robust.
Kernel
Updates to kernel subsystems/features, driver support, filesystems, and more.
SoC audio framework and RK3399 audio drivers
| Links | |
| rk3399_audio | URL: https://github.com/gonzoua/freebsd/tree/rk3399_audio |
Contact: Oleksandr Tymoshenko <[email protected]>
Most modern SoCs and devboards have audio support in one form or another, but it's one of the areas that are overlooked by FreeBSD driver developers. The most common architecture for the audio pipeline on a single-board computer consists of two DAIs (digital audio interfaces): CPU and codec, connected by a serial bus.
CPU DAI is a SoC IP block that operates with samples: obtains them from the driver for playback or provides them to the driver for recording through FIFOs or DMA requests. Audio samples leave (or arrive at) the SoC through a serial bus, usually I2S, that is connected to Codec DAI.
Codec DAI is an external (to the SoC) chip that packs one or more DAC/ADC blocks along with mixers, amplifiers, and probably more specialized devices like filters and/or sound effects. The analog part of the codec is connected to microphones/headphones/speakers. On SBCs, the codec usually communicates with SoC through two interfaces: data path, over which audio samples travel, and a control interface that is used to read/write chip registers and configure its behavior. The most common choices for these are I2S and I2C buses, respectively.
For FDT-enabled devices, an audio pipeline is described as a virtual DTB node that has links to the CPU and codec device(s), and which specifies the data format, and clock details that both the CPU and the codec chips would use. It also may have more than one CPU/codec pair.
Using Firefly-RK3399 as a test device, I was able to implement I2S driver for RK3399 SoC (PIO mode, playback only), the driver for Realtek's RT5640 chip (headphones playback only + mixer controls) and a base outline of SoC audio framework. Some bits of rk_i2s and the framework were ported from the NetBSD code developed by Jared McNeill. On my WIP branch, I can play mp3 audio and control playback volume.
The primary missing functionalities at the moment are recording support, multi-link audio cards, DMA support. The most critical among these is DMA support. In the current implementation, all buffer management is placed at the ausoc layer, which is not going to work for DMA, because only the CPU DAI driver would know about the memory constraints and access mechanisms. The current state of RK3399 support does not allow to implement DMA transfers for rk_i2s easily, but I plan to look into this right after adding recording support, which should not be a lot of work.
FreeBSD on Microsoft HyperV and Azure
| Links | |
| FreeBSD on MicrosoftAzure wiki | URL: https://wiki.freebsd.org/MicrosoftAzure |
| FreeBSD on Microsoft HyperV | URL: https://wiki.freebsd.org/HyperV |
Contact: FreeBSD Integration Services Team <[email protected]>
Contact: Wei Hu <[email protected]>
Contact: Li-Wen Hsu <[email protected]>
Wei is working on HyperV Socket support for FreeBSD. HyperV Socket provides a way for host and guest to communicate using common socket interfaces without networking support. Some features in Azure require HyperV Socket support in guest.
It is planned to commit the code by the end of February.
This project is sponsored by Microsoft. Details of HyperV Socket is available at https://docs.microsoft.com/en-us/virtualization/hyper-v-on-windows/user-guide/make-integration-service
Li-Wen and Wei are working on improving FreeBSD release on Azure. During this quarter, Wei has published the 11.3-RELEASE on Azure. Li-Wen is working on the FreeBSD release codes related to Azure for the -CURRENT and 12-STABLE branches.
This project is sponsored by Microsoft and FreeBSD Foundation.
FreeBSD on EC2 ARM64
Contact: Colin Percival <[email protected]>
In November 2018, Amazon Web Services announced the first Elastic Compute Cloud (EC2) instances built around the ARM64 platform. While FreeBSD supported the ARM64 platform, running on this specific virtual machines took some additional work, but by April 2019 the weekly snapshot builds performed by the Release Engineering Team included ARM64 AMIs for FreeBSD HEAD.
In November 2019 FreeBSD 12.1 was released, including the first "RELEASE" FreeBSD EC2/ARM64 AMIs. A few weeks later, FreeBSD/ARM64 was added as a new "product" to the AWS Marketplace.
At the re:Invent 2019 conference in December 2019, Amazon announced a second family of ARM64 instances, known variously as "Graviton 2" and "M6G". These are far more powerful than the first-generation ARM64 EC2 instances, and have a roughly 40% price/performance advantage over the "M5" family of x86 EC2 instances; and existing FreeBSD 12.1 and HEAD AMIs run "out of the box" on these instances.
Work is currently underway to improve kernel locking scalability on these instances; with high levels of parallelism (e.g. buildworld -j64) the G6M instances have approximately 1.5x higher sys:user ratios than equally-sized M5 instances, suggesting that there is room for improvement here.
Two issues have been recently identified, both likely relating to ACPI:
- EC2 "StopInstance" API calls, which translate to ACPI "power button" notifications, do not trigger FreeBSD to shut down; this results in a timeout from EC2 and a "hard poweroff".
- Hotplugging/unplugging EBS volumes, which normally operates via ACPI device notifications, does not work.
Help from developers familiar with ARM64 and ACPI would be much appreciated.
This project was sponsored by FreeBSD/EC2 Patreon.
ENA FreeBSD Driver Update
| Links | |
| ENA README | URL: https://github.com/amzn/amzn-drivers/blob/master/kernel/fbsd/ena/README |
Contact: Michal Krawczyk <[email protected]>
Contact: Maciej Bielski <[email protected]>
Contact: Marcin Wojtas <[email protected]>
ENA (Elastic Network Adapter) is the smart NIC available in the virtualized environment of Amazon Web Services (AWS). The ENA driver supports multiple transmit and receive queues and can handle up to 100 Gb/s of network traffic, depending on the instance type on which it is used.
Completed since the last update:
- Upstream of the driver v2.1.0 version, introducing:
- Netmap support
- Driver structure rework (split datapath code from initialization)
- Fix for keep-alive timeout due to prolonged reset
- Enable LLQ mode on arm64 instances by enabling memory mapped as WC
Work in progress::
- ENA v2.2.0 release, introducing new bug fixes, features and other improvements
This project was sponsored by Amazon.com Inc.
Architectures
Updating platform-specific features and bringing in support for new hardware platforms.
PowerPC on Clang
Contact: Justin Hibbits <[email protected]>
Contact: Brandon Bergren <[email protected]>
Contact: Alfredo Dal'Ava J�nior <[email protected]>
Shortly before the end of the year all 3 PowerPC targets (powerpc, powerpc64, powerpcspe) switched to Clang as the base compiler. This was an effort spanning nearly the full year, with several people involved. 32-bit PowerPC platforms (powerpc, powerpcspe) still require GNU ld, but powerpc64 uses LLD as the base linker. The other two platforms will migrate as soon as LLD is ready, which should be in the next several months.
With the switch to Clang and LLD, powerpc64 also switched to ELFv2, a modern ABI initially targeted for Linux powerpc64le (little endian), but the ABI itself is endian agnostic; however, ELFv2 is binary incompatible with ELFv1. FreeBSD is still big endian on all powerpc targets.
NXP ARM64 SoC support
Contact: Marcin Wojtas <[email protected]>
Contact: Artur Rojek <[email protected]>
The Semihalf team initiated working on FreeBSD support for the NXP LS1046A SoC
LS1046A are quad-core 64-bit ARMv8 Cortex-A72 processors with integrated packet processing acceleration and high speed peripherals including 10 Gb Ethernet, PCIe 3.0, SATA 3.0 and USB 3.0 for a wide range of networking, storage, security and industrial applications.
Completed since the last update:
- QSPI
- Network performance improvements
Todo:
- Upstreaming of developed features. This work is expected to be submitted/merged to HEAD in the Q1 of 2020.
This project was sponsored by Alstom Group.
Userland Programs
Changes affecting the base system and programs in it.
Linux compatibility layer update
Contact: Edward Tomasz Napierala <[email protected]>
Linux binaries of Linux Test Projects tests are now part of the FreeBSD Continuous Integration infrastructure. This makes it easy to track progress in improving the Linux compatibility layer, and to detect regressions.
There was a fair number of all kinds of improvements to the layer, ranging from updated linux(4) man page, to a new linux rc script, which now takes care of eg mounting Linux-specific filesystems or setting ELF fallback brand, to new syscalls, to tiny improvements such as making ^T work for Linux binaries.
From the user point of view, when running 13-CURRENT, Linux jails are now in a mostly working state: you can SSH into a jail with CentOS 8 binaries, run screen(1), Emacs, Postgres, OpenJDK 11, use yum upgrade... Of course there's still a bunch of things that need work:
- There is a patch from chuck@ that makes core dumps work for Linux binaries; this will make debugging much easier
- There are pending reviews for patches that add extended attributes support, fexecve(2) syscall, sendfile; they require wrapping up and committing
- There are over 400 failing LTP tests. Some of them are false positives, some are easy to fix bugs, some require adding new system calls. Any help is welcome.
This project was sponsored by FreeBSD Foundation.
Ports
Changes affecting the Ports Collection, whether sweeping changes that touch most of the tree, or individual ports themselves.
Ports Collection
Contact: Ren� Ladan <[email protected]>
Contact: FreeBSD Ports Management Team <[email protected]>
The Ports Management Team is responsible for overseeing the overall direction of the Ports Tree, building packages, and personnel matters. This entry shows what happened in the last quarter.
2019Q4 closed with a total of 38,200 ports and 2180 open PRs of which a small 470 PRs are unassigned. Last quarter saw 7907 commits from 157 committers to the HEAD branch and 358 commits from 61 committers to the 2019Q4 branch. This seems to suggest a small increase in activity compared to the quarter before.
During the last quarter, we welcomed Oleksii "Alex" Samorukov (samm@) and Scott Long (scottl@, already a source committer) as new ports committers. We also said goodbye to az@, brd@, dtekse@, eadler@, and johans@.
The default versions of some ports changed: Lazarus is now at version 2.0.6, Samba at 4.10, and Python at 3.7. The web browsers received their updates too: Chromium is now at version 78.0.3904.108, Firefox at version 72.0 and its ESR counterpart at version 68.4.0. Finally, the Qt stack got updated to version 5.13.2.
Some modernizations took place: the "palm" category was removed as well as the virtual "ipv6" category. IPv6 support (next to IPv4) is now considered the norm. Lastly, the CentOS 6 ports were removed after their CentOS 7 counterparts were made the default in the previous quarter.
As always, antoine@ was happy to take your exp-runs, this time a total of 30, for various ports and framework updates, default version updates, and the removal of OpenJDK 6 and OpenJRE 6.
KDE on FreeBSD
| Links | |
| KDE FreeBSD | URL: https://freebsd.kde.org/ |
| KDE Community FreeBSD | URL: https://community.kde.org/FreeBSD |
Contact: Adriaan de Groot <[email protected]>
The KDE on FreeBSD project packages the software produced by the KDE Community for FreeBSD. The software includes a full desktop environment, the art application https://kdenlive.org and hundreds of other applications that can be used on any FreeBSD desktop machine.
The monthly releases of KDE Frameworks, bugfix-releases of KDE Plasma Desktop and the quarterly feature release of KDE Plasma Desktop were all landed in the ports tree shortly after upstream releases. There were also monthly KDE Applications bugfix-releases which also landed in a timely manner.
Digikam landed a new release thanks to Dima Panov. We hope this gets rid of the instability caused by the previous release update from last quarter.
The open bugs list grew to 32 this quarter with a handful of strange build failures. We welcome detailed bug reports and patches. KDE packaging updates are prepared in a copy of the ports repository on GitHub and then merged in SVN. We welcome pull requests there as well.
Java on FreeBSD
| Links | |
| OpenJDK 11 repository at FreeBSD GitHub | URL: https://github.com/freebsd/openjdk-jdk11u |
Contact: Greg Lewis <[email protected]>
During Q4 the FreeBSD java porting effort features smaller updates than those of the previous quarters. However, the following changes are worth mentioning:
- Updated ports for OpenJDK 8u232, 11.0.5, and 13.0.1
- Removal of the EOL'ed Java 6, 9, and 10 ports
- Fixed remote debugging for Java 11+
- Fixed a problem with running external processes for Java 11+
This project was sponsored by FreeBSD Foundation.
Electron and VSCode
| Links | |
| Electron port | URL: https://github.com/tagattie/FreeBSD-Electron |
| VSCode port | URL: https://github.com/tagattie/FreeBSD-VSCode |
Contact: Hiroki Tagato <[email protected]>
Contact: Luca Pizzamiglio <[email protected]>
Electron is a popular framework to build desktop application using JavaScript, HTML and CSS. Few months ago, electronjs has been added to the ports tree. Currently version 4.x and 6.x are supported.
In the last quarter, a popular application, the powerful VSCode editor, has been added to the ports tree as well. VSCode is based on electron 6.x
atom, another popular editor, is still a work in progress and it's based on electron 4.x
Many thanks to Hiroki, for the hard work, and to Antoine, for support of the special poudriere configuration needed to build VSCode.
Bastille
| Links | |
| Bastille GitHub | URL: https://github.com/BastilleBSD/bastille |
| Bastille Templates | URL: https://gitlab.com/bastillebsd-templates |
| Bastille Website | URL: https://bastillebsd.org |
Contact: Christer Edwards <[email protected]>
What is Bastille?
Bastille is an open-source system for automating deployment and management of containerized applications on FreeBSD.
Bastille uses FreeBSD jails as a container platform and adds template automation to create a Docker-like collection of containerized software. The template collection currently validates 30-40 applications from the ports tree, and is growing!
Templates take care of installing, configuring, enabling, and starting the software, providing an automated way of building containerized stacks.
Bastille is available in ports at sysutils/bastille.
Q4 2019 Status
In Q4 2019 Bastille published three releases (for a total of ten releases in 2019). Highlights from these updates include:
- support for "thin" (shared base) and "thick" (unique base) jails
- support for INCLUDE and FSTAB in template system
- upgrade support for shared and unique base jails
- GitLab CI/CD testing for all official templates
- automatic template validation and CVE scan
- dedicated pf table for private IP jails
Bastille saw an increase in community contributions with six new GitHub contributors. These people generously improved error checking, release validation (sha256), firewall functionality, flexible networking and initial support for resource limits!
We want to thank everyone that contributed to Bastille in 2019. Your support has been amazing!
Universal Packaging Tool (upt)
| Links | |
| Upt repositories | URL: https://framagit.org/upt/ |
| Upt itself | URL: https://framagit.org/upt/upt/ |
| The FreeBSD backend | URL: https://framagit.org/upt/upt-freebsd |
Contact: The upt mailing list <[email protected]>
Contact: <#upt-packaging>
The Universal Package Manager (upt) is a tool designed to easily port software from common upstream package archives (such as https://rubygems.org/) to various operating systems, including FreeBSD, of course.
A lot of similar tools already exist: pytoport (which creates FreeBSD ports for PyPI packages), gem2deb (which creates Debian packages from a Ruby gem), and many others.
The main difference between these tools and upt is that the latter uses a modular design, allowing it to handle packages from many sources and support many different operating systems through plugins. You may try upt by installing sysutils/py-upt, sysutils/py-upt-pypi and sysutils/py-upt-freebsd. Suppose you would like to package "upt-cran", which is hosted on PyPI. You could do it like so:
# upt package -f pypi -b freebsd -o /usr/ports/sysutils/
upt-cran
$ tree /usr/ports/sysutils/py-upt-cran
/usr/ports/sysutils/py-upt-cran
|-- Makefile
|-- distinfo
`-- pkg-descr
$ cat sysutils/py-upt-cran/Makefile
# $FreeBSD$
PORTNAME= upt-cran
DISTVERSION= 0.1
CATEGORIES= sysutils python
MASTER_SITES= CHEESESHOP
PKGNAMEPREFIX= ${PYTHON_PKGNAMEPREFIX}
MAINTAINER= [email protected]
COMMENT= CRAN frontend for upt
LICENSE= BSD3CLAUSE
LICENSE_FILE= ${WRKSRC}/XXX
RUN_DEPENDS=
${PYTHON_PKGNAMEPREFIX}lxml>0:devel/py-lxml@${PY_FLAVOR}
\
${PYTHON_PKGNAMEPREFIX}requests>0:www/py-requests@${PY_FLAVOR}
\
${PYTHON_PKGNAMEPREFIX}upt>0:sysutils/py-upt@${PY_FLAVOR}
TEST_DEPENDS=
${PYTHON_PKGNAMEPREFIX}requests-mock>0:www/py-requests-mock@${PY_FLAVOR}
USES= python
USE_PYTHON= autoplist distutils
.include <bsd.port.mk>
Note that the Rubygems and CPAN frontends are also available (sysutils/py-upt-rubygems and sysutils/py-upt-cpan).
Bug reports and comments about this new tool are welcome.
Wine on FreeBSD
| Links | |
| Wine homepage | URL: https://www.winehq.org |
Contact: Gerald Pfeifer <[email protected]>
A lot has happened since our last quarterly report. The Wine 4 release series has been in our tree for nearly a year and proven rather stable. Both that port and wine-devel, which tracks bi-weekly development releases, have seen regular adjustments to infrastructure changes and small improvements, in particular also around non-default options.
Now we need help!
WoW64 (or Wine on Wine 64) allows running both 32-bit and 64-bit Windows applications in one installation. A volunteer has proposed
- a general framework for lib32- companion libraries https://reviews.freebsd.org/D16830
- an approach directly using our Wine ports https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=242625
to make this work and we do not have the expertise nor facilities to properly review, test, and maintain those ourselves.
If you can facilitate getting (at least one of) these into the tree, please help! And if you'd like to assume co-maintainership or sole maintainership of emulators/wine*, that is an option, too.
Third-Party Projects
Many projects build upon FreeBSD or incorporate components of FreeBSD into their project. As these projects may be of interest to the broader FreeBSD community, we sometimes include brief updates submitted by these projects in our quarterly report. The FreeBSD project makes no representation as to the accuracy or veracity of any claims in these submissions.
sysctlbyname-improved
| Links | |
| gitlab.com/alfix/sysctlbyname-improved | URL: https://gitlab.com/alfix/sysctlbyname-improved |
Contact: Alfonso Sabato Siciliano <[email protected]>
The FreeBSD kernel maintains a Management Information Base (MIB) where a component (object) represents a parameter of the system. The sysctl() system call explores the MIB to find an object by its Object Identifier (OID) and calls its handler to get or set the value of the parameter.
The sysctlbyname() syscall (or the old function) accepts the name of the object (instead of its OID) to identify it. The purpose of this project is to allow sysctlbyname() to handle:
- a CTLTYPE_NODE with a no-NULL handler, example "kern.proc.pid.\<pid\>";
- an object with some level-name equals to the '\0' character, example "security.jail.param.allow.mount.";
A sysctlbyname() clone is provided: sysctlbyname_improved(), the implementation core is a new sysctl internal node to get the OID of a node by its name eventually expanded with an input for its handler; both, can be installed via _sysutils/sysctlbyname-improved-kmod_. The internal node is also used by the sysctlmif_oidinputbyname() function of the _devel/libsysctlmibinfo2_ userland library and can be handled by the SYSCTLINFO_BYNAME macro of the sysctlinfo interface (described in the previous quarterly status report).
pot and the nomad pot driver
| Links | |
| Nomad pot driver | URL: https://github.com/trivago/nomad-pot-driver |
| Pot project | URL: https://github.com/pizzamig/pot |
| minipot | URL: https://github.com/pizzamig/minipot |
Contact: Luca Pizzamiglio <[email protected]>
Contact: Esteban Barrios <[email protected]>
The pot utility added support to private bridges: a group of jail can now use a dedicated bridge, instead of the public one, improving isolation. Moreover, several small bugs have been found and fixed, and support to pre/post start/stop hook script has been added.
The nomad pot driver received support for nomad restart without drain and improved configuration stability.
A new port called minipot has been added: this port will install configuration files and dependencies, converting a FreeBSD machine in a single node cluster. It will install nomad, consul, pot, the nomad pot driver and traefik, already configured and ready to use.
Experimental work has been done on a tool that allows to create and run pot images (FreeBSD jails) on other operating systems (Linux and Mac), adopting an approach similar to docker machine. We hope to make this tool available soon.
Next steps:
- add dual IP stack support to pot
- add private bridge support to the nomad pot driver
- improve usability to create images
This project was sponsored by trivago N.V..
7 Days Challenge
| Links | |
| 7 Days Challenge | URL: https://wiki.freebsd.org/MichaelCrilly/7dayschallenge |
Contact: Michael Crilly <[email protected]>
The 7 Days Challenge is an educational initiative to help people onboard with FreeBSD more easily.
It will use a combination of tutorials, guides and how-tos to get users engaged with FreeBSD quickly, target specific end goals the user might have for FreeBSD, and more.
The primary objective is to demonstrate FreeBSD's capabilities as a modern, relevant operating system in today's Cloud centric, automated business models.
This project was sponsored by OpsFactory Pty Ltd (Australia).
NomadBSD
| Links | |
| NomadBSD Website | URL: https://www.nomadbsd.org/ |
| NomadBSD Github | URL: https://www.github.com/NomadBSD/NomadBSD |
| NomadBSD Developer Mailing List | URL: https://www.freelists.org/list/nomadbsddevs |
Contact: NomadBSD Team <[email protected]>
NomadBSD is a persistent live system for USB flash drives, based on FreeBSD. Together with automatic hardware detection and setup, it is configured to be used as a desktop system that works out of the box, but can also be used for data recovery, for educational purposes, or testing FreeBSD's hardware compatibility.
After one release candidate the NomadBSD Team finished and released NomadBSD 1.3 on December 7th. This release is based on FreeBSD 12.1, fixed a lot of bugs and added new packages and features. Along those features are the option to install NomadBSD on ZFS and the use of an automatic configuration when running NomadBSD in VirtualBox.
New tools developed by the NomadBSD Team and added to version 1.3 are nomadbsd-dmconfig to select a display manager theme, nomadbsd-adduser which adds new user accounts and DSBBg to change the background image. All these are using the Qt-Toolkit.
In Q4 we added two mirrors in France and Germany and would like to thank nosheep.fr and fau.de for them.
We are looking for people to help the project. Help is much appreciated in all areas:
- Translation of program interfaces
- Design artwork
- Programming new tools, extend existing ones
- Tests and Bug reports / UX and feature suggestions
- Mirrors outside of Europe
Open tasks:
- Support installation on disk partitions and add a partition editor GUI.
- Complete disk encryption
- Add a user-friendly network manager
News Home | Status Home