Skip site navigation (1) Skip section navigation (2)

Introduction

This report will be covering FreeBSD related projects between April and June, and covers a diverse set of topics ranging from kernel updates over userland and ports, as well as third-party work.

Some highlights picked with the roll of a d100 include, but are not limited to, the ability to forcibly unmount UFS when the underlying media becomes inaccessible, added preliminary support for Bluetooth Low Energy, a introduction to the FreeBSD Office Hours, and a repository of software collections called potluck to be installed with the pot utility, as well as many many more things.

As a little treat, readers can also get a rare report from the quarterly team.

Finally, on behalf of the quarterly team, I would like to extend my deepest appreciation and thank you to salvadore@, who decided to take down his shingle. His contributions to not just the quarterly reports themselves, but also the surrounding tooling to many-fold ease the work, are immeasurable.

We hope you find the report as interesting as we have,
Daniel Ebdrup Jensen (debdrup@), on behalf of the quarterly team.

FreeBSD Team Reports

Projects

Kernel

Architectures

Userland Programs

Ports

Documentation

Miscellaneous

Third-Party Projects


    FreeBSD Team Reports

    Entries from the various official and semi-official teams, as found in the Administration Page.


    FreeBSD Foundation

    Contact: Deb Goodkin <[email protected]>

    The FreeBSD Foundation is a 501(c)(3) non-profit organization dedicated to supporting and promoting the FreeBSD Project and community worldwide. Funding comes from individual and corporate donations and is used to fund and manage software development projects, conferences and developer summits, and provide travel grants to FreeBSD contributors. The Foundation purchases and supports hardware to improve and maintain FreeBSD infrastructure and provides resources to improve security, quality assurance, and release engineering efforts; publishes marketing material to promote, educate, and advocate for the FreeBSD Project; facilitates collaboration between commercial vendors and FreeBSD developers; and finally, represents the FreeBSD Project in executing contracts, license agreements, and other legal arrangements that require a recognized legal entity.

    Here are some highlights of what we did to help FreeBSD last quarter:

    COVID-19 Impact on the Foundation

    Like other organizations, we put policies in place for all of our staff members to work from home. We also put a temporary ban on travel for staff members. We are continuing our work supporting the community and Project, but some of our work and responses may be delayed because of changes in some of our priorities and the impact of limited childcare for a few of our staff members.

    Partnerships and Commercial User Support

    We help facilitate collaboration between commercial users and FreeBSD developers. We also meet with companies to discuss their needs and bring that information back to the Project. Not surprisingly, the stay at home orders, combined with our company ban on travel during Q2 made in-person meetings non-existent. However, the team was able to continue meeting with our partners and commercial users virtually. These meetings help us understand some of the applications where FreeBSD is used.

    Fundraising Efforts

    Last quarter we raised $268,400! Thank you to the individuals and organizations that stepped in to help fund our efforts. We’d like to thank Netflix, employees of Nginx, Beckhoff Automation, and Mozilla Foundation for their large contributions last quarter, which helped bring our 2020 fundraising effort to $339k. We hope other organizations will follow their lead and give back to help us continue supporting FreeBSD.

    These are trying times, and we deeply appreciate every donation that has come in from $5 to $150,000. We’re still here giving 110% to supporting FreeBSD!

    We are 100% funded by donations, and those funds go towards software development work to improve FreeBSD, FreeBSD advocacy around the world, keeping FreeBSD secure, continuous integration improvements, sponsoring BSD-related and computing conferences (even the virtual events!), legal support for the Project, and many other areas.

    Please consider making a donation to help us continue and increase our support for FreeBSD.

    We also have the Partnership Program, to provide more benefits for our larger commercial donors. Find out more information about the partnership program and share with your companies!

    OS Improvements

    A number of FreeBSD Foundation grant recipients started, continued working on, or completed projects during the second quarter. These include:

    • WiFi improvements

    • Linuxulator application compatibility

    • DRM / Graphics driver updates

    • Zstd compression for OpenZFS

    • Online RAID-Z expansion

    • if_bridge performance improvements

    You can find more details about most of these projects in other quarterly reports.

    Staff members also worked on a number of larger projects, including:

    • Run-Time Dynamic Linker (rtld) improvements

    • Improved FreeBSD support on Microsoft HyperV and Azure

    • Fine-grained locking for amd64 pmap

    • 5-level paging structures for amd64

    • Non-transparent superpages

    • Migration to a Git repository

    • Tool chain modernization

    Many of these projects also have detailed entries in other quarterly report entries.

    Staff members also put in significant effort in many ways other than larger, individual projects. These include assisting with code reviews, bug report triage, security report triage and advisory handling, addressing syzkaller reports, and ongoing maintenance and bug fixes in functional areas such as the tool chain, developer tools, virtual memory kernel subsystem, low-level x86 infrastructure, sockets and protocols, and others.

    University of Waterloo Co-op

    Foundation co-op students Colin, Tiger, and Yang completed their winter 2020 work term during the second quarter, and continued on with the next school term in their respective programs. Although COVID-19 presented a unique challenge and prompted an abrupt transition to remote work just over half way through the term, all three learned a lot and provided positive contributions to the FreeBSD Project and to the Foundation.

    A few projects that were in progress or completed during the work term were committed to the FreeBSD tree in the second quarter.

    Continuous Integration and Quality Assurance

    The Foundation provides a full-time staff member who is working on improving continuous integration, automated testing, and overall quality assurance efforts for the FreeBSD project.

    During the second quarter of 2020, Foundation staff continued improving the Project's CI infrastructure, monitoring regressions and working with contributors to fix the failing build and test cases. The setting up of VM host for CI jobs and staging environment is in progress. We are also working with other teams in the Project for their testing needs. For example, we added jobs for running full tests on non-x86 architectures. We are also working with many external projects and companies to improve their support of FreeBSD.

    See the FreeBSD CI section of this report for completed work items and detailed information.

    Supporting FreeBSD Infrastructure

    The Foundation provides hardware and support to improve FreeBSD infrastructure. Last quarter, we continued supporting FreeBSD hardware located around the world. We started working on getting the new NYI Chicago colocation facility prepared for some of the new FreeBSD hardware we are planning on purchasing. NYI generously provides this for free to the Project.

    FreeBSD Advocacy and Education

    A large part of our efforts are dedicated to advocating for the Project. This includes promoting work being done by others with FreeBSD; producing advocacy literature to teach people about FreeBSD and help make the path to starting using FreeBSD or contributing to the Project easier; and attending and getting other FreeBSD contributors to volunteer to run FreeBSD events, staff FreeBSD tables, and give FreeBSD presentations.

    The FreeBSD Foundation sponsors many conferences, events, and summits around the globe. These events can be BSD-related, open source, or technology events geared towards underrepresented groups. We support the FreeBSD-focused events to help provide a venue for sharing knowledge, to work together on projects, and to facilitate collaboration between developers and commercial users. This all helps provide a healthy ecosystem. We support the non-FreeBSD events to promote and raise awareness of FreeBSD, to increase the use of FreeBSD in different applications, and to recruit more contributors to the Project. As is the case for most of us in this industry, COVID-19 has put our in-person events on hold. In addition to attending virtual events, we are continually working on new training initiatives and updating our selection of how-to guides to facilitate getting more folks to try out FreeBSD.

    Check out some of the advocacy and education work we did last quarter:

    • Silver sponsor of BSDCan 2020. The event was held virtually, June 2-6, 2020

    • Community Sponsor of Rootconf 2020. The event was held virtually, June 19-20, 2020

    • Annual FreeBSD Day, June 19. This year’s celebration was postponed in support of Juneteeth. However the activities surrounding FreeBSD Day have been transformed into an ongoing series of online sessions. See FreeBSD Fridays below for more information.

    • Presented 27 Years of FreeBSD and Why You Should Get Involved as part of a Linux Professional Institute series of webinars on June 24, 2020.

    • Attended and presented at the virtual Open Source Summit 2020.

    • Announced FreeBSD Fridays: A series of 101 classes designed to get you started with FreeBSD. Find out more in the announcement

    • Participated as an Admin for Google Summer of Code 2020

    • Participated in the new FreeBSD Office Hours series including holding our own Foundation led office hours. Videos from the one hour sessions can be found on the Project’s YouTube Channel. You can watch ours here.

    In addition to the information found in the Development Projects update section of this report, take a minute to check out the latest update blogs:

    Keep up to date with our latest work in our Bi-Monthly newsletters.

    Mellanox provided an update on how and why they use FreeBSD in our latest Contributor Case Study.

    We help educate the world about FreeBSD by publishing the professionally produced FreeBSD Journal. As we mentioned previously, the FreeBSD Journal is now a free publication. Find out more and access the latest issues on the Journal site.

    You can find out more about events we attended and upcoming events.

    We have continued our work with a new website developer to help us improve our website. Work is nearly complete to make it easier for community members to find information more easily and to make the site more efficient. We look forward to unveiling the refreshed site in Q3.

    Foundation Board Meeting

    Our annual board meeting was held on Tuesday June 2, 2020. We normally hold this meeting the Tuesday before BSDCan, in Ottawa, Ontario, Canada, but with the company travel ban, and the conference going virtual, our meeting went virtual for the first time. The purpose of the annual board meeting is to hold our board director and officer elections, review work accomplished over the past year, and put together strategic goals for the upcoming 12 months.

    The board generally has two all-day board meetings each year, this one, and a more informal one in January, typically held in Berkeley. Both meetings allow us to connect, reevaluate and discuss new ideas, while assessing what we should do to help the Project.

    Some of our longer-term goals include Growing User and Developer Communities, Developing Training and OS Course Content, Improving desktop/laptop experience, Promoting FreeBSD (as you can see in all the advocacy work listed above), and Improving Testing Capabilities.

    Results of the director and officer elections were:

    • Justin Gibbs (President)

    • Benedict Reuschling (Vice President)

    • Kirk McKusick (Treasurer)

    • Philip Paeps (Secretary)

    • Deb Goodkin (Assistant Secretary)

    • Robert Watson (Director)

    • Hiroki Sato (Director)

    • George Neville-Neil (Director)

    Find out more about the FreeBSD Foundation Board of Directors on our website.

    Legal/FreeBSD IP

    The Foundation owns the FreeBSD trademarks, and it is our responsibility to protect them. We also provide legal support for the core team to investigate questions that arise.

    Go to the FreeBSD Foundation's web site to find out how we support FreeBSD and how we can help you!

    FreeBSD Core Team

    Contact: FreeBSD Core Team <[email protected]>

    The FreeBSD Core Team is the governing body of FreeBSD.

    The Core Team held 10 meetings during the second quarter of 2020, including a 2020-05-21 joint meeting with members of the FreeBSD Foundation. Here are some highlights from that meeting:

    • Deb requested guidance on how the Foundation can support the community. Core and Foundation members believe that more developer support is necessary to fill gaps in areas where commercial customers do not provide backing. The clearest example of such a gap is the desktop experience, including graphics and wireless support. What makes this request different from past requests is that rather than support for one-time projects, ongoing positions are necessary for a consistently high-quality desktop experience.

      "FreeBSD not being able to run on your laptop is the first step to irrelevance." Ed Maste

    • Both teams discussed topics for upcoming sessions of FreeBSD Office Hours, informal FreeBSD video conferences that anyone can attend. Everyone agreed that the Office Hours have been a useful way for different parts of the Project to engage with each other and with the wider community. Kudos to Allan Jude for initiating the Office Hours and for everyone who has helped make them a success by hosting or attending sessions.

    • Both teams agreed that they should meet once per quarter.

    The second annual community survey closed on 2020-06-16. The purpose of the survey is to collect data from the public to help guide the Project's efforts and priorities. As an example, last year's survey results helped initiate the Project's conversion to Git. Thank you to all who took the time to respond. The results will be released soon.

    The Core-initiated Git Working Group continued to make progress, but there are still some remaining issues to be worked out with the translation from Subversion. Hopefully the new Git src repository will be ready for use this summer. A beta version has been published for people to test and a preliminary version of a Using Git for FreeBSD Development primer will soon be ready to share. Core, the Git Working Group, and Release Engineering are working towards the goal of releasing 12.2 from the new Git repository.

    Following the results of a Core-initiated developer survey, The FreeBSD Project has adopted a new LLVM-derived [code of conduct](https://www.freebsd.org/internal/code-of-conduct.html).

    The eleventh FreeBSD Core Team was elected by active developers. From a pool of 23, the 9 successful candidates for core.11 are:

    • Sean Chittenden (seanc, incumbent)

    • Baptiste Daroussin (bapt)

    • Kyle Evans (kevans)

    • Mark Johnston (markj)

    • Scott Long (scottl)

    • Warner Losh (imp, incumbent)

    • Ed Maste (emaste)

    • George V. Neville-Neil (gnn)

    • Hiroki Sato (hrs, incumbent)

    A new Core Team secretary, Muhammad Moinur Rahman (bofh), was unanimously approved by core.11. The outgoing core team met three times with the new core team to help with the transition. Core.10 wishes core.11 a successful term.

    FreeBSD Release Engineering Team

    Links
    FreeBSD 11.4-RELEASE announcement URL: https://www.freebsd.org/releases/11.4R/announce.html
    FreeBSD 11.4-RELEASE schedule URL: https://www.freebsd.org/releases/11.4R/schedule.html
    FreeBSD development snapshots URL: https://download.freebsd.org/ftp/snapshots/ISO-IMAGES/

    Contact: FreeBSD Release Engineering Team <[email protected]>

    The FreeBSD Release Engineering Team is responsible for setting and publishing release schedules for official project releases of FreeBSD, announcing code slushes, and maintaining the respective branches, among other things.

    During the second quarter of 2020, the Release Engineering Team started work on the 11.4-RELEASE cycle, the fifth release from the stable/11 branch. The release cycle went quite smoothly, with both BETA3 and RC3 removed from the schedule. This allowed the final release to occur one week earlier than originally scheduled, which was announced June 16. FreeBSD 11.4-RELEASE is expected to be the final 11.x release.

    The FreeBSD Release Engineering Team would like to thank everyone involved in this cycle for their hard work.

    Additionally throughout the quarter, several development snapshots builds were released for the head, stable/12, and stable/11 branches.

    Much of this work was sponsored by Rubicon Communications, LLC (netgate.com) and the FreeBSD Foundation.

    Cluster Administration Team

    Links
    Cluster Administration Team members URL: https://www.freebsd.org/administration.html#t-clusteradm

    Contact: Cluster Administration Team <[email protected]>

    The FreeBSD Cluster Administration Team consists of the people responsible for administering the machines that the Project relies on for its distributed work and communications to be synchronised. In this quarter, the team has worked on the following:

    • Upgrade all x86 ref- and universe-machines

    • Setup Amsterdam (PKT) mirror

    • Solve hardware issue for bugzilla and svnweb backend

    • Setup public beta git server

    • Decommission CyberOne Data (CYB) mirror

    • Ongoing systems administration work:

      • Accounts management for committers.

      • Backups of critical infrastructure.

      • Keeping up with security updates in 3rd party software.

    Work in progress:

    • Setup Malaysia (KUL) mirror

    • Setup Brazil (BRA) mirror

    • Review the service jails and service administrators operation.

    • Infrastructure of building aarch64 and powerpc64 packages

      • NVME issues on PowerPC64 Power9 blocking dual socket machine from being used as pkg builder.

      • Drive upgrade test for pkg builders (SSDs) courtesy of the FreeBSD Foundation.

      • Boot issues with Aarch64 reference machines.

    • New NYI.net sponsored colocation space in Chicago-land area.

    • Work with git working group

    • Check new hardware requirement from other teams

    • Searching for more providers that can fit the requirements for a generic mirrored layout or a tiny mirror

    Continuous Integration

    Links
    FreeBSD Jenkins Instance URL: https://ci.FreeBSD.org
    FreeBSD Hardware Testing Lab URL: https://ci.FreeBSD.org/hwlab
    FreeBSD CI artifact archive URL: https://artifact.ci.FreeBSD.org
    FreeBSD CI weekly report URL: https://hackmd.io/@FreeBSD-CI
    FreeBSD Jenkins wiki URL: https://wiki.freebsd.org/Jenkins
    Hosted CI wiki URL: https://wiki.freebsd.org/HostedCI
    3rd Party Software CI URL: https://wiki.freebsd.org/3rdPartySoftwareCI
    Tickets related to freebsd-testing@ URL: https://preview.tinyurl.com/y9maauwg
    FreeBSD CI Repository URL: https://github.com/freebsd/freebsd-ci

    Contact: Jenkins Admin <[email protected]>
    Contact: Li-Wen Hsu <[email protected]>

    Contact: freebsd-testing Mailing List
    Contact: IRC #freebsd-ci channel on EFNet

    The FreeBSD CI team maintains the continuous integration system for the FreeBSD project. The CI system firstly checks the committed changes can be successfully built, then performs various tests and analysis over the newly built results. The artifacts from the build jobs are archived in the artifact server for further testing and debugging needs. The CI team members examine the failing builds and unstable tests and work with the experts in that area to fix the code or adjust test infrastructure. The details of these efforts are available in the weekly CI reports.

    During the second quarter of 2020, we continue working with the contributors and developers in the project for their testing needs and also keep working with external projects and companies to improve their support of FreeBSD.

    Important changes:

    • All -test jobs will run tests under /usr/tests, previously only x86 architectures were doing this. See the Continuous Integration on !x86 section in this report for more information.

    • Compression algorithm of disk images on the artifact server has been changed to zstd to speed up compression and decompression.

    • The build and test results will be sent to the dev-ci mailing list soon. Feedback and help with analysis is very appreciated!

    New jobs added:

    • https://ci.freebsd.org/job/FreeBSD-head-armv7-test/

    • https://ci.freebsd.org/job/FreeBSD-head-aarch64-test/

    • https://ci.freebsd.org/job/FreeBSD-head-mips64-test/

    • https://ci.freebsd.org/job/FreeBSD-head-powerpc64-test/

    Work in progress:

    • Collecting and sorting CI tasks and ideas here

    • Testing and merging pull requests in the the FreeBSD-ci repo

    • Setting up a builder dedicated to run jobs using provisioned VMs.

    • Setting up the CI stage environment and putting the experimental jobs on it

    • Implementing automatic tests on bare metal hardware

    • Adding drm ports building tests against -CURRENT

    • Planning to run ztest and network stack tests

    • Adding external toolchain related jobs

    • Improving the hardware lab to be more mature and adding more hardware

    • Helping more 3rd software get CI on FreeBSD through a hosted CI solution

    • Working with hosted CI providers to have better FreeBSD support

    Please see freebsd-testing@ related tickets for more WIP information, and don't hesitate to join the effort!

    Sponsor: The FreeBSD Foundation

    Ports Collection

    Links
    About FreeBSD Ports URL: https://www.FreeBSD.org/ports/
    Contributing to Ports URL: https://www.freebsd.org/doc/en_US.ISO8859-1/articles/contributing/ports-contributing.html
    FreeBSD Ports Monitoring URL: http://portsmon.freebsd.org/index.html
    Ports Management Team URL: https://www.freebsd.org/portmgr/index.html

    Contact: Ren� Ladan <[email protected]>
    Contact: FreeBSD Ports Management Team <[email protected]>

    The Ports Management Team is responsible for overseeing the overall direction of the Ports Tree, building packages, and personnel matters. Below is what happened in the last quarter.

    There are currently 2,373 open ports PRs of which 526 are unassigned, for a total of 39,628 ports. In the last quarter there were 10,315 commits to HEAD and 476 to the quarterly branch by respectively 178 and 65 committers. Compared to the quarter before, this means a significant increase in commits and also a slight decrease in open PRs.

    During the last quarter, we welcomed Hiroki Tagato (tagattie@). We said goodbye to seanc@, zleslie@, gnn@ and salvadore@.

    A few default versions got bumped:

    • Java (new) at 8

    • Lazarus to 2.0.8

    It is now possible to write pkg scripts in Lua instead of sh.

    They have two advantages over their sh versions:

    • they run in a Capsicum sandbox

    • they respect rootdir, the directory which pkg will use as the starting point to install all packages under.

    Some user-facing packages were also updated:

    • pkg to 1.14.6

    • Firefox to 78.0.1

    • Thunderbird to 68.10.0

    • Chromium to 83.0.4103.116

    • Ruby to 2.5.8, 2.6.6, and 2.7.1

    • Qt5 to 5.14.2

    During the last quarter, antoine@ ran 55 exp-runs to test port version updates, make liblzma use libmd, flavor devel/scons and Lua ports, add and update library functions in the base system, make malloc.h usable again, remove as(1) from the base system, and augment sed(1) with -f.

    FreeBSD Office Hours

    Links
    Office Hours on the FreeBSD Wiki URL: https://wiki.freebsd.org/OfficeHours
    Poll: What time would you prefer Office Hours be at URL: https://forms.gle/3HjjRx9KMcM3SL4H7
    live.FreeBSD.org: Aggregation of Live streams URL: https://live.freebsd.org/

    Contact: Allan Jude <[email protected]>

    Starting on the first of April 2020, the FreeBSD project has started hosting regular video streams to foster greater communication within the wider FreeBSD community. The first of these sessions took the form of a public question and answer session, which drew over 60 participants. A second session was held two weeks later at a time more appropriate for those in Asia, but only drew 20 participants. With the help of the FreeBSD Foundation, we ran a poll to discover what times worked best for the greatest number of people.

    On May 13th the FreeBSD Foundation hosted a session where the community could ask questions of or about the foundation. On May 27th many of the candidates for the new FreeBSD Core Team joined an office hours session to answer questions from the community. Finally on June 24th another general question and answer office hours was held.

    Each office hours session consists of a video meeting of some FreeBSD developers or other subject matter experts, live streamed along with an IRC chat room for viewers to pose questions to the panel. The stream is recorded and posted to the official FreeBSD youtube channel.

    If you would like to host an office hours session, please contact:

    Sponsor: ScaleEngine Inc. (video streaming)

    Quarterly Status Reports Team

    Links
    Quarterly status reports URL: https://www.freebsd.org/news/status/
    Git repository URL: https://github.com/freebsd/freebsd-quarterly

    Contact: Quarterly Status Reports <[email protected]>
    Contact: Daniel Ebdrup Jensen <[email protected]>

    The Quarterly Status Reports Team collects and publishes the reports that you are reading right now.

    Many improvements have been done recently and thus we believe it is useful that the Quarterly Status Reports Team submits a report. Not all the changes below are specific to the last quarter, but we list them here anyway since we did not write an entry for earlier reports.

    • Reports are now built using Makefiles. Among the many advantages, this allows us to easily sort reports logically. Indeed, starting with 2019Q4, all reports are sorted logically, while before they were sorted alphabetically within each category.

    • The conversion from markdown to docbook was performed using a python script, with some known bugs. Salvadore has rewritten the script using perl fixing most of the bugs. Some features are missing and many improvements are possible, but the script is very unlikely to receive any change since it will become obsolete as soon as the conversion to Hugo/AsciiDoctor is completed.

    • Another perl script to ease the preparation of the mail version of the reports was written.

    • One more perl script has been written to allow the quarterly team to send quarterly calls automatically using a cron job. We used it this quarter for the first time.

    • As you might have noticed, last quarterly calls have been sent to freebsd-quarterly-calls@: this is a new mailing list to which you can subscribe to receive calls for quarterly reports. Please note this is a moderated list, with very low traffic and a high signal to noise ratio.

    • If you read carefully the last quarterly calls, you should have noticed that we now ask you to send reports to quarterly-submissions@ instead of quarterly@. This was done to help the quarterly team distinguishing internal discussions from submissions. Please keep in mind however that the quarterly team prefers receiving pull requests, as they ease the administrative work.

    We would like to thank philip@, from the postmaster team, for having created the freebsd-quarterly-calls@ mailing list and the quarterly-submissions@ address for us.


    Projects

    Projects that span multiple categories, from the kernel and userspace to the Ports Collection or external projects.


    FreeBSD on Microsoft HyperV and Azure

    Links
    FreeBSD on MicrosoftAzure wiki URL: https://wiki.freebsd.org/MicrosoftAzure
    FreeBSD on Microsoft HyperV URL: https://wiki.freebsd.org/HyperV

    Contact: FreeBSD Integration Services Team <[email protected]>
    Contact: Wei Hu <[email protected]>
    Contact: Li-Wen Hsu <[email protected]>

    HyperV socket for FreeBSD implemented by Wei was checked into FreeBSD head branch on May 20th as r361275. It supports guest/host communications without the need for networking. Some HyperV and Azure features rely on this to be available in guests.

    Details of HyperV Socket is available here.

    This project is sponsored by Microsoft.

    Li-Wen is working on the FreeBSD release code related to Azure for the -CURRENT, 12-STABLE and 11-STABLE branches. The work-in-progress is available here. The 12.1-RELEASE image on Azure Marketplace is published. The work on the 11.4-RELEASE image on Azure Marketplace is in progress.

    This project is sponsored by The FreeBSD Foundation.

    Git Migration Working Group

    Links
    Git conversion tooling repo URL: https://github.com/freebsd/git_conv
    FreeBSD-git mailing list URL: https://lists.freebsd.org/mailman/listinfo/freebsd-git
    Beta doc git repo URL: https://cgit-beta.FreeBSD.org/doc
    Beta ports git repo URL: https://cgit-beta.FreeBSD.org/ports
    Beta src git repo URL: https://cgit-beta.FreeBSD.org/src

    Contact: Ed Maste <[email protected]>
    Contact: Warner Losh <[email protected]>
    Contact: Ulrich Sp�rlein <[email protected]>

    Work continues on FreeBSD's migration from Subversion to Git. Ulrich has iterated on updates to svn2git in order to improve the fidelity of the conversion, particularly in regards to vendor (contrib) code updates. We believe the conversion is now at an acceptable state, but may make minor adjustments if additional issues are found. We expect to push modifications to the converter every two weeks (first and third Sunday of the month). This means that commit hashes in the beta repo will remain stable for at least two weeks at a time, to allow others to test and experiment with the beta repo.

    We are now working on updating FreeBSD processes and documentation. This includes:

    • Writing a Git Primer, akin to the existing Subversion primer

    • Updates to the Security Team's tools and processes

    • Release engineering updates

    • Ports and packages process updates

    Those with an interest in the migration to Git are encouraged to subscribe to the FreeBSD-git mailing list and test out the beta src, ports, and/or doc repositories.

    You are also welcome to check out the wiki, issues, README and other documentation at the Git conversion tooling repo.

    We expect to be ready for the migration in the next quarter.

    Sponsor: The FreeBSD Foundation (in part)

    Lua Usage in FreeBSD

    Contact: Ed Maste <[email protected]>
    Contact: Kyle Evans <[email protected]>
    Contact: Ryan Moeller <[email protected]>

    Lua is a small, efficient scripting language that FreeBSD imported before FreeBSD 12.0 for use in the bootloader. Since then, several projects outside of the bootloader have gained some amount of traction with Lua usage:

    • /usr/libexec/flua is now installed for internal usage

    • makesyscalls.sh was rewritten in Lua

    • pkg has gained support for lua scripts

    • lldb in the base system now supports lua scripting

    FreeBSD Lua ("flua") is a version of the lua interpreter that has several modules built-in for convenient usage within the base system. flua is installed with a non-standard name and in a location not included in $PATH so that it is not accidentally found by third-party software or configure scripts. The FreeBSD project makes no guarantees about upgrade cadence or module stability. That said, it is available for use by downstream projects and FreeBSD users aware of those limitations.

    Previous work with flua includes, for example, adding libucl support and future work includes libifconfig support for scripting usage.

    People interested in working with Lua in FreeBSD are welcome to get in contact to discuss other project ideas. To name a couple of potential projects, some interesting modules that have not been started but could prove useful (listed in no particular order):

    • libcrypt

    • libexpat

    • libjail

    • libnv

    • libxo

    Linux compatibility layer update

    Contact: Edward Tomasz Napierala <[email protected]>

    Earlier Linuxulator work focused on code cleanups and improving diagnostic tools. Work has now shifted from cleanups to fixing actual applications. Current status is being tracked at Linux app status Wiki page. Initial focus is on applications that don't involve X11, mostly because they tend to be easier to test and debug, and the bug fixes are not application-specific.

    Example problems fixed include buggy madvise(2) handling, which could break applications linked against jemalloc; uname(2) returning wrong results for 32 bit apps, which caused problems for Steam; recvmsg(2) and accept(2) being broken in some circumstances, which was breaking Redis; and missing support for SO_REUSEPORT, SO_SNDBUFFORCE, SO_RCVBUFFORCE, and SO_PROTOCOL, which spammed the log files when running the Python regression test suite. The default soft open files limit is now automatically adjusted to 1024, as several Linux apps iterate over all the file descriptors up to that limit instead of using closefrom(2).

    There's ongoing work on cleanups and the debugging framework for Linux compatibility, such as logging warnings for unrecognized system call parameters, or adding the compat.linux.debug sysctl to turn the warnings off.

    The Linux Test Project tests that are being run as part of the the FreeBSD Continuous Integration infrastructure has been upgraded to 20200605 snapshot. This raised the number of test cases from 3670 to 3749, and, predictably, also the number of failures, from 583 to 647.

    There's still a lot to do:

    Sponsor: The FreeBSD Foundation

    NFS over TLS implementation

    Contact: Rick Macklem <[email protected]>

    In an effort to improve NFS security, an internet draft which I expect will become an RFC soon specifies the use of TLS 1.3 to encrypt all data traffic on a Sun RPC connection used for NFS.

    Although NFS has been able to use sec=krb5p to encrypt data on the wire, this requires a Kerberos environment and, as such, has not been widely adopted. It also required that encryption/decryption be done in software, since only the RPC message NFS arguments are encrypted. Since Kernel TLS is capable of using hardware assist to improve performance and does not require Kerberos, NFS over TLS may be more widely adopted, once implementations are available.

    The project to implement this has largely been completed. The code will slowly be merged into head/current and at least the kernel portion should be in FreeBSD-13.

    To support clients such as laptops, the daemons that perform the TLS handshake may optionally handle client X.509 certificates from a site local CA. There are now exports(5) options to require client(s) to provide a valid X.509 certificate.

    The code is now available for testing. See: https://people.freebsd.org/~rmacklem/nfs-over-tls-setup.txt Setting up system(s) for testing is still a little awkward, as explained by the above rough document.

    The main limitation in the current implementation is that it uses TLS1.2 and not TLS1.3. This should change once the KERN_TLS rx patch includes TLS1.3 support. Also, testing of pNFS configurations has not yet been done, but will be tested soon.

    Third party testing would be appreciated.


    Kernel

    Updates to kernel subsystems/features, driver support, filesystems, and more.


    SoC audio framework and more audio drivers

    Links
    rk3399_audio URL: https://github.com/gonzoua/freebsd/tree/rk3399_audio

    Contact: Oleksandr Tymoshenko <[email protected]>

    SoC audio framework made a good progress since last report.
    Support for AUX devices was added (devices like auxiliary amplifiers that are not part of main CODEC chip).
    To verify the framework design following audio drivers were added: recording support for RT5640 CODEC(Firefly-RK3399), Allwinner I2S, Alwinners sun8i and A64 CODECs (Pine A64+), both recording and playback.
    Current work in progress is RK3328 CODEC (Rock64) and ES8316 CODEC (RockPro64 and Pinebook Pro).

    bhyve - NVMe emulation improvements

    Links
    bhyve NVMe reviews URL: https://reviews.freebsd.org/search/query/xvbcF20W__Km/

    Contact: Chuck Tuffli <[email protected]>

    The University of New Hampshire InterOperability Laboratory (a.k.a. UNH IOL) develops a suite of tests to determine if an NVMe device conforms to the specification and is interoperable with other NVMe products. This quarter, I undertook getting bhyve's emulated NVMe device to pass the mandatory tests. Changes include:

    • implement Flush command

    • implement Format NVM command

    • implement AER support

    • implement Namespace Identification Descriptor

    • fix Active Namespace list

    • fix queue creation and deletion

    • validate Deallocate range values

    • handle zero length DSM ranges

    • fix Get Log Page command

    • implement SMART data I/O statistics

    • validate the LBA start and count

    • add basic Firmware Commit support

    • add more compliant Get/Set Features

    • add Feature, Interrupt Vector Config

    • fix Get Features, Predictable Latency

    This was also a good opportunity to restructure parts of the code to make it more modular and easier to enhance. This includes

    • convert logging statements to parameterized macros

    • refactor I/O command handling

    • add locks around queue accesses

    • consolidate CQ update

    • base pci_nvme_ioreq size on advertised MDTS

    You can help by testing and/or commenting on the code reviews.

    Bluetooth Support

    Contact: Marc Veldman <[email protected]>

    Bluetooth is a wireless technology for creating personal networks, connecting peripherals like keyboards and mice but also speakers and heart rate monitors.

    FreeBSD has limited Bluetooth Basic Rate (BR) support and no functional Bluetooth Low Energy (LE) support.

    During this quarter many small improvements have been made to help the development of Bluetooth LE support. A number of commands have been added to hccontrol(8), mainly to do LE functions. It is now possible to scan for LE devices within range using hccontrol. A panic that occurred when enabling LE support has also been fixed.

    Work is still needed to add Attribute Protocol (ATT) and Generic Attribute Profile (GATT) support.

    DRM Drivers Update

    Links
    drm-kmod URL: https://github.com/freebsd/drm-kmod/

    Contact: Emmanuel Vadot <[email protected]>

    The drm drivers for FreeBSD 13-CURRENT have been updated to match Linux 5.3 This brings us a little bit closer to the last LTS release of Linux (5.4).

    The current plan is to first update the driver to match 5.4 and then look at making it work on FreeBSD-12-STABLE to have it ready for the 12.2 release.

    Sponsor: The FreeBSD Foundation

    DTS Update

    Contact: Emmanuel Vadot <[email protected]>

    DTS files (Device Tree Sources) were updated to be on par with Linux 5.7 for HEAD and 5.6 for the 12-STABLE branch.

    ENA FreeBSD Driver Update

    Links
    ENA README URL: https://github.com/amzn/amzn-drivers/blob/master/kernel/fbsd/ena/README

    Contact: Michal Krawczyk <[email protected]>
    Contact: Artur Rojek <[email protected]>
    Contact: Marcin Wojtas <[email protected]>

    ENA (Elastic Network Adapter) is the smart NIC available in the virtualized environment of Amazon Web Services (AWS). The ENA driver supports multiple transmit and receive queues and can handle up to 100 Gb/s of network traffic, depending on the instance type on which it is used.

    Completed since the last update:

    • Fixes for Rx refill to improve stability on low memory conditions (also released as an errata notice for FreeBSD-12.1)

    • Upstream of the v2.2.0 driver, introducing:

      • Add driver support for the upcoming HW features (reporting Tx drops, disabling meta caching)

      • Add sysctl tuneables for IO queue number

      • Create IO queues with optional size backoff

      • Rework the way of configration of drbr and Rx ring size to be more robust and stable

      • New HAL version

      • Driver is now marked as epoch ready

      • Default RSS key is created using RNG to improve security

      • Other minor fixes and improvements

    • MFC of the ENA v2.2.0 driver to the FreeBSD 11.4

    Sponsor: Amazon.com Inc

    Forcible Unmount of UFS/FFS Filesystems on Disk Failure

    Links
    Phabricator Details URL: https://reviews.freebsd.org/D24088

    Contact: Chuck Silvers <[email protected]>
    Contact: Kirk McKusick <[email protected]>

    Commit -r361491 on May 25, 2020 enables a UFS file system to do a forcible unmount when the underlying media fails or becomes inaccessible. For example when a USB flash memory card hosting a UFS file system is unplugged.

    The rest of this report describes in more detail how forcible unmounts are done. Surprisingly, less than 500 lines of file system code were added or changed.

    The strategy for handling disk I/O errors when soft updates are enabled is to stop writing to the disk of the affected file system but continue to accept I/O requests and report that all future writes by the file system to that disk actually succeed. Then initiate an asynchronous forced unmount of the affected file system.

    There are two cases for disk I/O errors:

    • ENXIO, which means that this disk is gone and the lower layers of the storage stack already guarantee that no future I/O to this disk will succeed.

    • EIO (or most other errors), which means that this particular I/O request has failed but subsequent I/O requests to this disk might still succeed.

    For ENXIO, we can just clear the error and continue, because we know that the file system cannot affect the on-disk state after we see this error. For EIO or other errors, we arrange for the geom_vfs layer to reject all future I/O requests with ENXIO just like is done when the geom_vfs is orphaned. In both cases, the file system code can just clear the error and proceed with the forcible unmount.

    This new treatment of I/O errors is needed for writes of any buffer that is involved in a dependency. Most dependencies are described by a structure attached to the buffer's b_dep field, but some are created and processed as a result of the completion of the dependencies attached to the buffer.

    Clearing of some dependencies require a read. For example if there is a dependency that requires an inode to be written, the disk block containing that inode must be read, the updated inode copied into place in that buffer, and the buffer then written back to disk.

    Often the needed buffer is already in memory and can be used. But if it needs to be read from the disk, the read will fail, so we fabricate a buffer full of zeroes and pretend that the read succeeded. This zero'ed buffer can be updated and "written" back to disk.

    The only case where a buffer full of zeros causes the code to do the wrong thing is when reading an inode buffer containing an inode that still has an inode dependency in memory that will reinitialize the effective link count (i_effnlink) based on the actual link count (i_nlink) that we read. To handle this case we now store the i_nlink value that we wrote in the inode dependency so that it can be restored into the zero'ed buffer thus keeping the tracking of the inode link count consistent.

    Because applications depend on knowing when an attempt to write their data to stable storage has failed, the fsync(2) and msync(2) system calls need to return errors if data fails to be written to stable storage. So these operations return ENXIO for every call made on files in a file system where we have otherwise been ignoring I/O errors.

    Sponsor: Netflix

    i.MX 8M support

    Links
    D25274 URL: https://reviews.freebsd.org/D25274

    Contact: Oleksandr Tymoshenko <[email protected]>

    i.MX 8M is the family of application processors from NXP based on Arm� Cortex�-A53 and Cortex-M4 cores. The initial code drop for the platform support includes CCM driver and clock implementation, GPC driver, clock tree for i.MX 8M Quad. Most of the drivers from i.MX 6 can be reused for i.MX 8M systems with relatively minor modifications. Common changes include adding clock support and extending list of FDT compat strings.

    With the linked patch FreeBSD successfully booted to multiuser with NFS root on Nitrogen8M SBC.

    Intel wireless and 11ac update

    Links
    Initial project announcement URL: https://lists.freebsd.org/pipermail/freebsd-wireless/2020-April/009055.html
    The freebsd-wireless mailing list URL: https://lists.freebsd.org/mailman/listinfo/freebsd-wireless

    Contact: Bjoern A. Zeeb <[email protected]>

    The Intel Wireless cards are one of the most commonly used and asked for in FreeBSD notebooks.

    This project has three main goals:

    • newer Intel Wireless device support,

    • newer WiFi standards support for Intel Wireless,

    • integration of 802.11ac client support and infrastructure in FreeBSD.

    The first one is needed as iwm(4) currently does not support the latest generations of Intel Wireless cards at all. The second is needed as in FreeBSD iwm(4) does not even support 802.11n. The third one we want to catch up and use the improvements the new Wifi standard offers, e.g., speed.

    One of the decisions made was: rather than improving iwm(4) this work uses the dual-licensed native Linux driver under BSD license and the linuxkpi framework to stay as close to upstream as possible as a first step. This will give us several advantages, such as, the full support for all cards, quick support for new chipsets, vendor bug fixes, but also the ability to contribute back.

    At this point the lower level hardware attachments and the firmware loading and initialisation works. I plan to release a patchset for testing before mid-July, you can see if your currently supported or unsupported hardware will be detected. This first cut will not support any wireless operation yet, which will follow later in the year.

    If you want to help testing, please watch the freebsd-wireless list.

    Sponsor: The FreeBSD Foundation

    amd64 5-Level Paging Structures support

    Links
    Patch URL: https://reviews.freebsd.org/D25273
    Intel SDM URL: https://software.intel.com/content/www/us/en/develop/articles/intel-sdm.html
    Intel whitepaper URL: https://software.intel.com/content/www/us/en/develop/download/5-level-paging-and-5-level-ept-white-paper.html

    Contact: Konstantin Belousov <[email protected]>

    Since its introduction, x86 Long Mode (AKA 64bit execution mode, amd64 in FreeBSD terminology) uses 4-level paging structures, which provides 48 bits of virtual address space (LA48). FreeBSD evenly divides the space between userspace and kernel, giving both 47 virtual address bits.

    In near future Intel CPUs will start providing 5-level paging structures, i.e. giving 57 bits for virtual addresses (LA57). This means, with preservation of the existing divide between KVA and UVA, 56 bit for UVA, or 2^9 = 512 times more virtual memory.

    The amd64 pmap was modified to support both LA48 and LA57, defaulting to LA57 if hardware supports it. The tunable is provided to force using LA48 even if hardware can do LA57.

    The most interesting part of the patch is the switch from boot paging mode to LA57. Loaders, either legacy or UEFI, pass control to the kernel in Long Mode, which implies that the paging is turned on. This necessarily means that it is LA48 mode. SDM states that paging mode cannot be switched while Long Mode is active, so kernel has to create new page table structures, turn Long Mode off, then load new %cr3 and finally re-enable Long Mode.

    I decided to only provide the larger virtual address space to usermode for the initial step, leaving KVA layout intact. The main motivation is that changing KVA arrangements requires changing the auto-tuning settings, which deserve separate work. Another argument for it is that most of the kernel memory is non-swappable, so cannot be over-commited. We have 2:1 ratio of useful KVA to physical memory (due to direct map), and until machines get more physical address lines, increasing KVA is not useful.

    After this was decided, creating a 5-level paging structure for kernel pmap from existing 4-level one is quite straightforward; we need to add one page for top level, create one PML5 entry to point to existing PML4 page, and create the famous self-referential entry for vtopte()/vtopde().

    Care was taken to provide binary compatible layout of UVA for binaries that cannot be executed correctly with larger address space. For instance, programs could have knowledge about used bits in the addresses and used upper bits for other data, or implemented compressed pointers. Even if system runs in LA57 mode, specific binary can request LA48-compatible UVA by procctl(2) or by the flag in the FreeBSD features ELF note.

    Since I do not have access to a machine with LA57, development was done using QEMU. It would be interesting to try it on the real hardware.

    Sponsor: The FreeBSD Foundation

    Not-transparent superpages

    Links
    Patch URL: https://reviews.freebsd.org/D24652

    Contact: Konstantin Belousov <[email protected]>

    FreeBSD already provides excellent support for superpages, in a manner completely transparent to applications. It tries to proactively prevent fragmentation, reserves contiguous runs of the physical pages for linear allocations in managed objects, and auto-promote runs of small pages when they form complete superpage.

    The shortcomings of this approach directly follows from its strength: some applications want to get guaranteed superpage mappings, typically because the underlying physical memory is also offloaded into a hardware which also has memory mapping unit. For instance, Infiniband RMDA adapters do memory registration and remapping, which is more efficient with large pages. In such cases transparent (non-guaranteed) support cannot be used.

    The extension was developed for POSIX shared memory subsystem to allow the creator request that the shared memory object was backed by physically contiguous pages, with runs of specified size. The mmap(2) syscall is aware of such objects, and if the requested mapping is properly aligned, it will be served by superpages.

    The new type of the shared memory objects are backed by a modified physical pager, which only allocates contiguous physical memory. The VM ensures that mappings of the objects are never split (clipped) on a non-superpage boundary. The fault handler is specially optimized to be very fast by quickly installing the superpage PTE, and to avoid touching all small pages constituing it.

    Currently the required pmap support is provided for amd64 with 2M and 1G superpage sizes.

    Sponsor: The FreeBSD Foundation

    NXP ARM64 SoC support

    Contact: Marcin Wojtas <[email protected]>
    Contact: Artur Rojek <[email protected]>
    Contact: Dawid Gorecki <[email protected]>

    The Semihalf team initiated working on FreeBSD support for the NXP LS1046A SoC

    LS1046A are quad-core 64-bit ARMv8 Cortex-A72 processors with integrated packet processing acceleration and high speed peripherals including 10 Gb Ethernet, PCIe 3.0, SATA 3.0 and USB 3.0 for a wide range of networking, storage, security and industrial applications.

    Completed since the last update:

    • Improve code in a couple of review cycles and merge following new features to the FreeBSD-HEAD (r361458 - r361464):

      • QorIQ platform clockgen driver

      • LS1046A clockgen driver

      • GPIO support for QorIQ boards

      • QorIQ LS10xx AHCI driver

      • VF610 I2C controller support

      • TCA6416 GPIO expander

      • Epson RX-8803 RTC

    Todo:

    • Upstreaming of the QorIQ SDHCI driver - it is expected to be submitted/merged to HEAD in the Q3 of 2020.

    Sponsor: Alstom Group

    amd64 pmap Fine-grained pv lists locking

    Links
    Patch URL: https://reviews.freebsd.org/D24217

    Contact: Konstantin Belousov <[email protected]>

    FreeBSD kernel Virtual Memory subsystem handles 'normal' application memory, i.e. anonymous or file-backed shared and private mappings, with so called managed pages. Managed page is fully controlled by VM, which tracks it status. In particular, managed page can be made read-only for write-back to the file, or unmapped for reuse (paging).

    The machine-dependent VM layer, pmap, must support managed pages, for instance it must provide operations such as pmap_remove_write() to downgrade all mappings to read-only, or pmap_remove_all() to unmap the page from all address spaces. To implement this kind of operations, while not causing the overhead of scanning all page tables, pmap must track existing mappings of the page. The tracking is done by allocating a small data structure 'pv entry' per mapping, and linking all pv entries for the given page into pv list.

    Since pv entries come from context of different address spaces, pmap must provide synchronization to guarantee correctness of the list structures. Current pmap allocates one mutex per one 2M physical superpage in NUMA configurations, and MAXCPU == 256 locks hashed by the page physical address for non-NUMA. The end result is often undeserved lock aliasing causing pv list locks contention, since all 4k pages in the 2M superpage share the same lock, and reservations typically cause adjusted pages to come from the same superpage.

    The proposed patch creates a new kernel synchronization primitive called one byte mutex, which is embedded into the currently unused padding in machine-dependent portion of the struct vm_page. This way each page gets dedicated pv list lock without using any more memory. In the ever-important buildkernel benchmark on non-NUMA config, this change provides 2x reduction of the system time.

    One complication is that old locking distribution scheme made a natural fit for superpages promotion and demotion, since all embedded small pages shared the same pv list lock, and the operations basically fold/unfold corresponding pv entries. Now the promotion and demotion operations require taking all locks for constituent small pages, which provides small but measurable impact on them. It is possible to optimize it further by providing the 'superlock' on the first page from the superpage run, but the affected operations are relatively rare so that it is not even obvious that implementing the optimization would not slow down other pathes.

    Another important nuance of the pv entries handling is that sometimes pv entries allocator must not fail. Typically this is required when kernel makes a call to pmap_enter() which must establish new mapping, and for managed page this includes allocating the pv entry if existing cannot be reused. If allocator cannot get a fresh page from the vm_page_alloc(9), it opts to destroy some other managed mapping to either get a reusable pv entry from current pmap, or destroy enough managed mappings from some other pmap to free whole page.

    To do the reclamation, currently all pages from which with pv entries are allocated, are linked in the global pv chunk list, which is protected by global (per-NUMA domain) mutex. Any allocation or free of pv entry has to lock the mutex, which is apparently a contention point for large machines.

    Patch removes the global list of chunks, instead linking all pmaps in the global list like it is done on i386 (but for different reason). Now the global lock is only taken for pmap creation and free, which corresponds to fork/exec and exit of a process, and when pv allocator starts reclaiming from other pmaps (which is normally does not happen).

    Sponsor: The FreeBSD Foundation

    Lockless routing lookups and scalable multipath

    Links
    Implementation of scalable multipath URL: https://reviews.freebsd.org/D24141#change-ZOjdMqgDgUr7

    Contact: Alexander Chernikov <[email protected]>

    The primary goal of this work is to bring scalable routing multipath implementation, enabled by default. Another goal is enabling high-performance routing lookups.

    Multipath will close long-standing feature gap that modern networking OS must have. Lockless routing lookups will remove lookup bottlenecks, improve both dataplane and control plane performance for the setups with large number of routes.

    Background

    The initial routing kpi was introduced back in 1980. It was a nice generic approach back then, as no one knew how the protocols would evolve. It has been enormously successful as it was able to survive for 20+ years.

    Unfortunately, this kpi does not try to protect subsystem internals from the outside users, resulting in tight coupling with other subsystems. As a result, making changes is hard, leading to compromises and piling technical debt.

    Implementation overview

    Most changes are based on introduction of the concept of nexthops. Nexthops are separate datastructures, containing all necessary information to perform packet forwarding such as gateway, interface and mtu. They are shared among the routes, providing more pre-computed cache-efficient data while requiring less memory. Interested reader can find more detailed description in D24141. Another overview can be found in Nexthop object talk describing Linux implementation.

    Multipath implementation extends nexthop concept further by introducing nexthop groups.

    Each route has a pointer to either nexthops or a nexthop group, decoupling lookup algorithm from the routing stack internals. Both nexthops and nexthop groups are immutable and use epoch(9)-backed reclamation.

    A pre-requisite for lockless routing lookup is the introduction of modular lookup framework, allowing to attach any longest-prefix-match algorithm implementation to any IPv4/IPv6 fib.

    Currently there are plans to use modified DIR-24-8 algorithm from DPDK for both IPv4 and IPv6 families as an example of base lockless implementation.

    Status

    • Nexthop objects [ DONE ]

      • Introduction of nexthop objects [ DONE ]

      • Conversion of old KPI users to the new one [ DONE ]

      • Conversion of route caching to nexthop caching [ DONE ]

      • Conversion of struct rtentry field access to nhop field access [ DONE ]

      • Eliminating old lookup KPI and hiding struct rtentry [ DONE ]

    • Multipath [ IN PROGRESS ]

      • Switch control plane consumers to use (rtentry, nhop) pairs instead of rtentry to allow multipath changes happen transparently [ 90% DONE ]

      • Introduce nexthop group objects

      • Add mutipath support for the rib (routing information base) manipulation functions

      • Add flowid generation for outbound traffic to enable load balancing

    • Modular longest-prefix-match lookup algorithm [ IN PROGRESS ]

      • Design control plane framework for attaching algorithms [ 90% DONE ]

      • Port IPv6 lockless lookup algorithm [ DONE ]

      • Port IPv4 lockless lookup algorithm

    ZSTD Compression in ZFS

    Contact: Allan Jude <[email protected]>

    Zstandard (ZSTD) is a modern high-performance compression algorithm designed to provide the compression ratios of gzip while offering much better performance. ZSTD has been adopted in FreeBSD for a number of other uses, including compressing kernel crash dumps, as a replacement for gzip or bzip for compressing log files, and for future versions of pkg(8).

    This effort to complete the integration of ZSTD into ZFS is sponsored by the FreeBSD Foundation.

    Integrating ZSTD into ZFS will further extend the transparent compression feature of ZFS by offering higher compression ratios without the performance penalty associated with gzip. ZSTD offers compression levels ranging from 1 (low compression) to 22 (maximum compression), plus ZSTD-Fast levels that offer less compression but even greater speed. This will allow the storage administrator to select the performance-vs-compression tradeoff that best suits their needs.

    Tasks remaining to be completed:

    • Extend ZFS to support compression algorithms with large numbers of levels

    • Solve issues around the inheritence of compression settings

    • Restore compression level when reading blocks from disk

    • Create a future-proofing scheme to handle changing versions of ZSTD

    • Extend ZFS replication to handle backwards compatibility with pools that do not yet support ZSTD

    • Resolve issues around backwards compatibility when ZSTD is configured but not used

    Sponsor: The FreeBSD Foundation

    CheriBSD 2020 Q2

    Links
    CHERI-CPU URL: http://www.cheri-cpu.org
    DARPA FETT Bug Bounty Program URL: https://fett.darpa.mil

    Contact: Alex Richardson <[email protected]>
    Contact: Andrew Turner <[email protected]>
    Contact: Brooks Davis <[email protected]>
    Contact: Edward Tomasz Napierala <[email protected]>
    Contact: Jessica Clarke <[email protected]>
    Contact: John Baldwin <[email protected]>
    Contact: Robert Watson <[email protected]>
    Contact: Ruslan Bukin <[email protected]>

    CheriBSD extends FreeBSD to implement memory protection and software compartmentalization features supported by the CHERI instruction set extensions.

    Support for CHERI-RISC-V in CheriBSD has continued to mature this quarter in tandem with refinements to the CHERI-RISC-V architecture. We have recently made CheriBSD's "pure capability" (CheriABI) process environment the default ABI rather than a compatibility layer. It has grown support for:

    • dynamically linked binaries (previously only statically-linked binaries were supported)

    • C++ including exceptions

    • sealed return address and function pointer capabilities ("sentries") which provide additional CFI protection

    • initial MMU protections for loading and storing tags

    At this point, CHERI-RISC-V support in CheriBSD is generally on par with support for CHERI-MIPS.

    Much of this effort has been focused on preparing CheriBSD on CHERI-RISC-V for inclusion as a demonstrator system in DARPA's Finding Exploits to Thwart Tampering (FETT Bug Bounty program).

    In addition, work has begun this quarter on porting CheriBSD to Arm's Morello SoC. Morello is a prototype demonstrator board which adds CHERI extensions to ARMv8-A.

    We've recently switched to a dev-branch model where active work takes place on the dev branch and we periodically merge to master with synchronization between CheriBSD and dependencies like CHERI-LLVM.

    For those interested in what it's like to program for CHERI, we've recently released a CHERI C/C++ Programming Guide.


    Architectures

    Updating platform-specific features and bringing in support for new hardware platforms.


    Continuous Integration on !x86

    Contact: Edward Tomasz Napierala <[email protected]>

    For quite a while the FreeBSD CI infrastructure has been running FreeBSD builds and regression tests, making it easy to spot regressions. While CI was building images for all architectures, the regression tests were only run on amd64 and i386, which means they couldn't detect architecture-specific runtime breakage on non-x86 architectures. This poses a problem not only for FreeBSD itself, but also for people working on FreeBSD forks for !x86, such as the CHERI project at University of Cambridge and SRI International.

    The goal of this project is to run regression tests on the remaining architectures supported by FreeBSD: ARM, ARM64, MIPS, POWER, and RISC-V. The tests are being run using common, mostly machine-independent scripts. Those required some changes to make it possible to use QEMU in addition to Bhyve, the hypervisor used for x86 tests. The sysutils/u-boot-qemu-arm and sysutils/u-boot-qemu-arm64 ports were added to the Ports Collection. Finally, each of the architectures required some tweaks, to account for different configuration requirements - for example the MIPS kernel doesn't support VirtIO disks, or even AHCI, whereas on ARM64 the U-Boot gets confused with more than one VirtIO disk.

    On ARM, we're currently at 52 failures and 590 skipped tests, of 5925 tests ran (https://ci.freebsd.org/job/FreeBSD-head-armv7-test/lastCompletedBuild/testReport/). On ARM64 it's 19 failures and 160 skipped (https://ci.freebsd.org/job/FreeBSD-head-aarch64-test/lastCompletedBuild/testReport/). On MIPS it's 172 failures and 734 skipped (https://ci.freebsd.org/job/FreeBSD-head-mips64-test/lastCompletedBuild/testReport/). For POWER, and RISC-V the results are not available yet.

    Remaining work:

    • Failing regression tests need to be fixed.

    • The tests are quite slow on QEMU, for example the ARM64 run takes about five hours. Running them automatically after each commit would quickly overload the CI cluster. A solution would be to e.g. run them daily.

    • Some of the jobs still fail to produce results: powerpc64 deadlocks at the end of regression test suite due to an unkillable process, riscv64 panics randomly, and on mips64 kyua(1) often crashes on jemalloc assertion. Those might be fixed by an upcoming QEMU port update.

    Sponsor: DARPA

    FreeBSD/RISC-V Project

    Links
    Wiki URL: https://wiki.freebsd.org/riscv

    Contact: Mitchell Horne <[email protected]>

    Contact: freebsd-riscv Mailing List
    Contact: IRC #freebsd-riscv channel on freenode

    The FreeBSD/RISC-V project is providing support for running FreeBSD on the RISC-V Instruction Set Architecture. Since RISC-V is still a young and evolving platform, one of our goals is to have FreeBSD be a well-supported option for users as RISC-V hardware increases in availability.

    This quarter saw a number of improvements to the boot process.

    The physmem interface used by arm and arm64 to enumerate physical memory resources was moved to machine-independent code and adopted on RISC-V. As a result, the kernel is now able to detect and exclude physical memory reserved by devices or firmware. A bug that prevented the kernel from using physical memory below its load address was fixed. This typically did not manifest in much waste, as the kernel is loaded 2MB after the start of physical memory by default. In future boot configurations, the impact would have been much larger.

    Our port for OpenSBI was updated to v0.8, bringing several new features and fixes. In particular, it brought the Hardware State Management (HSM) extension, which can be used to start and stop CPUs. FreeBSD will now use this extension whenever it detects that it is available.

    There has also been a lot of work done to port FreeBSD's standard bootloader, loader(8), to RISC-V. This has big advantages in terms of boot flexibility, and brings us closer to what's needed to produce official FreeBSD/RISC-V release images. By leveraging UEFI support from u-boot, loader.efi can be used in a manner similar to arm and arm64. Next quarter will likely bring u-boot ports for RISC-V targets, pending the v2020.07 release. Booting the kernel directly via SBI firmware will continue to be supported.


    Userland Programs

    Changes affecting the base system and programs in it.


    Import of new implementation of bc and dc

    Links
    Official repository URL: https://git.yzena.com/gavin/bc
    Repository mirror on GitHub URL: https://github.com/gavinhoward/bc/

    Contact: Stefan Esser <[email protected]>
    Contact: Gavin D. Howard <[email protected]>

    A new version of bc and dc has been imported into FreeBSD-CURRENT and enabled by default. An import into 12-STABLE is planned before the end of July, but it will not be enabled by default (will require "WITH_GH_BC=yes" to be set in /etc/src.conf).

    This version has been developed by Gavin D. Howard with the goal to provide a highly portable and POSIX compatible implementation. It offers GNU bc compatibility and should be a drop-in replacement for the bc in FreeBSD, except for standard-violating behavior of the bc currently in FreeBSD (e.g., the modulo operator).

    Additional features:

    • High performance (up to more than a factor of 100 faster than the current FreeBSD implementation in some tests)

    • support of message catalogs with a large number of languages supported in the current release (contributions of further translations are welcome).

    • Extra built-in functions and operators.

    • Extended library of advanced math functions

    • Detailed man-page explaining standard conformant and extended features

    • One shared binary for bc and dc (bc is not just a pre-processor that relies on dc for the actual computations)

    The only dc feature not supported by the dc is the execution of sub-processes, since the author considers it a security hazard for a calculator to have.

    This code is also available as a port and package (math/gh-bc or gh-bc), e.g. for use with a FreeBSD binary release.

    Binutils Retirement

    Links
    GPL in Base wiki page URL: https://wiki.freebsd.org/GPLinBase

    Contact: Ed Maste <[email protected]>

    We have been working on migrating to a modern and copyfree or permissively licensed toolchain for quite some time. In the last quarter we retired two obsolete GNU bintuils: objdump, and as.

    Many uses of objdump can be replaced with readelf, and llvm-objdump is also available in the base system. Ports that depend on objdump have been updated to rely on the GNU binutils port or package.

    The GNU as utility was used by both the base system and by ports. As with objdump, ports that require GNU as have generally been updated to depend on binutils. One file in the base system (skein_block_asm.s) proved troublesome during earlier attempts to migrate to using Clang's integrated assembler (IAS). However, after the update to Clang 10 (and with some trivial modifications to the source) IAS can assemble the file.

    Both GNU as and objdump have been removed from FreeBSD-CURRENT and will be absent from FreeBSD 13.0.

    TODO

    The final task in the binutils retirement project is to remove GNU GDB 6.1. It is currently retained for crashinfo(8).

    Sponsor: The FreeBSD Foundation

    Run-Time Dynamic Linker improvements

    Contact: Konstantin Belousov <[email protected]>

    Rtld gets some number of small bug fixes and improvements.

    RTLD_DEEPBIND dlopen(3) flag was implemented, despite being a strange and even unsafe idea, for compatibility with glibc.

    Several improvements to the direct execution mode were made. Most interesting are perhaps the '-v' switch to report some configuration parameters for rtld, the ability to specify argv0 different from the executed binary name, and fixes to properly set osrel/ABI for the directly executed binary.

    The link_map structure that is used by tools that need to know the list of loaded shared objects (like gdb and wine) was made more compatible with glibc, while keeping existing FreeBSD ABI intact.

    In the course of the link_map work, it become apparent that rtld sometimes needs to report presence of features that cannot be deduced by just a runtime test for symbol presence or for function behavior. For that, a scheme of reporting features with uniformingly named symbols was designed - see the rtld(1) man-page (in CURRENT) for an explanation.

    Position-independent (PIE) binaries on FreeBSD are now marked with the DF_1_PIE DT_FLAG1 flag. Otherwise, such binaries are just ET_DYN objects and it is quite hard to distinguish proper dynamically shared object (DSO) from PIE binary. The problem is that for binaries, the static linker strips some information which is required for proper loading as a DSO, and additonally, binaries contains relocations like copy-relocations that cannot be handled for non-main binaries at all.

    With the flag addition, rtld properly detects binaries and refuses to load them with dlopen() or as DT_NEEDED dependency. ldd(1) also misdetected PIE vs. DSO, and required a fix to parse dynamic segments to not try to dlopen() them.

    Sponsor: The FreeBSD Foundation

    VHDX support in mkimg(1)

    Contact: Oleksandr Tymoshenko <[email protected]>

    VHDX is the successor of Microsoft's VHD virtual drive file format. It increases maximum capacity of the virtual drive to 64TB and introduces features to better handle power/system failures.
    VHDX is the required format for 2nd generation Hyper-V VMs.


    Ports

    Changes affecting the Ports Collection, whether sweeping changes that touch most of the tree, or individual ports themselves.


    Bastille

    Links
    Bastille GitHub URL: https://github.com/bastillebsd/bastille
    Bastille Templates URL: https://gitlab.com/bastillebsd-templates
    Bastille Website URL: https://bastillebsd.org

    Contact: Christer Edwards <[email protected]>

    Bastille is an open-source system for automating deployment and management of containerized applications on FreeBSD.

    Bastille Templates automate container setup allowing you to easily reproduce containers as needed.

    Bastille is available in ports at sysutils/bastille.

    Q2 2020 Status

    In Q2 2020 Bastille merged some exciting new features into GitHub. Changes include:

    • experimental support for new Bastillefile template syntax

    • added mount and umount sub-commands

    • added a default Vagrantfile for simple testing

    • experimental support for empty containers

    • improvements to VNET DHCP support

    • cosmetic bugfixes in error output

    • extended config file documentation

    • updated bastille help output

    • option to (-f) force destroy container

    sysutils/bastille was updated to 0.6.20200414 (latest).

    New Bastille templates added this quarter include:

    • Percona database server

    • Asterisk SIP server

    • dnsmasq DNS/DHCP server (VNET required)

    • nginx pkg server for poudriere

    Everything mentioned here was done under COVID-19 quarantine. Special thanks to everyone that contributed during this time.

    KDE on FreeBSD

    Links
    KDE FreeBSD URL: https://freebsd.kde.org/
    KDE Community FreeBSD URL: https://community.kde.org/FreeBSD

    Contact: Adriaan de Groot <[email protected]>

    The KDE on FreeBSD project packages the software produced by the KDE Community for FreeBSD. The software includes a full desktop environment KDE Plasma, IDE KDevelop, a PIM suite Kontact and hundreds of other applications that can be used on any FreeBSD desktop machine.

    This quarter has been an ever-so-peculiar one. While we are used to working remotely, collaborating over the internet to update the ports tree, it's qualitatively different when the whole world locks down. Meanwhile, software continues to be released, so this quarter the kde@ team:

    • Restored a patch that closes down a remote TCP held by X11 applications that use the ICE library. Thanks to Colin Percival for reporting it. It went missing in one of the port updates. To prevent this in the future, the patch has been upstreamed. PR 229772.

    • Chased KDE-adjacent software like CMake, Cutelyst, Latte-dock and Nheko through new releases. In particular CMake takes a lot of effort every time because it is a build-time dependency of over 2000 ports.

    • graphics/poppler was updated to the latest upstream release. This is a low-level dependency for many document-viewing applications, and like CMake requires chasing a lot of other software. Poppler is one of the components shared between various software stacks (and "desktop environments") under the desktop@ group, in which kde@ participates.

    • KDE Frameworks release like clockwork, reaching KDE Frameworks 5.70 mid-may.

    • KDE Applications -- the KDE release service, really, which delivers libraries, applications, and add-ons -- had one large release, with 20.04.1 landing in the ports tree also mid-may and its monthly update 20.04.2 in mid-june.

    • Some new Wayland support for KDE Plasma -- we have not tested this on FreeBSD -- has appeared and has been duly packaged.

    • A great deal of preparation has gone into Qt 5.15. Many ports have been pre-emptively patched for this new -- and last -- LTS release of Qt 5. The update itself has not yet landed, pending a few last bits of fallout.

    The kde@ team would like to thank Antoine for many exp-runs, mikael@ for useful tips, swills@ for patience and kai@ for dealing with WebEngine.

    The next big round of updates for the KDE stack is slated: CMake 3.18, Qt 5.15 LTS, and KDE Frameworks 5.71.

    Haskell on FreeBSD

    Links
    Haskell language homepage URL: http://www.haskell.org/
    Ports development repo URL: https://github.com/freebsd/freebsd-ports-haskell

    Contact: Gleb Popov <[email protected]>

    Haskell is a general-purpose strictly-typed pure functional language. The Haskell on FreeBSD projects strives to provide the up-to-date Haskell toolchain as well as various application written in this language.

    This quarter brought the long-awaited GHC update, which is now at version 8.8.3. Along the compiler, the Haskell build system frontend, cabal-install, was also upgraded to 3.0.2.0. During this update, numerous Haskell ports were updated too.

    All existing ports of Haskell applications were migrated to USES=cabal, which implements Go-style build proccess - all dependencies are compiled as part of the build. As a consequence, ports for Haskell libraries have been deprecated and removed.

    Upgrading GHC became a tedious task for a single person, so a new GitHub repository was created under the FreeBSD organization - freebsd-ports-haskell. Right now, work is being done on preparing another GHC upgrade in the ghc-upgrade-810 branch. Any contributions are welcome.

    rtsx - Porting driver for Realtek SD card reader from OpenBSD

    Links
    rtsx URL: https://github.com/hlh-restart/rtsx
    PR204521 URL: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=204521

    Contact: Henri Hennebert <[email protected]>

    The rtsx driver for Realtek SD card reader has been ported from OpenBSD. Its development snapshot is available via the sysutils/rtsx-kmod port.

    From March to May 2020, the code has been completed with the help of Gary Jennejohn (gj@) and Jesper Schmitz Mouridsen (jsm@). Some tweaks have been imported from the Linux counterpart.

    The driver has been successfully tested with:

    • RTS5209 under head (Lenovo ThinkPad L520)

    • RTS5227 under stable/11 and releng/12.1 (HP ProBook 430 g2, Lenovo ThinkPad T450/T450s)

    • RTS5229 under releng/12.1 (Lenovo IdeaPad 120S-14IAP)

    • RTS522A under releng/12.1 and head (Intel NUC8i5BE, Lenovo ThinkPad P50s)

    • RTS525A under releng/12.1 (Dell Latitude E5570)

    • RTL8411B under stable/12 (Acer Aspire E 15 E5-576-77W6)

    The driver should also work for Realtek RTS5249, RTL8402 and RTL8411.

    More tests are welcome, especially for the devices not yet tested. These devices may require more tweaks.

    PR204521 contains the bulk of exchanges for completion of the code.

    Valgrind updates

    Links
    Patch for valgrind URL: https://reviews.freebsd.org/D25452

    Contact: Paul Floyd <[email protected]>
    Contact: Kyle Evans <[email protected]>

    A large amount of work has been done to rebase FreeBSD support on top of Valgrind 3.17.0, and to address numerous test suite failures. Currently, almost all of the regression tests pass on amd64. This is a major improvement over the current state of affairs, in which the Valgrind is quite out of date and is missing important functionality. Some follow-up work aims to make FreeBSD an officially supported target platform for Valgrind.

    The devel/valgrind-devel port is in the process of being updated to point at the new work.


    Documentation

    Noteworthy changes in the documentation tree, in manpages, or in external books/documents.


    FreeBSD Translations on Weblate

    Links
    Translate FreeBSD on Weblate wiki URL: https://wiki.freebsd.org/DocTranslationOnWeblate
    FreeBSD Weblate Instance URL: https://translate-dev.freebsd.org/

    Contact: Danilo G. Baio <[email protected]>
    Contact: Edson Brandi <[email protected]>

    This quarter was improved the renderization of RTL (Right-to-left) languages on the FreeBSD Documentation. We faced this issue after the first RTL language joined the translations effort (fa_IR).

    We are looking forward to receive more languages and translators to the project as well.

    Q2 2020 Status

    • 10 languages (No new languages)

    • 80 registered users (33 new users since last quarter)

    Languages

    • Chinese (Simplified) (zh_CN)

    • Chinese (Traditional) (zh_TW)

    • French (fr_FR)

    • German (de_DE)

    • Italian (it_IT)

    • Norwegian (nb_NO)

    • Persian (fa_IR)

    • Portuguese (pt_BR)

    • Spanish (es_ES)

    • Turkish (tr-TR)

    We want to thank everyone that contributed, translating or reviewing documents. And please, help promote this effort on your local user group, we always need more volunteers.


    Miscellaneous

    Objects that defy categorization.


    FreshPorts

    Links
    FreshPorts URL: http://freshports.org/
    FreshPorts blog URL: http://news.freshports.org/

    Contact: Dan Langille <[email protected]>

    FreshPorts, and its sister site, FreshSource, have reported upon FreeBSD commits for 20 years. They cover all commits, not just ports.

    FreshPorts tracks the commits and extracts data from the port Makefiles to create a database of information useful to both port developers and port users.

    For example, https://www.freshports.org/security/acme.sh/ shows the history of this port, back to its creation in May 2017.

    git

    Work on git started back in September. It was ignored for a while and started back in mid-June with the creation of new git-specific jails for commit ingress (commit processing gitdev) and for the website.

    Serhii (Sergey) Kozlov created a script to transform GIT commit entries into XML digestible by FreshPorts. This was a huge step foward for the effort.

    The next step include:

    • incorporate a that script the automated processes of FreshPorts

    • migrate to new test & stage versions of FreshPorts

    • test

    • get ready for prod

    Help wanted

    git is not far away now. I could use helpers to

    • review code

    • watch the commits on the devgit websites

    • catch missing stuff

    Thank you

    Packages

    FreshPorts now displays the packages version available from the repo sources. This covers all primary tiers (e.g. FreeBSD:12:amd64) and all secondary tiers (e.g. FreeBSD:13:powerpc64). This helps uses know what versions they can expect and when then repo was last built.

    Dependency lines

    Some things are easiest done via copy/paste. If you are working on a port Makefile and need to add a new dependency, FreshPorts shows the dependency line for that port. For example: 

    
acme.sh>0:security/acme.sh

    Libraries are also covered by this feature.

    Python ports were recently adjusted to display 

    
 ${PYTHON_PKGNAMEPREFIX}virtualenv>0:devel/py-virtualenv

    instead of 

    
py37-virtualenv>0:devel/py-virtualenv

    You can read more about this change in [issue #73](https://github.com/FreshPorts/freshports/issues/73).

    Watch ports I maintain

    The search page has long had the "Ports I Maintain" button (if you are logged in). This feature recently branched out to a new automated watch list option: Watch ports I maintain.

    This report subscription will notify you of any commits to the ports you maintain. Your email address on FreshPorts must match the value in the MAINTAINER field of the port. This is always a daily report.

    From time to time, an infrastructure change will occur which touches your port. This feature ensures you know about that change.

    Repology links

    Repology links were requested. This allows you to see what versions of that port are in the repositories of other systems. A link to repology.org appears on every port page.

    Further reading

    Based upon things you didn’t know FreshPorts can do

    There are many things FreshPorts can do, including search Makefile's and pkg-plist. This is from a recent blog post:

    • provides example dependency line. e.g. p5-XML-RSS>0:textproc/p5-XML-RSS

    • list of dependencies for a port

    • list of ports depending upon this port

    • see default configuration options

    • what packages install a given file (e.g. bin/unzip)

    • find out what ports a person maintains

    • find Makefiles which contain references to bunzip

    • search results can be plain-text consisting of a list of foo/bar ports

    • the Maximum Effort checkbox on the search page does nothing

    • committers can be notified of sanity test failures after the commit

    • find a commit, any commit, based on SVN revision number, e.g. : https://www.freshports.org/commit.php?revision=352332

    Javascript help wanted

    We recently upgraded some outdated Javascript modules. This broke our [JavaScript based graphs](https://www.freshports.org/graphs2.php). We could use some help on fixing that please. The starting points are listed on that URL. If you need a working website to play with, please contact me with a ssh public-key.

    Sponsor: hardware provided by iXsystems

    PCI passthrough with bhyve on Intel and for OpenBSD guests

    Links
    bhyve Intel bug report URL: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=229852
    bhyve OpenBSD bug report URL: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=245392
    PCI passthrough with bhyve (FreeBSD wiki article) URL: https://wiki.freebsd.org/bhyve/pci_passthru

    Contact: Anatoli <[email protected]>
    Contact: Callum <[email protected]>
    Contact: Peter Grehan <[email protected]>

    bhyve(8) is a hypervisor that supports running a variety of guest operating systems in virtual machines. bhyve(8) includes support for PCI devices passthru, a technique to pass host PCI devices to a virtual machine for its exclusive control and use.

    For some years, PCI passthrough (ppt) in bhyve was not working on some Intel systems and for OpenBSD guests due to two bugs. The first one was crashing FreeBSD host when bhyve was started with ppt on Intel processors with two VT-d translation units (IOMMU), included in most Skylake and newer Intel processors.

    The second bug was preventing correct interrupts handling for OpenBSD guests. As a result, OpenBSD guests running on bhyve were not able to use any PCI devices passed through to them from the host.

    During the last 2 months the second bug was identified and fixed and they both were backported to 12.1-RELEASE (p7). So now it's possible to fully take advantage of PCI passthrough (ppt) with bhyve in a production-ready RELEASE version.

    The most typical case for ppt is to pass to the guest network adapters for its complete control, but you can also pass through USB devices (including external HDDs). Note though, passthrough of VGA and GPU devices is not supported yet (for more details see the 3rd link).

    A particularly interesting case for ppt is to use OpenBSD guest as a firewall and a router for a FreeBSD server.

    With ppt you can achieve this all inside a single server. You could pass to the OpenBSD guest a network adapter connected to the internet and it would take a complete control of it. After filtering the traffic, it could pass good packets via virtual network interfaces to other guests or to the host.

    Once a network adapter is passed through, a FreeBSD host not only doesn't see it and hence doesn't handle the network traffic, it doesn't even have to initialize the adapter (e.g. in case of a WiFi card, it's the guest that loads the firmware).

    In simple terms the host only passes the device interrupts to the guest as they come from the hardware. Everything related to the device management happens inside the guest so there's no danger that some network traffic exploits some issue in the host's network stack and causes the host to crash or misbehave in other ways.

    SageMath

    Links
    SageMath site URL: https://www.sagemath.org/

    Contact: Thierry Thomas <[email protected]>

    SageMath is a free open-source mathematics software system licensed under the GPL. It builds on top of many existing open-source packages: NumPy, SciPy, matplotlib, Sympy, Maxima, GAP, FLINT, R and many more. Thanks to SageMath it is possible to access their combined power through a common, Python-based language or directly via interfaces or wrappers.

    The goal is creating a viable free open source alternative to Magma, Maple, Mathematica and Matlab.

    This is a complex port, with a lot of dependencies, and it has been broken for some time. Upstream is working on easing its packaging, and many previously bundled applications can now be replaced by external packages.

    If you are interested, it would be nice to create a team of maintainers

    • to maintain some of the dependencies;

    • to maintain SageMath itself and prepare the next release (9.2 is coming!).


    Third-Party Projects

    Many projects build upon FreeBSD or incorporate components of FreeBSD into their project. As these projects may be of interest to the broader FreeBSD community, we sometimes include brief updates submitted by these projects in our quarterly report. The FreeBSD project makes no representation as to the accuracy or veracity of any claims in these submissions.


    chaifi - a tool to simplify joining public WiFi networks

    Links
    chaifi URL: https://github.com/gonzoua/chaifi

    Contact: Oleksandr Tymoshenko <[email protected]>

    chaifi is a TUI (text UI) utility aimed at simplifying the process of joining public WiFi networks in places like coffee shops or libraries. It replaces the process of scanning and manually editing wpa_supplicant.conf with an interactive dialog. The utility is in no way a replacement for full-featured network managers in major desktop environments. Still, if you're working from a console or using a tiling window manager, it may save you some seconds (or in worst case minutes) of your time.

    MixerTUI

    Links
    mixertui URL: https://gitlab.com/alfix/mixertui

    Contact: Alfonso Sabato Siciliano <[email protected]>

    MixerTUI is a volume mixer with a Terminal User Interface built on the FreeBSD sound system. It can show the current Sound Driver configuration and select an audio device: to get its information, to change the volume or to set it as default, the last feature allows to switch easily audio from/to laptop and hdmi, headphones and speakers, and so on.
    MixerTUI can be installed via the audio/mixertui port.

    I would like to thank the FreeBSD community for the tips, feedbacks and patches to improve this project.

    Potluck - Flavour & Image Repository for pot

    Links
    Potluck Repository & Project URL: https://potluck.honeyguide.net/
    Potluck on github URL: https://github.com/hny-gd/potluck
    pot project URL: https://pot.pizzamig.dev

    Contact: Stephan Lichtenauer <[email protected]>

    pot is a jail management tool that also supports orchestration through nomad.

    Potluck aims to be to FreeBSD and pot what Dockerhub is to Linux and Docker: A repository of pot flavours and complete images for usage with pot.

    This should simplify setting up complex software with many packages and ports in comparison to manual configuration: Potluck aims to provide a content library as an additional layer of abstraction, on top of existing infrastructure like pkg, that pot has to offer.

    Pot "flavour" files are provided on [Github]((https://github.com/hny-gd/potluck)) and fed into a Jenkins instance. On the Potluck Repository, for each flavour, detailed descriptions as well as ready-made images to be imported by pot are provided.

    The initial project has been set up, and three simple flavours, along with a complete Jitsi Meet instance in a jail has been created as a Proof of Concept that should allow running a fully-fledged video conference system with just a few easy commands within a few minutes.

    As only the initial versions have been set up and implemented so far, general feedback, tests, as well as additional, useful flavours are very welcome!

    News Home | Status Home