Meta(
    title: "Installing FreeBSD in 2023",
    summary: Some("Revisiting my favorite BSD flavor on real hardware again. Some things are definitely better, while there are obvious signs of trouble elsewhere."),
    published: Some(Time(
        year: 2023,
        month: 6,
        day: 15,
        hour: 6,
        minute: 29,
        second: 22,
    )),
    tags: [
        "freebsd",
        "bsd",
        "zfs",
    ],
)
---
I have a bit of a history with FreeBSD. While my first major encounter with open source happened to be Linux, I ran FreeBSD as my daily driver from around 2007 to 2013. My first installation was in about 2005/2006 on a spare desktop system based around an Athlon K6 with 256Mb of ram, a system which had largely been pieced together from discards and literal garbage picking. It's crazy to think that I was compiling pretty much everything from source using ports on such a crap machine, but I learned a ton and fell in love with "real" Unix in a way that Linux just hadn't really managed to captivate me up to that point.

One of the things I loved about that first install was the text terminal screensaver, Beastie in ascii art form bouncing around the screen. That was just the tip of the iceberg of true geekery that drew me in. I mean, a text mode screensaver? How cool is that? But there was also something magical about a system that could rebuild itself from source with just a couple of commands and never seemed to fail, which totally flew in the face of the experience I had seen compiling from source on Linux. This was my introduction to the concept of a truly self hosting system that could bootstrap itself from an earlier version to the latest code, using only the tiny core system that it shipped with.

I obviously see that crappy box and primitive OS with some rosy tinted glasses. There were definitely papercuts along the way. Once I made it my daily driver on my newer hardware I began to learn what some of the drawbacks were to using a system that was considered too geeky for the geeks. Even among fellow nerds my system stood apart, and in short order I learned the sad truth that increasingly over time the software being written for Unix was in fact actually being written for Linux, with BSD of any kind being an afterthought. Sure, there were heroes working behind the scenes to try and keep things patched together as fast as the Linux oriented ubergeeks could break them, but it's kind of a slow battle against entropy. Over time concessions are made and a little functionality is given up here or there. Devd might have existed way before Hal or Udev, but the major desktop projects never noticed and just plumbed their code into whatever interface Linux provided, however half baked it might have been at the time it was released. I got to see Linux abandon OSS in favor of Alsa, then build Pulse on top of Alsa. Now, when Pulse is finally working pretty much trouble free, they're ditching it in favor of Pipewire. Funny. OSS still works just fine on FreeBSD. The ports guys had to deal with Pulse, because so many programs got written expecting it to be there, and they made it work with OSS because hey, nobody else was going to do it. I assume they'll try to do the same with Pipewire, too.

Anyway, I've been mostly on Linux for the past ten years. I gave up originally somewhat after Gnome3 came out, which was obviously not going to be running on BSD any time soon, and then Xfce stopped supporting automatic handling of removable media anywhere other than Linux because they didn't have the manpower to handle supporting both Udev and Devd. I've missed FreeBSD since then, and usually have either a home server tucked away somewhere or at the very least a VM to keep pace with the goings on. But something happened over on Mastodon the other day that got me kinda pissed, and motivated me to install FreeBSD on real hardware again.

## Gnome, Workbench, Flatpak and the `sheer fucking hubris`
The developer of Workbench, a Gnome tool for designing Gtk+ interfaces in a graphical fashion (sort of like Glade used to be) has decided to add code to his program which checks whether you are running from the flatpak or not, and shuts the program down if you're attempting to run it from a distribution package or really any way other than the official flatpak. Ok, fine. That's stupid, but I don't use the program anyway.

He then took to Mastodon, without prompting, and announced what he had done and again, without prompting, proclaimed that he knows what's best for his users because his program allows amateurs to run arbitrary code on their computer. Wow. What. A. Dick. What a condescending and insulting way to treat the intended audience for your software. Fuck that guy.

Apart from what his comments say about how he views his users, lets examine the consequences of limiting the application to only run as a flatpak. Flatpak is a Linux centric technology through and through. It requires Linux cgroups as well as the Linux only Overlayfs to implement it's containerization approach. While it's quite technically possible to do similar things with FreeBSD jails and either null mounts or Unionfs, they specifically chose Linux only interfaces and have built the technology so firmly around them that in reality, the likelihood of Flatpak running anywhere other than Linux is close to nil. So this asshat is not only giving his users the finger, he's giving the finger to every other non-Linux Unix like OS in existence. Once again, fuck that guy.

### It exemplifies a larger problem
A while ago I switched from Arch to Void Linux as my daily driver. A certain amount of that decision was due to SystemD, but a much larger part of it is my longstanding belief that Glibc is probably the worst Libc implementation in existence and Void offers Musl as an alternative. I've definitely noticed, however, that programmers are if anything getting lazier about portability in their code.

Flatpak advertises that they make the developer experience better. That's not true. They hide a lot of the hard parts of being a good developer and allow you to target a stable platform without caring whether your code actually runs anywhere else. Gnome is all in on Flatpak, and it's beginning to show. I've had more crashes with Gtk4 than I've ever seen in Gtk+ up until now. If you write an app using Gtk4 and it works as a Flatpak, there is no guarantee that it's going to work on a Linux distro that uses slightly different versions of the same libraries, let alone a different Libc. Let alone a different operating system altogether. But those things should not be hidden from developers. That's how you get lazy developers. They need to actually understand the trade-offs that they are making between features and compatibility. A few extra lines of code to work around requiring an interface that only exists on one platform are worth the effort.

## So I did the installation
Anyway, that was my thinking and why I decided I needed to have FreeBSD on physical hardware again. The hardware is my Dell Inspiron from 2017 with a 7th gen i5 and 8gb of ram. It's a respectable machine even six years on, with four cores running at 3.5ghz each, but I guess 8gb is considered modest today. Funny, that Frankenbox I talked about earlier started out with 128MB and I was ecstatic when I added a second stick of memory into that POS.

The FreeBSD installer is great in that it's simple and easy to understand. I hate to have to report that it's also majorly broken in a couple of different ways.

### Wifi issues
When I got to the network configuration step in the installer and tried to configure my wireless, it returned no results for the ssid scan, no matter how many times I tried it. That's bad. It's 2023 now, and almost nobody is going to plug in to a physical port on their router to finish the installation, hoping that they will be able to fix the issue later. They're much more likely to just abort the install right there and go back to something easier and potentially more reliable. Of course once I brought the new system up I was able to get networking going with a couple lines in each of /etc/wpa_supplicant.conf and /etc/rc.conf. So the wireless stack works just fine, but just not in the installer. For reference I tried it on an HP laptop and had the exact same result. This is the sort of major bug that should hold a release until it's fixed.

### Zfs issues
I chose to install the base system on a 100GB UFS partition and put /home on a 320GB zfs pool. UFS is a much simpler filesystem with somewhat better performance in terms of memory usage, which means it's just fine for your root filesystem. For my own personal data in /home I like the idea of being able to easily back up using snapshots. To me this is a great compromise. Anyway, the installer said it created the zpool, but when I booted into the new system it dumped me to a root shell because it couldn't mount /home. Turns out that it didn't add the line to /etc/rc.conf to enable zfs and start the associated service. What the actual fuck? Ok, I fixed it, because I knew how and it was easy.

### Installers get neglected
It's a sad fact, but installers get neglected a lot when it comes to projects where the user base is largely technically oriented and are by and large good problem solvers. Arch Linux is a great example. They basically didn't really have an installer for most of Arch's existence, just a set up instructions on the Wiki, and when they did first roll out an official installer it was broken on day one. The theory is that you only install once, and then you just keep updating, so why bother with the installer? Why indeed. But that's a bad way to present your project to anyone from from the outside, that's why.

## What FreeBSD is like in 2023 compared with 'back in the day'
When I first learned FreeBSD, the `pkg` package manager didn't yet exist and you had the old `pkg_add` set of utilities descended straight from 4.4BSD. The set of binary packages that were available was small and lagged behind the ports collection pretty badly, so it was pretty common to compile from source using ports. These days those things are pretty much flipped on their head and it's uncommon to use ports unless you have a very specific need to build something with non-standard flags.

A similar situation exists in the base system itself, where back in the day I always updated by pulling the source tree down over CVS and building it from source. These days it's expected that the average user is going to use the `freebsd_update` utility instead, which is the best thing that ever came out of PCBsd and fetches binary patches to update the base system and kernel.

FreeBSD comes with two shells. The tcsh shell is the same C-shell that is available on Linux as a package and which hardly anyone uses. They always included it because /bin/sh was terrible as an interactive shell. This is the real deal descended from the original Bourne shell sh, which sounds cool and all, but they only recently added tab completion and history to turn it into something useful for day to day use, which is why the default install also had tcsh. Personally I tend to just install zsh and be done with it, which I've been doing for so long I forget when I actually made the switch. It is nice to see that sh finally got some love, though. It's actually nice to use now.

Vi is the rewrite of ex/vi that was undertaken when 4.4BSD was released because of patent BS. It's basically the same vi that you get in NetBSD and OpenBSD. It's fast and can handle files large enough to bring Vim or Neovim to their knees crying, but since I don't tend to load 50k line csv files into my editor I just install Neovim because I happen to like those extra features.

I realize I'm being snarky. Let's look at some positives for a sec.
* BSD libc is 1000% better written than Glibc
* Dtrace allows you to do a ton of debugging and figure out where things are going wrong even without debugging symbols
* The sysctl utility enables a lot of performance and security tuning even on a running kernel
* FreeBSD jails are every bit as effective as Linux cgroups for process isolation
* FreeBSD was the first OS to adopt zfs after Sun open sourced the Solaris code, and it shows in how mature and integrated the file system is on FreeBSD. You can also use it without worrying that your kernel module is going to have to be rebuilt every time the package manager updates your kernel, because it's just part of the OS.
* The documentation is complete, up to date and factually correct, unlike what you're used to in the Linux world.

I've griped about libc implementations before. When I talk about how much better BSD libc is, I don't think most people really get it. Let me give an example from a recent project of mine that to me perfectly illustrates the difference in mentality and problem solving between the Linux community at large and BSD. The /etc/passwd file is traditionally the plain text database of not only users but also their passwords. The passwords are not stored in plain text, but via an encryption generated hash using libcrypt. Somewhere along the way someone decided that this wasn't really good enough and that nobody except root should be able to see the hashes.

What happened in Linux is that we got a brand new project, the Shadow suite of tools. The idea is this. The /etc/passwd file still exists, but everyone's password hash is replaced by a * character. The actual hashes are stored in /etc/shadow, which has 0600 permissions and thus can only be read by root. The shadow project took over maintenance of the full suite of user manipulation related utilities to work with the new system, and this is what we have today (ignoring for a moment things like PAM).

In BSD, they just rewrote the libc functions which return a `pw` struct such as `getpwuid` so that they only return the hashed password if the caller is root, and then gave /etc/passwd 0600 permissions. Nothing else had to change. This is pretty much what normally happens and you can see it time and time again, where someone goes off half cocked and comes up with an entirely too complex solution to a rather simple problem and the whole community runs with it, completely ignoring the obvious and elegant solution that the BSD devs came up with.

You can see this trend pop up in other places such as networking tools. Linux distros once upon a time used the `ifconfig` command to set up almost everything about network interfaces. Along the way. it turned out that one of the kernel interfaces that ifconfig was using was deemed to be completely broken by design in Linux, and since Linus insists that you don't ever change an interface lest you break userland code, a decision was made to do a full rewrite and we got the iproute2 set of tools which includes the `ip` command that most are familiar with today. FreeBSD took a different approach over the years and just continually refined and added features to ifconfig as they were needed, such as dealing with wireless stacks, with the result that you didn't have to learn a completely new set of commands one day when your old tool got deprecated. Instead, you had to learn a new subcommand and a couple flags to use the old tool for scanning wireless ssid's. It was nice. Your old skills stayed relevant.

And really that describes a lot of the difference that you'll see if you switch from Linux to FreeBSD. You have to use it over a fairly long period of time to really appreciate it, IMO. Where Linux will forever be switching to the new and shiny, only to discard it in a year or two for the next new and shiny, FreeBSD might not release features at the same rate but when a new feature lands it 100% works on day one and continues to work even after new releases land.

## A word about the other BSD projects
I've done a fair bit of fiddling with NetBSD. I've used PkgSrc on a few different operating systems and I don't think it can be stressed enough that those guys do a lot to keep software portable between different OS's. In addition, the NetBSD work on running rump kernels to aid driver development in userspace was some really impressive stuff. I literally cheered when, immediately following Heartbleed, OpenBSD forked OpenSSL and in the first weeks deleted literally tens of thousands of lines of code as part of their cleanup. I think BTRFS and ZFS can both safely be assumed to have gotten some ideas from HammerFS, the first file system to be designed to scale into the petabyte range by the Dragonfly BSD folks. All of these projects are important, and they all share a common Unix heritage. They have all made huge contributions to the software ecosystem.

That said, there's the whole entropy thing that I mentioned above. These projects are small compared with Linux. OpenBSD, while it does have other devs, is definitely a Theo DeRaadt project and when he's gone it's hard to say what's going to become of it. NetBSD's PkgSrc still has Plasma4 while we're reading about Plasma6 development. Dragonfly was created by a guy who was incredibly brilliant but was also such an asshole to other developers that his commit bit was revoked by the FreeBSD project, and when looking at the project it's hard to see it as anything but one man's vision of where the code should have gone.

My point is this. The other BSD projects are definitely dwindling in importance. FreeBSD probably is as well, but they are at least making an attempt to stay relevant. Their source tree and ports repository are in git now (finally) while both NetBSD and OpenBSD are still using CVS. That alone is a huge blocker for attracting new developers. NetBSD ships their own modified version of Xfree86. FreeBSD is the only BSD with Wayland support, and the Hikari Wayland compositor is developed on FreeBSD. I have a huge amount of love and respect for all of these projects, but I feel like NetBSD and OpenBSD have been going about their business and largely ignoring the past couple of decades going on around them. In short, of the BSD projects I have serious doubts about their viability in, say, ten years time. I think FreeBSD will tough it out. Their ports tree is the most up to date and they have more manpower than all of the other projects combined. They aren't in as good of shape as I want them to be in, but they aren't dead and they aren't allowing themselves to become irrelevant just by willfully ignoring what everyone else is doing.
