jeang3nie/capsule
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

New post about Void

Browse source
This commit is contained in:
Nathan Fisher 2023-04-07 11:25:41 -04:00
parent 5ccfbba73f
commit dd2c9a24bc
2 changed files with 165 additions and 0 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

113
content/gemlog/daily_driving_void.gmi Normal file
View file

@ -0,0 +1,113 @@
Meta(
title: "Daily driving Void",
summary: Some("I\'ve switched from Arch to Void as my everyday OS and development platform"),
published: Some(Time(
year: 2023,
month: 4,
day: 7,
hour: 15,
minute: 25,
second: 17,
)),
tags: [
"Arch",
"Void",
"Linux",
"Distros",
],
)
---
I've had Void as a second boot option for months now, and before that I had a fairly long running VM. I've been interested in moving to Void for a while for a couple of reasons, but the biggest would be that I wanted a distro based on Musl libc.
I already made this particular jump on one machine, the Raspberry Pi 4 which runs this gemlog, my Gitea instance and an Apache server. It's been rock solid and I haven't had to do anything other than install updates since the first week I brought it up. I haven't had that kind of stability in a long time.
## Clearing some blockers
One of the things that had been holding me back for a while was that the versions of Gtk and LibAdwaita in the Void repositories were behind what I needed to compile some of my projects, namely Eva, Gfret and Vapad. I've been used to getting library updates on Arch very soon after they become available. That highlights one of the differences between the Arch and Void release models. Both are rolling, but Void is more "Rolling Stable" for lack of a better term. More on that below. Anyway, short of compiling libraries myself this one was beyond my control. In the future I'll hold off updating my code until the libraries are available on Void, which coincidentally matches up better with what a lot of the LTS distros are doing.
The other issue I was having it turns out was my own fault. It's no secret I do most of my coding these days in Rust, and it has definitely spoiled me in a lot of ways. The version of Rust in the Void repos was, up until last week, 1.64. There were a number of really cool features release in Rust 1.65 which I bought into right away. The biggest two were let..else and labeled blocks.
```
// old syntax
let thing = if let Some(t) = some_function_returning_option() {
t
} else {
return Err(NoThingError);
};
// with let..else
let Some(thing) = some_function_returning_option() else {
return Err(NoThingError);
};
```
The other feature, labeled blocks, is pretty much the same thing as Zig's named blocks, and it's a geat feature. It allows one to return early from a code block with a value, including breaking out of an outer loop in nested loops. The syntax is a little odd, but I've already used it in a couple of places.
```
let liff = 'outer: loop {
loop {
break 'outer 42;
}
}
```
Anyway, I really wanted those newer features and not having them was going to be a deal breaker for me. Ordinarily I'd be using Rust from the official installer, rustup, but I was getting weird random issues every time I tried using the toolchain provided that way. I thought it had something to do with Musl, and that the Void devs had found a workaround. Turns out it was just me not realizing that I had previously installed Rust via the distro packages and had neglected to remove the `rust-std` package. Doh.
Note that about the same time I fixed that issue, the distro packages were updated. But I'm sticking with the rustup toolchain because the distro packages are missing rustfmt and clippy, which are too nice to not use.
## Rolling release vs rolling stable
This seems like a subtle distinction, but in practice is has some noticable effects. In Arch, every package gets updated as soon after a new stable version is released as it can be packaged and tested. This includes libraries and kernels. This is rolling release.
In Void, those libraries don't get updates beyond path releases until the applications that are using them require the new versions. Void also uses the most recent LTS series kernel. This leads to less churn overall, while still having the benefit of a system that never has to be reinstalled or go through an upgrade from one OS version to another.
This leads to some interesting contradictions. On Arch, I had gtk-4.10 for months, while all of the Gnome applications that were installed only needed gtk-4.8. But when Gnome43 came out, it took almost two months for the dev team to build, test and release it. Void, on the other hand, waited until the desktop was being updated and updated all of the libraries at the same time. So when Gnome44 came out last week Void got a large influx of updates and the entire desktop was available within days. It's probably not as well tested as Arch, but considering the manpower constraints how could it be?
> Note: Fedora and Ubuntu both time their releases to occur shortly after Gnome major releases, so when the next major version of those distros ship it can take advantage of what is at the time the latest version. So the way Void works is actually pretty similar, except no need to download an iso and upgrade from one distro release to another or to change repositories.
At any rate, I really rather like this "Rolling Stable" approach. It definitely calmed things down when it came to running the OS as a server, and it seems to work out nicely on the desktop as well.
## Some interesting things I've encountered in Musl
As most OSS these days is written for Linux, and the vast majority of distros ship Glibc rather than Musl, it's only natural that some differences would be noticable, particularly if like me you tend to experiment and write a fair bit of low level code. Just this morning, I was working on bringing up an aarch64 cross compiler toolchain (as per the instructions in my post about building cross compilers) and Binutils failed to build. The offending code was in the `gprofng` module, where it was looking at the members of the `sigevent` structure individually, and expecting one to be there that does not exist in Musl. Rather than investigate and patch I just disabled gprofng via the configure script. Luckily that was a thing that I could do. This sort of thing is not always the case.
Another difference I encountered when writing some code which read usernames and validated passwords using the libc `pw` and `spw` structs. My original pass at this (using unsafe Rust) compiled and ran fine with Glibc. With Musl, it compiled and errored at runtime because some of the data I was trying to read was a null pointer (I did at least have proper error handling and null checks in the code). Anyway the reason it was happening was that Musl re-uses the same memory address for each subsequent construction of `pw`. When I looked at the POSIX spec, sure enough it said that some Libc implementations do this and it is fine. There were two possible solutions. One was to copy the bytes from that memory location into an owned piece of memory. The other method was to move all of the logic using the data in the first `pw` instance to before the creation of the second `pw` instance. The second method, requiring one less allocation, is what I used. This is a pretty nice demonstration of the sort of trouble you can get into when writing `unsafe` Rust, but because the language has trained me to account for all code paths instead of just the happy path it was a pretty easy diagnosis and fix.
That's two examples of the sort of things I've encountered in Musl, and I think they're pretty indicative of the sort of subtle differences one might encounter. That is to say, you're only going to encounter them if you're poking at the system. The average person really isn't going to notice any difference whatsoever because everything pretty much just works. A look at the relative sizes of tthe code is enlightening.
```
# Musl, according to Tokei
===============================================================================
Language Files Lines Code Comments Blanks
===============================================================================
GNU Style Assembly 312 7938 7165 256 517
Autoconf 35 6624 236 6273 115
C 1583 65083 50968 7551 6564
C Header 668 40754 35683 301 4770
Makefile 12 314 201 34 79
Shell 4 925 638 172 115
===============================================================================
Total 2614 121638 94891 14587 12160
===============================================================================
# Glibc
===============================================================================
Language Files Lines Code Comments Blanks
===============================================================================
GNU Style Assembly 2345 463470 328440 93738 41292
Autoconf 94 112969 112642 182 145
C 10186 1015110 708606 209622 96882
C Header 3341 416395 304183 79574 32638
C++ 35 2447 1660 525 262
Happy 1 387 330 0 57
JSON 2 100 100 0 0
LD Script 3 25 7 17 1
Makefile 202 21349 15600 2869 2880
Module-Definition 21 2958 2759 0 199
Perl 3 864 635 142 87
Python 68 18705 14720 2383 1602
Shell 83 8713 6102 1671 940
TeX 1 11672 7162 3697 813
Plain Text 8 63381 0 63159 222
===============================================================================
Total 16393 2138545 1502946 457579 178020
===============================================================================
```
I've compiled both Musl and Glibc numerous times while playing around with HitchHiker. Glibc takes a solid 15 minutes start to finish, while Musl is done in around a minute or less. Also, why do I want or need both Perl and Python in a Libc distribution? They might not be available (much less wanted) while bootstrapping a system from source.
The coding *style* is also enlightening. Drew Devault did a great writeup on one of his investigations into why a certain piece of code segfaulted with Glibc and not with Musl.
> Maybe, just maybe, the behavior of this function should not depend on five macros, whether or not you’re using a C++ compiler, the endianness of your machine, a look-up table, thread-local storage, and two pointer dereferences.
=> https://drewdevault.com/2020/09/25/A-story-of-two-libcs.html Drew Devault: A tale of two libcs
Anyway, even though I'm not writing much C these days it cannot be denied that C is the foundation upon which Linux (and most other modern software) is built. So for a long time I've really wanted to switch to a distro that uses what I firmly believe is a better Libc implementation. Void scratches that itch quite nicely. Bonus points because it uses Runit for Init, and I'm a big fan of service supervision suites like Runit and S6.

52
content/gemlog/hitchhiker_linux_an_origin_story.gmi Normal file
View file

@ -0,0 +1,52 @@
Meta(
title: "Hitchhiker Linux: an origin story",
summary: Some("I once accidentally built a Linux distro. A few years later, I did it again on purpose"),
published: Some(Time(
year: 2022,
month: 9,
day: 14,
hour: 17,
minute: 34,
second: 7,
)),
tags: [
"linux",
"software",
],
)
---
Once upon a time, about 2011 to be a little more precise, live was not going my way. My marriage had blown up, my kids were strangers to me and I found myself rather on the far side of sanity. This is not in fact the worst thing that can happen to a man of course because I was still breathing. But I found myself at that time without a job, without a car, and without much in the way of motivation.
One gets bored after a time and finds a way to pass the time. I had left the vast majority of my earthly possessions behind in a storage unit and had at that time no computer. I missed it, however, and found a discarded carcass that I deceided to bring back to life (as one does) with a hard drive but no operating system. Having been a full time Linux and FreeBSD user previously, I wanted a Linux installation. This might be a good time to point out that I also had no internet connection, which combined with not having and operating system to begin with puts a damper on installing Linux. Among what meager possessions I did have with me, however, was an old briefcase with a 512MB pendrive and a CD tucked into a pocket, upon which was written a version of Puppy Linux.
So armed, I started making trips to the local public library in order to use their internet connection and began stuffing as many Slackware packages as possible onto my little 512MB pendrive. I would arrive on foot after a four mile walk (did I mention that I was at the time in the middle of nowhere?), ask to use a computer, and just load it up. So armed, I booted up the computer carcass with the trusty old Puppy Linux CD, formatted the disks to the then shiny ext3 filesystem and extracted each Slackware package individually onto the drive, examining the post install scripts to see if anything further needed done and manually running any needed commands. It took about two weeks to get a full graphical environment this way. And you thought that you had it bad installing from floppy....but I digress.
## The Star Wars connection
Several years prior to this I was at a midnight showing for Revenge of the Sith. Being a huge nerd I showed up early, with all of the other huge nerds. Having the better part of an hour to pass until the movie started I had brought along reading material, in this case a training manual for Red Hat Linux. Being among other fellow nerds it should perhaps not come as a surprise but the enormous bearded guy seated next to me was a Linux enthusiast and wasted no time in telling me that Red Hat was a waste of time. He also just so happened to have a full printed copy of the Linux From Scratch book in his car, which he proceeded to retrieve as a gift for his new friend. We enjoyed the movie and went our separate ways, and after glancing at the book I was intrigued but somewhat intimidated.
Fast forward to my shiny new Slackware installation with it's oh so compelling Fluxbox desktop. I figured in for a penny, what the hell. And so I began making more trips to the library and downloaded the current version of the book along with the required source packages and tarballs. Having nothing if not time on my hands I fired through the book in a day, and then began on Beyond Linux From Scratch. I caught the bug, and was having fun. So much fun that I decided that I would automate everything.
## Enter HitchHiker Linux Mark I
The resulting build system was pretty cool I thought. It wrapped most of the build using a master Makefile and a number of included .mk files such that you could build everything just by entering the source directory and typing `make buildworld`, just like on FreeBSD. Sure, it was kind of brittle, but it worked, and it was way faster than manually typing all of the commands. I even began work on a ports tree, which could quickly get a LAMP server up and running along with a host of other console base utilities. It developed to about the halfway point to a working graphical environment and then stalled, as I had to resume normal life with a job and adult responsibilities again. I saved the work in Git and figured on picking it back up at some later time.
## The SystemD resolvconf incident
It was not long after that when I encountered Arch Linux and began using it in favor of Slackware as my daily driver. It was great in that once configured it just kept running without ever needing a reinstall. I still use Arch on several computers, but there are still a few things that annoy me about Arch. One of them is SystemD. I know that's not an Arch specific thing, and I do see and admit some of the benefits that SystemD brought to the Linux desktop and server. That said, SystemD gave me the impetus to resurrect Hitchhiker. Upon setting up a new system (a Raspberry Pi) with a static IP I began encountering issues with the network. I've been doing this for years, and after double checking I was sure that I had dome it all correctly. But some programs would fail to resolve dns, and some had intermittent issues. When I did finally track the issue down it was due to resolv.conf no longer being the two line file that I had placed there to designate my chosen dns servers, but a symlink to a systemd controlled monstrosity that had replaced my own configuration. I was pissed. Not only does SystemD think that it should have control over this file, it thinks that a convoluted setup involving dnssec should be default. I happen to disagree. My long experience has been that two lines naming a primary and secondary server are ALL. THAT. YOU. EVER. NEED.
Nor was this an isolated thing. I've found that in it's default configuration, SystemD is just way too opinionated. The defaults work great for most people, but the moment that you want to do anything even remotely advanced it throuws up strange roadblocks. After looking around at the various other distros that used a different init system I was unsatisfied. I've since discovered Void Linux, which is excellent, but again I digress.
## HitchHiker Mark II
Thinking that my code was safe turned out to be a mistake. I had chosen Google Code as a platform to keep the original build system while I went off to live my life, back at the time when Google's slogan had been "Do no Evil". While I knew that things had changed in the interim, I was annoyed to discover that my code was all gone. Every project that I had entrusted to Google Code was just gone without a trace. Oh well, I remembered how I had done it once, so I knew how to do it again. More to the point, I knew what didn't work as well as I intended and had a good idea how to fix it.
I went a lot further this time. Linux From Scratch builds a temporary toolchain and environment, chroots into it and uses those tools to build the final system. This is kind of ugly to automate. HitchHiker instead creates a sysroot compiler and then builds the entire base system without having to chroot into it. A number of packages had to be patched. The benefit is that cross compilation is also possible. The system can build from just about any Linux distribution with a functional and relatively modern C compiler, on any architecture, for a number of architectures. It's tested to build correctly for x86_64, x86, aarch64, armv7, and riscv64.
Being a fan of the BSD's I ported a lot of the userland over to Linux. Some came from the Outils project and was ported from OpenBSD already. Some Linux specific utilities came from suckless, and I wrote a fair bit of others in C myself. You get lksh as /bin/sh (the Posix compatible version of the MirBSD Korn shell) and Zsh as a user shell. There are two editors, ee which comes from FreeBSD and heirloom Ex/Vi. So I'm typing this right now using a descendent of the original Unix Vi written by Bill Joy. For a time I was using The One True Awk, but had to replace it with Gawk as there were compatability issues when compiling a lot of third party software. The idea is to have a smallish base system which includes just enough operating system to bootstrap itself from source, upon which you can build whatever you want. The base system has a lot of C code imported directly into it, as is done for BSD, but tracks certain major pieces such as Gcc and Glibc and keeps them up to date. Third party packages are provided via NetBSD pkgsrc. I'm running the Mate desktop right now, with the option of Xfce and a long list of small Window Managers as optional.
Installation is pretty manual. You have to start from a working Linux environment, either making a dual boot or using a live image. You make your partitions, mount them and extract the tarball for the root filesystem. Then either add a Grub configuration entry (for a dual boot) or else install grub. You'll have to edit some config files in /etc and chroot into the system to set a root password. Init is via S6. Once installed, I'm going to be providing a tarball with a bootstrapped version of pkgsrc so that you can just install binary packages using the pkgin package manager from NetBSD. Pkgin works pretty much the same as Apt from a user perspective, so it's not going to seem too foreign for most users. Down the road HitchHiker is going to have it's own ports tree, as there's a lot of stuff that is Linux specific and not included in pkgsrc. I have some of the infrastructure in place for it, but it's not ready for public consumption yet.
I'm also working on a handbook that will guide users through installation and various configuration tasks like setting up the network, and working with the S6 init and supervision suite. It's partway done, but not gauranteed accurate yet as the system keeps evolving.
## When?
I'll be officially putting up the binaries for x86_64 sometime this quarter, followed at some later date with aarch64. I'm looking to get my hands on the next StarV board, after which I'll make it available on RiscV as well. 32-bit can be built for but I'm not going to compile packages for it myself. The source is already available and quite liberally licensed as BeerWare, with third party sources retaining their original licensing of course.
=> https://hitchhiker-linux.org HitchHiker Linux homepage
=> https://git.hitchhiker-linux.org Hitchhiker Linux git repository