Gemlog post about trimming dependencies from Agis
This commit is contained in:
parent
776c38eb2a
commit
fe6bff94e8
4 changed files with 126 additions and 0 deletions
35
content/gemlog/agis_updates.gmi
Normal file
35
content/gemlog/agis_updates.gmi
Normal file
|
@ -0,0 +1,35 @@
|
|||
Meta(
|
||||
title: "Agis updates",
|
||||
summary: Some("I trim some fat from Agis, my Spartan protocol server"),
|
||||
published: Some(Time(
|
||||
year: 2024,
|
||||
month: 5,
|
||||
day: 29,
|
||||
hour: 16,
|
||||
minute: 26,
|
||||
second: 21,
|
||||
)),
|
||||
tags: [
|
||||
"agis",
|
||||
"spartan",
|
||||
"rust",
|
||||
"programming",
|
||||
],
|
||||
)
|
||||
---
|
||||
I've had very little time recently to devote to programming, but there have been a few things that have received some attention in the past few months which are beginning to bear some fruit in more tangible ways. I've been focusing more on library code recently than applications. A side effect is that I'm finding that often an application that I wrote in the past might benefit from applying that library code.
|
||||
|
||||
Case in point is my Spartan server, Agis. What is Agis? It's a multithreaded Spartan protocol server written in Rust with some nice features. It serves this capsule over Spartan, actually. Being written in Rust has benefits and drawbacks. One of the benefits of writing something in Rust is having access to the entire crates ecosystem. One of the serious drawback of writing an application in Rust is having access to the entire crates ecosystem, because adding one dependency is generally going to pull in 3, 4, or 50 transient dependencies. This morning I finished swapping out two of the original dependencies in Agis with my own code as an experiment to see how much the dependency graph is brought down. The results are promising.
|
||||
|
||||
OsRand is a tiny crate I wrote a while back for random number generation. Osrand had the explicit goals of simplicity and no dependencies beyond `std`. Pretty much every RNG on crates.io generates randomness via cryptographic code. That's fine, and it's even good if you want to be able to use it in embedded applications without an OS, but what about when your application is running on some form of Unix? In Unix we already have an RNG at /dev/random and a psuedo-RNG at /dev/urandom. In OsRand I just use the rust std::io::Read interface to read random bytes from one of those device files, configurable at compile time through a feature flag. So yeah, it's tiny and simple.
|
||||
|
||||
But one of the most common use cases for an RNG isn't numerical at all actually. No, a lot of the time we want a random string rather than a random number. So I included a random string generator in OsRand, and to the best of my knowledge it's the only RNG on crates.io which has this capability. Seems like a huge oversight to me, but it's the way that the ecosystem developed I guess. Anyway, one of the original deps for Agis was the `tempfile` crate, which was used to create temporary files with a bit of randomness in their pathname. But tempfile, as I mentioned before, commits the sin of pulling in five other crates when you add it to your Cargo.toml. By replacing tempfile with OsRand I was able to remove five transient dependencies and 8k of binary size.
|
||||
|
||||
Agis has built in logging which is pretty verbose. Every log entry gets the time prepended. As I originally built the program I was using the chrono crate to get the time and make it human readable. Now chrono is a great library that covers 99.99% of anything related to timekeeping. It's a great library. But pulling in all of chrono, plus it's dependencies, was overkill for Agis. My own crate `epoch` does about 70% of what chrono does in a fraction of the code size and with no dependencies beyond rust std. The only drawbacks right now are that I haven't published it to crates.io so you have to get it from git, and I haven't resolved completely getting the current time for a completely stupid reason. Allow me to explain..
|
||||
|
||||
Unix records time using timestamps which are counted as the number of seconds since the Unix epoch, which occurred on Jan 1, 1970. Incidentally that's where the name of the crate came from. Anyway, those seconds were originally recorded as a signed 32-bit integer, or i32, which was way too small and was definitely going to overflow if not made larger. That's a solved problem now, because we all settled on using an i64 instead. By 'all' I mean every extant Unix derivative or clone. But here comes Rust and they decided in their infinite wisdom that we aren't going to care about dates previous to the Unix epoch and we want to go even further into the future, so we're going to have the standard library timekeeping functions use an unsigned 64 bit integer, but we're also going to hide that in an opaque struct and make you call a member function to compare two of those structs and return the difference, the second version being derived from the Unix Epoch, just to get at that internal u64. Fucking. Dumb. But now we're stuck with this shitty interface because stability.
|
||||
Anyway, right now the `DateTime::now()` constructor relies on this shitty interface, which leaves a messy error situation where we now have a potentially fallible conversion from u64 back to i64, which won't ever actually occur anyway because on Unix the rust std interface uses gettimeofday from the C library anyway which returns an i64 (as I've already explained). This is shit code. I'm going to bypass this trainwreck and just get the timestamp from the OS directly. I just haven't done it yet, which is why the crate remains unpublished right now. I'm going to give three options for the `now` constructor, behind a feature flag. Option one is going to be using the libc crate. Option two will be to use a system call via my fork of the `sc` crate, and it's going to specifically be my fork because I've been adding support for other OS and architecture combinations since the only non-linux Unix that upstream seems to care about is x86_64 on FreeBSD and they haven't responded to pull requests in any way. But option two is nice because it's not pulling in the rather large `libc` crate for ONE function. Option three will be to exclude the `now` constructor altogether, because I don't like external dependencies.
|
||||
|
||||
Now that I've written that, I suppose there is a simpler way. I could create a binding that only includes the gettimeofday function, bypassing the libc crate. Something to ponder. Why include bindings to every part of libc when I just need that function? Well because libc is pulled in by other deps that I haven't gotten around to removing. Shit.
|
||||
|
||||
Anyway, just those two crate replacements reduce the dependency graph from 51 crates in a release build down to 41, and shave 12k (12,264 bytes) from the binary size. Now, 12k is a LOT of dead code. This is a lot of the reason I've been souring on Rust in the first place. The Zig compiler would happily just optimize all that dead code away, for instance, and the binary sizes after this sort of twiddling wouldn't be much different. But even ignoring the dead code, which is pretty much unreachable from the binary so not really a security issue, those extra ten dependencies I consider a huge win because the attack surface has shrunk by a fifth in one measure. That is to say, the number of external maintainers being trusted not to have screwed something up which might cause a security issue has been reduced by a fifth. It's still shit in that by including 11 total dependency entries in Cargo.toml I wind up with 41 dependencies due to transients, but it's better.
|
52
content/gemlog/electric_cars.gmi
Normal file
52
content/gemlog/electric_cars.gmi
Normal file
|
@ -0,0 +1,52 @@
|
|||
Meta(
|
||||
title: "Electric Cars",
|
||||
summary: Some("I give my take on electric cars as they exist today"),
|
||||
published: None,
|
||||
tags: [
|
||||
"automobiles",
|
||||
"sustainability",
|
||||
"complexity",
|
||||
],
|
||||
)
|
||||
---
|
||||
> But on the other hand, automobiles in general are a disaster, and most people shouldn't have one.
|
||||
=> gemini://the-brannons.com/ev/
|
||||
|
||||
In a lot of ways I agree with this sentiment, even though I love cars. In the US in particular we have created a way of life that revolves around the car. It is next to impossible to be a full citizen without car ownership. We have long ago decided against mixed use neighborhoods and rejected local first supply chains in favor of cheap crap sold in big box store chains that can only be accessed via car, on the outskirts of town, away from where everyone lives. That's one of the most insidious aspects of the corner we have painted ourselves into, but it's not the one that bothers me the most.
|
||||
|
||||
## I love cars. Old cars.
|
||||
Owning a car is a liberating experience. For a few generations now the first car has been a right of passage, and one of the most prominent steps on the road to independence.
|
||||
|
||||
That said, I hate modern cars. With a passion.
|
||||
|
||||
My dream car is a very simple machine. An inline six cylinder engine is plenty of power and is less complex than any modern engine. One camshaft. One exhaust manifold. Usually a single barrel carburetor. Points ignition. Couple it with a three speed manual shift with overdrive, and let the top cruising speed be about 60mph. But the most efficient cruising speed is 45-50. Roll up windows, with a little vent window in the corner. Manual door locks. A simple three way switch for the windshield wipers - mounted on the dash instead of on a turn signal stalk. Cable operated heater valve and no AC.
|
||||
|
||||
I wouldn't mind going even more primitive with the valves in the block and a magneto ignition. Even less moving parts. Simple to understand. Simple and cheap to maintain. Can be kept on the road for decades.
|
||||
|
||||
This type of transportation was made for slower speeds. People drive 80-90mph on the freeway regularly today, and that makes it scary to even try to drive an older car except on the backroads. It also leads to a lot of accidents and deaths. We should all slow down a bit.
|
||||
|
||||
## I want to like EV's
|
||||
The EV has the potential to be dramatically simpler than an internal combustion powered car. There's no transmission required because all of the torque is available at any motor speed. No driveshafts, Universal Joints or Constant Velocity Joints because we can build the motors right into the wheel hubs. Eliminating the constraints of a "conventional" driveline also frees up design options that can make better use of space and increase aerodynamics. I'm on board with all of that.
|
||||
|
||||
What I don't want is pretty much every other feature they're putting in today's EV's.
|
||||
|
||||
We're literally seeing doors without physical handles or locks.
|
||||
|
||||
Every one of these cars has more sensors pointing at the vehicle's occupants than at the outside world, recording everything we do or say and selling that data to not just the highest bidder but literally every bidder.
|
||||
|
||||
Battery packs are all specific to the manufacturer and never interchangeable.
|
||||
|
||||
These cars are all loaded up with backup cameras and self driving / driver assist features that likely cause more accidents than they prevent.
|
||||
|
||||
## There are other paradigms worth exploring
|
||||
One of the biggest issues with the EV has always been limited range. This issue is closely followed by the need to have the car sit immobile in a charging station in order to refuel, perhaps for hours, before being able to continue on a longer trip. Range is improving, as is charging infrastructure. However, it's likely never going to be as convenient as a gasoline powered car.
|
||||
|
||||
The cost of replacing the entire battery pack in an EV is a significant fraction of the price of the car when it was new. This is another huge problem that we're not even looking at yet, but is going to be a huge problem for the working class who are not fortunate enough to be the target audience for new cars. Instead, they're going to have no lower cost options.
|
||||
|
||||
I want to briefly look at a completely different industry and infrastructure for a moment. That industry, and it's associated infrastructure, is propane fueled barbecue grills. Consider how one re-fuels their grill. You turn the fuel valve off, unscrew the lines and put the propane cylinder into your vehicle. You then drive to a store and exchange your empty tank for a full one. The company that services the tanks owns the tanks, not the consumer. They perform all of the maintenance on the tanks and make sure they are within safe operating specifications. They handle repairs or replacement.
|
||||
|
||||
I can imagine a similar system for EV's. Standard configuration battery packs, which are able to be hot swapped quickly from the vehicle. The company servicing the battery swap facility handles maintenance and replacement. Your car is worth the same amount of money whether the battery that's currently in it is new or old. It takes five minutes to drop a battery pack and lift a new one into place, rather than hours sitting in a charging station drinking gas station coffee.
|
||||
|
||||
You could still plug your car in at night, of course, making this style of EV perfectly suited to the short commute lifestyle that today's EV's are already suited for. But longer trips would no longer be a problem and the most expensive aspect of EV ownership could be mitigated by providing good infrastructure.
|
||||
|
||||
Before dismissing this idea as impractical, let me also point out that when we start seeing large numbers of older EV's out there, their aged battery packs are going to be a liability. Those things aren't going to be properly recycled by cash strapped working class folks who are barely scraping by when they buy a used car with bad batteries, or when they finally have to give up on an older car that can only go five miles between charges and decide to scrap it. Let's be realistic. The batteries are going to be dumped at the back of urban lots, thrown into ravines, left in basements, stacked behind the garage or any number of awful scenarios. Trust a guy who has been poor his entire life - that's what's going to happen. But if, instead, a large energy company is responsible for their maintenance, then when they get caught dumping them (and they likely will), then we'll at least have someone with money to sue in court...
|
23
content/gemlog/mental_health_and_guns.gmi
Normal file
23
content/gemlog/mental_health_and_guns.gmi
Normal file
|
@ -0,0 +1,23 @@
|
|||
Meta(
|
||||
title: "Mental health and guns",
|
||||
summary: Some("Some reflection on the second amendment and mass shootings"),
|
||||
published: None,
|
||||
tags: [
|
||||
"guns",
|
||||
"healthcare",
|
||||
"lewiston",
|
||||
"family",
|
||||
],
|
||||
)
|
||||
---
|
||||
My son is a sophomore at Bowdoin College in Maine. Bowdoin is a fairly small but very old liberal arts school situated in the town of Brunswick, part of the Portland metropolitan area. It's a quiet area most of the time. Quiet has taken on a completely different meaning these past two days, however.
|
||||
|
||||
About a half hour to the North from my son's dorm is the town of Lewiston, the site of the horrific shooting that took place this past Wednesday evening which has claimed 18 lives so far and spawned a massive manhunt, along with shelted in place orders for many of the surrounding communities. The college has cancelled classes the past two days and the students are all being cautioned to stay in their dorms as much as possible. Note that while Lewiston is around a half hours travel to the North, the shooter, Robert Card, apparently abandoned his vehicle in the town of Lisbon, which is South of the site of the shooting. As a dad, it's horrifying to think that the suspect was definitely traveling nearly straight towards my son's location the last place he was known to be. My son is smart and I trust him to play things cautious. Also, there's not much chance that Robert Card is still both alive and in the area considering the time that has passed. None of that makes me feel better right now. I doubt I'll sleep much until I know he's been found, or until my kid is safely back home with us.
|
||||
|
||||
## What caused this?
|
||||
One thing that always happens right after a tragedy like this is that people on one side of the debate around gun control use it as an opportunity to push their cause to the front. I've wanted to punch a number of people for doing this these past couple days. It's not that I disagree with them, it's that they need to shut the fuck up until people either know their loved ones are safe or they have had a chance to grieve, for one thing. The other thing I want these particular assholes to consider is that it's not as simple as they want it to be.
|
||||
|
||||
### Mass shootings are not a recent thing
|
||||
The first time in my life that I remember hearing about mass shootings was Columbine. Probably like a lot of other people, I've been living under the delusion for a large chunk of my life that this is a relatively recent phenomenon. A few months back I actually went looking for data that would support this in order to shut up someone who was preaching gun control. The thing is, my family have always been very much in favor of the second amendment. I went hunting with my dad and my older brother for the first time when I was 14. My grandfather ran a sportsman's supply business and helped establish a hunting and fishing resort in a rural part of Canada. Both of my brothers own multiple firearms. Even my mother owns a handgun. And while I don't own any guns, that was always due not to ethical misgivings but being broke and not very interested. No, I had no problem with private gun ownership and I've always read the second amendment as a very simple prohibition against the government curtailing weapons ownership in any form.
|
||||
|
||||
Imagine my surprise when my research immediately showed that mass shootings have been a problem in the US almost from day one. It's bad. If you look at the history and timeline there are some truly horrific incidents. It made me question my beliefs in a big way.
|
16
content/gemlog/yak_shaving.gmi
Normal file
16
content/gemlog/yak_shaving.gmi
Normal file
|
@ -0,0 +1,16 @@
|
|||
Meta(
|
||||
title: "Yak Shaving",
|
||||
summary: None,
|
||||
published: None,
|
||||
tags: [
|
||||
"software",
|
||||
"programming",
|
||||
"rust",
|
||||
],
|
||||
)
|
||||
---
|
||||
It's funny the little tangents I'll go on sometimes when I'm working on a project. I have a bad NIH complex at times which is getting worse in relation to Rust. Often this is spurred on because I actually care about binary sizes and dependency graphs, so I will gladly reinvent a few wheels in order to trim out some bloat in both of those regards.
|
||||
|
||||
I've gotten my little archiver, Haggis, working pretty well now. At least the Rust implementation is working and pretty well tested. I have a tested C library called SeaHag which is fairly complete, just in need of a binary to complete the proof of concept. I also have a Zig implementation which is nearly as complete as the C one, which is to say I've spent way too much time farting around. But I did actually begin working a bit more towards building my package manager for HitchHiker, which was after all the goal here. Then I went down another rabbit hole.
|
||||
|
||||
I wanted some really flexible versioning for my package manager. You can *mostly* count on open source projects to use SemVer, but there are jackasses like Mozilla who like to just give you a single number as the version, and there's even a few shops that like to give you a four number version of SemVer. Then there's the need account for pre-releases, and properly order them. I went so far as to enable using a Git revision as a pre-release, and that's when the rabbit hole opened up again.
|
Loading…
Add table
Reference in a new issue