New news post

content/news/packaging-for-2021q2.md

@@ -0,0 +1,84 @@
++++
+title = "Packaging for 2021q2"
+date = 2021-02-26
+[taxonomies]
+tags = ["NonGNU","Packages","Pkgsrc","C Programming","POSIX"]
++++
+I spent a good deal of the past couple weeks working on actually getting init up
+and running, preparing kernel configurations for multiple architectures, and
+learning about Das U-Boot in order to put up bootable images. Taking a little
+break from that the last few days, I am now once again working at building a set
+of packages using pkgsrc.<!-- more -->
+
+Inevitably considering how many utilities have been replace in Hitch Hiker for
+either BSD versions or scratch implementations, I have been running into breakage
+in odd places. Most are relatively easy fixes. A few kept repeating in an
+infuriating manner...
+
+GNU autotools is an attempt to make a framework that allows building software on
+multiple architectures and operating systems. I won't attempt to elaborate much
+on how it works, but in order to support as many different systems as possible
+the lowest common denominator is assumed when it comes to the infamous 'configure'
+shell script, meaning that it is supposed to be POSIX compliant and not use any
+extensions to the POSIX spec in relation to shell grammar. I'm mentioning this
+because a whole bunch of Gnome related packages use the same test for intltool in
+their configure sript. The gist of the test is this:
+```Bash
+intltool-update --version | head -1 | cut -f" " -d 3
+```
+I previously ran into this when compiling and testing Servo, the browser engine
+formerly sponsored by Mozilla written in Rust. I was pissed at the time, because
+it's such a simple thing. Basically, the POSIX spec only includes the -n option
+when describing the behavior of the head utility. And my version of head did not
+support the older syntax ```head -[num]``` as it is not possible to code that
+sort of thing by just using getopt. The BSD head actually parses and rewrites
+argv in order to support this. Not sure what GNU does but I'm sure it's a few
+hundred lines that shouldn't be necessary.
+
+The fact that this is the same test, with exactly the same syntax and whitespace,
+is a dead giveaway that the package authors basically have just been copying and
+pasting the same code without ever reading or understanding it. It's also indicative
+of another thing I hate; passing --version to a utility should return a string of
+numbers, not multiple lines including a copyright statement and author credit.
+There should be no need for calling head to begin with.
+
+Furthermore, checking for package versions is bad practice to begin with. Say I
+created a fully functional drop-in replacement for intltool. It does exactly what
+intltool does, it accepts the exact same input and gives identical output. If I
+then tried to build any of these packages that check for a specific version of
+intltool they would fail. Best practice is to check for capabilities, not version
+numbers.
+
+To fix the issue I did what I had said previously that I didn't want to do; made
+the head utility in Hitch Hiker accept the non-conforming syntax. I also hacked
+our BSD originating **uname** utility while I was at it to accept the GNU -o
+option and just alias it to -s, as a few things had choked on that option.
+
+### Creating a Hitch Hiker overlay for pkgsrc
+One of the changes that I have been tinkering with in the base system without
+success is the ability to build the other components of gcc beyond the C and C++
+compilers. I have had no luck building the Go, Fortran, Objective C or Objective
+C++ frontends with the sysroot build. So I tried to build the gcc10 version from
+pkgsrc, only to have the Go compiler fail at the last moment. Interestingly, I
+was able to build all of the components manually with zero issues. In looking at
+the Makefile in pkgsrc/lang/gcc10 I realized that there's quite a lot of
+conditionals to aid in building on different platforms and there are also quite a
+lot of options passed to configure beyond what I normally use. In short, for our
+limited purpose I thought that I could do better.
+
+What I have created is a Hitch Hiker specific overlay for pkgsrc. Basically, the
+overlay gets extracted in the same location as pkgsrc and adds some Hitch Hiker
+specific packages, prefixed with hhl_ in their name. The hhl_gcc10 package is
+beautifully simple, builds in approximately 1/3 the time as the pkgsrc version
+(we skip the three stage bootstrap of gcc as we're compiling it with the same
+version from the base system) and builds the C, C++, Fortran, Go, Objective C and
+Objective C++ frontends. Additionally, the pkgsrc gcc10 package puts everything
+under a different prefix, /usr/pkg/gcc10, rather than /usr/pkg. I don't like
+adding extra prefixes in general, so I didn't follow suite with that practice.
+
+I think this is actually a better solution in the long run than continually
+patching pkgsrc for things that fail to build on Hitch Hiker. If a package
+requires a specific change, I can create a new package that does build and it can
+be installed with the name of the original to satisfy dependencies while we try
+to get changes pushed upstream. In this way we can still have a fairly maintainable
+set of Hitch Hiker specific changes, as they will live in their own repo.