From ce26054ec856d61f0a6346b23554542130ae01d8 Mon Sep 17 00:00:00 2001 From: Nathan Fisher Date: Sun, 28 Feb 2021 21:20:12 -0500 Subject: [PATCH] Extended installation and pkgsrc chapters --- src/SUMMARY.md | 6 +++ src/installation/binary/downloading.md | 2 +- src/installation/source/build-process.md | 37 +++++++++++++ src/installation/source/downloading.md | 23 +++++++- src/installation/source/preparing.md | 33 ++++++++++++ src/pkgsrc/README.md | 18 +++++++ src/pkgsrc/bootstrapping.md | 33 ++++++++++++ src/pkgsrc/building-source.md | 28 ++++++++++ src/pkgsrc/extending.md | 28 ++++++++++ src/pkgsrc/getting-pkgsrc.md | 8 +++ src/pkgsrc/installing-binary.md | 67 ++++++++++++++++++++++++ 11 files changed, 281 insertions(+), 2 deletions(-) create mode 100644 src/installation/source/build-process.md create mode 100644 src/pkgsrc/bootstrapping.md create mode 100644 src/pkgsrc/building-source.md create mode 100644 src/pkgsrc/extending.md create mode 100644 src/pkgsrc/getting-pkgsrc.md create mode 100644 src/pkgsrc/installing-binary.md diff --git a/src/SUMMARY.md b/src/SUMMARY.md index 87304a0..3157e77 100644 --- a/src/SUMMARY.md +++ b/src/SUMMARY.md @@ -14,6 +14,7 @@ - [Qemu](installation/binary/riscv/qemu.md) - [BeagleV](installation/binary/riscv/beagle-v.md) - [Installing from Source](installation/source/README.md) + - [The Build Process](installation/source/build-process.md) - [Downloading HHL Source](installation/source/downloading.md) - [Preparing to Build](installation/source/preparing.md) - [Building HitchHiker](installation/source/building-hitchhiker.md) @@ -22,6 +23,11 @@ - [Setting up Networking](configuration/networking.md) - [Managing Processes](s6/README.md) - [Installing Third Party Software](pkgsrc/README.md) + - [Getting Pkgsrc](pkgsrc/getting-pkgsrc.md) + - [Bootstrapping Pkgsrc](pkgsrc/bootstrapping.md) + - [Installing Binary Packages](pkgsrc/installing-binary.md) + - [Building Source Packages](pkgsrc/building-source.md) + - [Extending Pkgsrc](pkgsrc/extending.md) - [Updating HitchHiker Linux](updating/README.md) - [From Binary Sets](updating/binary.md) - [From Source](updating/source.md) diff --git a/src/installation/binary/downloading.md b/src/installation/binary/downloading.md index 398fd3f..8e459ea 100644 --- a/src/installation/binary/downloading.md +++ b/src/installation/binary/downloading.md @@ -1 +1 @@ -# downloading +# Downloading Hitch Hiker Linux binary tarballs diff --git a/src/installation/source/build-process.md b/src/installation/source/build-process.md new file mode 100644 index 0000000..07c0843 --- /dev/null +++ b/src/installation/source/build-process.md @@ -0,0 +1,37 @@ +# The Build Process +This section describes the steps that the HitchHiker build tree takes when +building itself from source. Note that this is not a list of the commands that +must be run by the user, those are actually quite simple. Rather, this page is +here because it is both useful and reasshuring to understand what is being done +when one does run the commands. + +It is expected that certain programs are available on the *host* system, ie the +system that HitchHiker is being built on. In the case that certain programs do +not exist, those programs are compiled in the 'bootstrap' portion of the build. +HitchHiker puposely keeps this part light and prefers to use the native tools of +the host. + +In the second half of the bootstrap phase, a sysroot compiler and linker is built +and installed under ```/toolchain```. This toolchain may or may not be a cross +toolchain, but in either case is of the same version of gcc and binutils as are +to be used in the finished system and are built in such a way that they search +for headers and libraries under ```/src/build``` rather than ```/```. + +In the case that we are compiling for an architecture that is different than the +one which is being used to build the system, this toolchain is what is known as +a *cross* toolchain. What this means is that the compiler and linker run on our +*build* machine (the machine performing the build) but produce machine code that +runs on our *host* machine (the machine the finished system will in fact run on). + +The actual build of the sysroot compiler takes place in several steps: +1. The **binutils** package, which contains our linker and assembler. +2. A temporary and minimally capable **gcc** compiler is built, which is only +capable of building our C library. +3. The *linux* kernel headers are installed into our *sysroot*. +4. The temporary gcc and binutils built in step 1 are used to build *glibc*, which +is installed into our *sysroot*. +5. The final *sysroot* gcc compiler is built, which is a fully capable C and C++ +compiler for our final system. + +We then use the toolchain and bootstrap tools that we have built to compile the +rest of the final system and install it into our *sysroot*. diff --git a/src/installation/source/downloading.md b/src/installation/source/downloading.md index 398fd3f..0283b92 100644 --- a/src/installation/source/downloading.md +++ b/src/installation/source/downloading.md @@ -1 +1,22 @@ -# downloading +# Downloading the Hitch Hiker Linux source +The Hitch Hiker Linux source code is available as an xz compressed tarball from +the hitchhiker-linux.org website, or can alternatively be cloned from the main +git repository. + +The current stable source tarball can be fetched using *wget*: +```Bash +wget -c https://hitchhiker-linux.org/pub/stable/src/src.tar.xz +tar -xJf src.tar.xz +``` +Alternatively, should one wish to build the most recent development version of +HitchHiker, the source can be checked out via *git* over https. +```Bash +git clone https://git.hitchhiker-linux.org/hitchhiker/src.git +``` +>**_NOTE:_** +> +> Currently, the build tree expects to be located in the top level directory +> **/src**, and it is therefore necessary to either ```cd /``` prior to running +> the previous commands, or else to *bind mount* the actual source directory +> onto /src prior to building. This requirement may be changed in the future in +> order to increase flexibility. diff --git a/src/installation/source/preparing.md b/src/installation/source/preparing.md index 11906af..eda8505 100644 --- a/src/installation/source/preparing.md +++ b/src/installation/source/preparing.md @@ -1 +1,34 @@ # Preparing to Build +If Hitch Hiker is being built on a Linux system other than itself, the following +software is expected to be present: +* Binutils - version 2.25 or greater +* Bison - version 2.7 or greater +* a functional **bzip2** and **bunzip2** command +* a functional **diff** command +* a functional **find** command +* Gcc - version 6.2 or greater +* a functional **grep** command +* a functional **gzip** and **gunzip** command +* Linux kernel version 3.2 or greater +* M4 - version 1.4.10 or greater +* GNU make +* a functional **patch** command +* GNU sed is required to build the Linux kernel +* a functional **tar** command +* Texinfo - version 4.7 or greater +* a functional **xz** and **unxz** command +Change directory to /src and create your **config.mk** file by copying +**config.mk.sample** and editing it appropriately. The comments in the sample +explain what each variable does. +```Bash +cd /src +cp config.mk.sample +vi config.mk +``` +As a final step before building, it is neccessary to prepare the environment in +which we are going to build Hitch Hiker. There is a script provided for this +purpose which cleans up some environment variables and sets one **MAKEFLAG** in +order to let **make** find our include files. +```Bash +source scripts/setenv.sh +``` diff --git a/src/pkgsrc/README.md b/src/pkgsrc/README.md index 5a1a8d3..bfcae5c 100644 --- a/src/pkgsrc/README.md +++ b/src/pkgsrc/README.md @@ -1 +1,19 @@ # Installing Third Party Software +Hitch Hiker uses the [pkgsrc](https://www.pkgsrc.org/) framework for installing +extra software on top of the base system. Contrary to what is common in the world +of Linux, there is a clear distinction between the base system and all third +party packages. This, in turn, allows for more developer focus on the base system +and less time packaging software. This is the model followed by all of the various +BSD systems, where the base system and packages are considered to be separate +projects. + +### What is Pkgsrc +Pkgsrc is a cross platform framework for packaging and installing third party +software which is developed and maintained by the NetBSD project. Currently it +works on all BSD systems, Minix, Darwin,MacOS X, and Linux to name a few, across +multiple architectures. +### Further Reading +In addition to this chapter you are highly encouraged to read +[The pkgsrc guide](http://www.netbsd.org/docs/pkgsrc/index.html) and keep it +handy as reference, as a full reference to pkgsrc is beyond the scope of this +chapter. diff --git a/src/pkgsrc/bootstrapping.md b/src/pkgsrc/bootstrapping.md new file mode 100644 index 0000000..39b0fee --- /dev/null +++ b/src/pkgsrc/bootstrapping.md @@ -0,0 +1,33 @@ +# Bootstrapping Pkgsrc +Before installing packages from source or from binary, it is necessary to +bootstrap pkgsrc. This is neccessary in order to build the tools that pkgsrc +requires which are not a part of the Hitch Hiker base system. +```Bash +cd /usr/pkgsrc/bootstrap +./bootstrap +``` +Once this is done it is necessary to either reload the shell startup files or +log out and back in again, in order to add /usr/pkg/bin and /usr/pkg/sbin to our +$PATH. +```Bash +exec zsh --login +``` +As a final step, it is neccessary to add /usr/pkg/lib to the default linker path. +This path is configured in **/etc/ld.so.conf** and can be extended with files +placed in **/etc/ld.so.conf.d**. Finally, after configuring the linker paths, we +run **ldconfig** to update the linker's cache of shared libraries. +```Bash +install -d /etc/ld.so.conf.d +echo '/usr/pkg/lib' > /etc/ld.so.conf.d/pkgsrc.conf +ldconfig +``` + +>**_NOTE:_** +> +> Ordinarily the preceding command would be run as the root user. It is, however, +> possible to install packages as an unprivileged user by first making the package +> prefix directory writable by that user. In the case of a normal installation +> this would entail creating /usr/pkg and transferring ownership of it using the +> **chown** command *before* bootstrapping pkgsrc. It is also possible to +> bootstrap pkgsrc in one's own home directory. However, as this will break +> compatability with any provided binary packages it is therefore discouraged. diff --git a/src/pkgsrc/building-source.md b/src/pkgsrc/building-source.md new file mode 100644 index 0000000..15fd94c --- /dev/null +++ b/src/pkgsrc/building-source.md @@ -0,0 +1,28 @@ +# Building Source Packages +The first step in building a package is to locate the directory within the pkgsrc +tree that contains that package, and then entering that directory. There is a +provided search utility called **pkglocate** which can be used to do a keyword +search. However, in practice pkglocate is not a very effective search tool. + +A much better search tool can be found at ```pkgtools/pkgfind```. To install it: +```Bash +cd /usr/pkgsrc/pkgtools/pkgfind +bmake install clean clean-depends +``` +Now running ```pkgfind ``` should return a much better list of search +results. +> **_NOTE:_** +> +> It is good practice to always run the ```clean``` and ```clean-depends``` +> targets after running ```bmake install```, to avoid leaving old work directories +> all over the tree +As demonstrated by our installation of ```pkgfind```, the general procedure to +build a package using pkgsrc is to: +1. Locate the directory containing the package we wish to install. +2. ```cd``` into that directory +3. Run ```bmake install``` in that directory. +> **_NOTE:_** +> +> We run ```bmake``` as opposed to just ```make``` because pkgsrc uses the NetBSD +> version of the make utility, which supports somewhat different ```Makefile``` +> syntax than GNU make. diff --git a/src/pkgsrc/extending.md b/src/pkgsrc/extending.md new file mode 100644 index 0000000..9222b33 --- /dev/null +++ b/src/pkgsrc/extending.md @@ -0,0 +1,28 @@ +# Extending Pkgsrc +While pkgsrc contains an extensive collection of software out of the box, there +are times when there are packages that have not yet been packaged or have build +failures on Hitch Hiker. For those feeling somewhat adventurous, here are two +ways to extend the available packages, providing software not in the default +distribution or alternative packages with Hitch Hiker specific bug fixes. +## pkgsrc-wip +The [pkgsrc-wip project](https://pkgsrc.org/wip/) is a collection of work in +progress packages which for one reason or another have not yet been accepted into +the main distribution. Much of the time these packages are perfectly functional, +but might not work on every OS that pkgsrc supports. The pkgsrc-wip repository +is managed via git, and due to the main pkgsrc framework being managed under CVS +it is safe to clone pkgsrc-wip right into pkgsrc. +```Bash +cd /usr/pkgsrc +git clone https://pkgsrc.org/wip/ wip +``` +This will put all of pkgsrc-wip right inside pkgsrc at **/usr/pkgsrc/wip**. Once +there, using wip is exactly the same as using the rest of pkgsrc. +## pkgsrc-hhl +The pkgsrc-hhl project is and extension to pkgsrc containing packages that are +intended specifically for Hitch Hiker. These packages may add functionality not +available in pkgsrc, fix build issues, or integrate packages from pkgsrc better +with Hitch Hiker. Installation and usage of pkgsrc-hhl is similar to pkgsrc-wip. +```Bash +cd /usr/pkgsrc +git clone https://git.hitchhiker-linux.org/hitchhiker/pkgsrc-hhl.git hhl +``` diff --git a/src/pkgsrc/getting-pkgsrc.md b/src/pkgsrc/getting-pkgsrc.md new file mode 100644 index 0000000..202f8e2 --- /dev/null +++ b/src/pkgsrc/getting-pkgsrc.md @@ -0,0 +1,8 @@ +# Getting Pkgsrc +The current stable pkgsrc can be downloaded from +[this link](http://cdn.netbsd.org/pub/pkgsrc/stable/pkgsrc.tar.bz2). It should +then be extracted to /usr/pkgsrc. +```Bash +wget -c http://cdn.netbsd.org/pub/pkgsrc/stable/pkgsrc.tar.bz2 +tar -xjf pkgsrc.tar.bz2 -C /usr +``` diff --git a/src/pkgsrc/installing-binary.md b/src/pkgsrc/installing-binary.md new file mode 100644 index 0000000..eab09ee --- /dev/null +++ b/src/pkgsrc/installing-binary.md @@ -0,0 +1,67 @@ +# Installing Binary Packages +It is entirely possible to skip this step if one wishes to build all software +from source. However, should one wish to work with binary packages provided by +Hitch Hiker, it is recommended that the first package built be the +[pkgin](https://pkgin.net/) package manager. +```Bash +zaphod@magrathea% cd /usr/pkgsrc/pkgtools/pkgin +zaphod@magrathea% bmake install clean clean-depends +``` +To configure pkgin, edit the file ```/usr/pkg/etc/pkgin/repositories.conf``` +to point towards the package repository appropriate for your architecture and +version of Hitch Hiker. The [hitchhiker-linux.org](https://hitchiker-linux.org) +server is structures as follows: +``` +pub +└── 2021q2 # Release version + ├── src # Source tree + ├── aarch64 # Architecture + │   ├── packages + │   │   └── All # Package repository + │   ├── release # Release tarballs +``` +Therefore, if the Hitch Hiker release was 2021q2 and the architecture of your +machine was riscv64, the repository can be added with the following command: +```Bash +zaphod@magrathea% echo 'http://hitchhiker-linux.org/pub/2021q2/riscv64/packages/All' \ + >> /usr/pkg/etc/pkgin/repositories.conf +``` +Once pkgin is installed and configured it functions similarly to any good package +manager one might have previously used under Linux, such as apt or pacman. To +see the list of possible operations just run the ```pkgin``` command without +arguments: +```Bash +zaphod@magrathea% pkgin +Usage: pkgin [-cdfhlnPtvVy] command [package ...] + +Commands and shortcuts: +list (ls ) - List installed local packages +avail (av ) - List all available remote packages +search (se ) - Search for a remote package +install (in ) - Install or upgrade packages +update (up ) - Refresh local and remote package lists +upgrade (ug ) - Upgrade all packages +full-upgrade (fug ) - Upgrade all packages (deprecated) +remove (rm ) - Remove packages and any dependent packages +keep (ke ) - Mark packages that should be kept +unkeep (uk ) - Mark packages that can be autoremoved +export (ex ) - Display PKGPATH for all keep packages +import (im ) - Import keep package list from file +show-keep (sk ) - Display keep packages +show-no-keep (snk ) - Display autoremovable packages +autoremove (ar ) - Remove orphaned dependencies +clean (cl ) - Remove downloaded package files +show-deps (sd ) - List remote package direct dependencies +show-full-deps (sfd ) - List remote package full dependencies +show-rev-deps (srd ) - List local package reverse dependencies +provides (prov) - Show which shared libraries a package provides +requires (req ) - Show which shared libraries a package requires +show-category (sc ) - List all packages belonging to a category +show-pkg-category (spc ) - Show categories a package belongs to +show-all-categories (sac ) - List all known categories +pkg-content (pc ) - Show remote package content +pkg-descr (pd ) - Show remote package long-description +pkg-build-defs (pbd ) - Show remote package build definitions +stats (st ) - Show local and remote package statistics +zaphod@magrathea% +```