LFS : Linux from Scratch

 Why

Linux installation is straightforward.  For an RPi you just download an image in Windows, burn it on to an SD card, insert the card in RPi with screen, keyboard attached and turn the power on.  Adding network capability is similarly easy, in Raspbian you specify the package name to the apt command and the necessary steps are completed to install dependencies and the requested software.

From other activities I know that setting up partitions and choosing a kernel are also important.  Building a kernel is a matter of ensuring you have all the necessary files and running one long compile job.

The missing part of the picture is the ability to set up all the basic software in files and folders so that the system works.  It turns out that, as expected, there is a lot of work involved.  Thanks to the LFS (Linux from Scratch) project and the PiLFS ( RPi LFS) guide I have been able to successfully build my own minimal system, sign on, add packages etc.    

It is a challenging objective and the LFS team have done an amazing job in making each of the 100 or so tasks understandable so that you can create the Linux system you want.  Even with this  I would have struggled to build my own system.  PiLFS provides a guide to how to implement LFS on a Pi system together with a host configuration containing working build software and two large scripts to automate many of the tasks.  I feel that LFS is awesome and I'm grateful to PiLFS for making it practical to build with my patchy Linux knowledge.

Host System

You need a set of software running on the host system to be able to build linux, the list includes kernel, c compiler, make and various utilities:

bash  binutils  bison  bzip2  coreutils  diffutils  findutils  gawk  GCC  glibc  grep  gzip  Linux-kernel
m4  make  patch  perl  python  sed  tar  texinfo  xz

This isn't a long list bearing in mind the scale of the work we are undertaking.  I use a RPi4 as my host system.  PiLFS provide a RPi image so we can be sure we start with the correct pre-requisites.  We boot RPi with the image and we can start building. 

The first task is to make a new partition as a target for the software which is built.  Following PiLFS suggestion create a new partition (20GB) on the SD card.  All the software will be installed in the root file system; on completion we boot software from the new partition.

Download software and create LFS user

A number of tarballs are downloaded containing sources for software to be installed on our new system.  They are installed in the /sources folder on the new system.  There are also some patch files which will need to be applied.  Software is downloaded from the linux from scratch website which ensures that packages are all at compatible levels.

A userid LFS is setup to carry out compilation tasks for the new system.  Good practice says that root should only be used when necessary, LFS is able to carry out most of the work.

Create Toolchain and utility tools

The first thing we need to do as user lfs is build a toolchain for the new system.  Binutils is created first, it provides a linker and assembler.  Next we compile GCC onto the new system - of course we are using GCC on the host system to do the compilation.  Our task is made easier because both our host and target systems are RPis so GCC is going to work.  A proper cross-compiler would be (even) more challenging.

The remaining toolchain software we need is linux-headers, glibc and libstdc (for c++).  These five builds are quite complex and PiLFS provides a script which automates the process for us.

The script also compiles utilities which will be required on the new system m4, ncurses, bash, coreutils, diffutil, file, findutils, gawk, grep gzip, make, patch, sed, tar, xz. 

Entering Chroot

The chroot command allows us to use the new filesystem we have created with the existing kernel. The chroot environment doesn't have access to the host software.  Before entering chroot we have to setup the virtual filesystem that the kernel uses to manage / communicate with software running in the environment. 

Within chroot, as root user, we create some essential files (/etc/hosts. pwsswd, group) and compile some more tools (gettext, bison, perl, python, texinfo, util-linux).

Building new system software

We are finally ready to do the proper build of all the software on the new system.  There are 73 steps to carry out and PiLFS has simplified the process by putting all the builds into a script for us.
Many hours later the build is finished.

Finale

One of the last tasks is to setup a filesystem table (/etc/fstab) for the target system.  It looks very similar to the host except that the root partition is changed to the one we have just been building.

In our simplified environment we are going to use the same kernel as before - there is no particular advantage in building a new one and we can do it as a separate exercise whenever.  In addition we have been working for many hours to get this far and I want to see if it works... 

it does :):):)

Amazingly we have our own working linux system.  Anything that works has been done by us and anything that doesn't is yet to be installed.  It is very satisfying.  Thanks to the LFS project I know (or can find out by checking the doc) what all the components are, how they are built and what they do.  In particular LFS has an excellent summary of the relevant configuration options used to create the software.

Next Steps

There are a few extras we need to add for a basic working system:
openssh - needed to login from a terminal session, in particular to be able to copy / paste commands.
wget - needed to download packages to extend our system
ntp - RPi has no clock so we need to obtain the time at each system startup.

We can also install other software installs facilitated by LFS or attempt our own installations from github etc.

In conclusion this is a wonderful exercise to help me understand what I am using on a Linux system.