Customizing Ubuntu/Debian kernels on i.MX8 boards

Published on August 31, 2020

This post is a modification of Customizing Ubuntu/Debian kernels on i.MX6/7 boards to make it work on i.MX8 boards also. Also I try to give you a guide how to modify and customize the system image on the host PC before installing it on SD card. You can automatize the process if you want and you'll get the a customized and ready to work system image as result.

1. Preparing and building kernel

As usual, the first step is cloning and building the kernel as it was described earlier in our Cross Compiling Kernels post: ~/$ git clone https://github.com/boundarydevices/linux.git && cd linux ~/linux$ export KERNEL_SRC=$PWD ~/linux$ export INSTALL_MOD_PATH=$KERNEL_SRC/ubuntunize64/linux-staging ~/linux$ export ARCH=arm64 ~/linux$ export CROSS_COMPILE=aarch64-linux-gnu- ~/linux$ git checkout boundary-imx_4.14.x_2.0.0_ga ~/linux$ make -C ubuntunize64 prerequisites ~/linux$ make boundary_defconfig ... make your customization, code or configuration changes here. ~/linux$ make Image modules dtbs -j8 ~/linux$ make -C ubuntunize64 tarball ~/linux$ cd .. Let's see line by line:
  1.  Clone Boundary's linux repository
  2.  Set KERNEL_SRC variable, it's needed for building modules.
  3.  Set INSTALL_MOD_PATH variable, it's needed for installing modules here and later too.
  4.  Set the target architecture. It's arm64 now.
  5.  Set the CROSS_COMPILE prefix.
    • We recommend using the official cross-toolchain that comes with Ubuntu/Debian.
  6.  Checkout the kernel branch you need. At the time of this writing, you can choose between the following versions (later this list might be extended):
    • boundary-imx_4.14.x_2.0.0_ga
    • boundary-imx_4.19.x_1.1.0
    • boundary-imx_5.4.x_2.3.0
    • boundary-imx_5.10.x_2.0.0
    • boundary-imx_5.15.y
    • boundary-imx_6.1.y
  7.  Install the arm64 cross-toolchain.
  8.  Create the config file. If you have one of these boards, you need to use boundary_defconfig. Custom designs have their own specific defconfig.
  9.  Now you can customize the kernel configuration by running make menuconfig enable/disable options/modules as you wish.
  10.  Build the kernel image (Image), the modules (.ko) and the device tree blob (.dtb) files using 8 CPU cores. The kernel image is usually uncompressed on arm64 platform, not zImage but Image.
  11.  This command copies the new files in a directory named linux-staging, and then creates a tarball of it.
    • Although the folder name is "ubuntunize64", this process also works with Debian images.
    • The kernel's tarball is placed into the ubuntunize64/ directory, and is named with the kernel version and localversion. If you modified the kernel source, then a -dirty flag will be appended to the name.
~/linux$ ls -l ubuntunize64 total 12828 -rwxrwxr-x 1 tele tele 573 Aug 26 00:43 apt-upgrade.sh -rw-rw-r-- 1 tele tele 2194 Aug 26 01:25 common.mk -rwxrwxr-x 1 tele tele 171 Aug 26 00:43 create-initrd.sh.in -rwxrwxr-x 1 tele tele 1989 Aug 26 01:28 customize-kernel.sh -rw-rw-r-- 1 tele tele 13096839 Aug 26 02:09 linux-4.14.98-g90af3cd4ee6e.tar.gz drwxrwxr-x 4 tele tele 4096 Aug 26 02:09 linux-staging -rw-rw-r-- 1 tele tele 2179 Aug 26 02:08 Makefile -rw-rw-r-- 1 tele tele 2870 Aug 26 00:43 Merge.mk -rw-rw-r-- 1 tele tele 133 Aug 26 01:27 pbuilderrc -rwxrwxr-x 1 tele tele 479 Aug 26 00:43 uninstall-kernel.sh

2. Building galcore and other modules

The galcore is a kernel module which is necessary to run Vivante GPU acceleration packages. Since the galcore module version must match exactly the version of the libraries in the rootfs, it is important to be able to build it independently from the kernel. Here is the procedure to generate the module using the version of your choosing. ~/$ git clone https://github.com/Freescale/kernel-module-imx-gpu-viv.git && cd kernel-module-imx-gpu-viv ~/kernel-module-imx-gpu-viv$ git checkout upstream/6.2.4.p4.6 ~/kernel-module-imx-gpu-viv$ sed 's,-Werror,-Werror -Wno-error=misleading-indentation,g' -i ./kernel-module-imx-gpu-viv-src/Kbuild ~/kernel-module-imx-gpu-viv$ make -j8 ~/kernel-module-imx-gpu-viv$ make modules_install ~/kernel-module-imx-gpu-viv$ cd .. Before you start building this module you have to select the proper galcore version that matches your release. It depends on rootfs you are going to use, so you have to check that first. Please boot the original Ubuntu/Debian system (what you are going to modify) on your board. Open a console window and type: ~/$ dp imx-gpu-viv In the 2nd column you'll find the version.
  1. Checkout the kernel branch you need. This version needs to match the one detailed in the release blog post.
Here is the line by line details of the procedure:
  1.  Clone the galcore repository from Freescale's Github.
  2.  Set the matching branch as described above.
  3.  This sed command is only necessary if you use gcc 6 or higher, which will be used in Debian Stretch for instance.
    • This is a new kind of warning, but it would be considered as error, because of strict module building rules, so it must be disabled.
  4.  In this line we build the module. You don't need to worry about KERNEL_SRC variable, because it was set earlier. We use 8 CPU cores.
  5.  Install the galcore module in place. You don't need to worry about INSTALL_MOD_PATH variable, it was set earlier.
    • The module will be copied to a directory named extra.
Similarly to galcore, you can build any other kernel modules. For example if you have a Silex WiFi module, you'll have to build it's driver as follows: ~/$ git clone https://github.com/boundarydevices/qcacld-2.0.git && cd qcacld-2.0 ~/qcacld-2.0$ export CONFIG_CLD_HL_SDIO_CORE=y ~/qcacld-2.0$ git checkout boundary-CNSS.LEA.NRT_3.1 ~/qcacld-2.0$ make -j8 ~/$qcacld-2.0 make modules_install ~/qcacld-2.0$ cd .. Based on the above examples you can build your own kernel modules as well. Fetch your sources to a working directory, set your environment variables and build flags as necessary, then simply run make and make modules_install. The environment variables KERNEL_SRC and  INSTALL_MOD_PATH has already been set earlier, you don't need to worry about them. After you have built all modules you wanted you need to refresh the tarball: ~/$ make -C $KERNEL_SRC/ubuntunize64 targz

3. Merging rootfs with kernel and modules

In this step we will merge the new kernel and modules and the original rootfs to a complete image tarball. I assume that you have already downloaded a Ubuntu or Debian system image from this page, (after selected your board):

3.1 Merging on the target

Please create the SD card as usual, as described in the paragraph "Programming the image" in the system image's blog post. Then copy the tarball file what you have just created (linux--.tar.gz) to the SD card's ${HOME} directory. Don't forget to type sync to finish copy. Now insert the card in the target board's socket, and boot the new system up. After booting please do this: ~/$ rmlinux ~/$ sudo apt-get purge qcacld-module ~/$ sudo tar --numeric-owner --keep-directory-symlink -xf linux-4.14.98-g90af3cd4ee6e.tar.gz -C / ~/$ sudo update-initramfs -c -k4.14.98-g90af3cd4ee6e ~/$ sudo apt-get update && sudo apt-get dist-upgrade ~/$ sync && sudo reboot
  1. Uninstall the old kernel first
  2. Extract the new kernel
  3. Create an initrd.img ramdisk
  4. Refresh the system, this is optional, it might take about 15-20 minutes
  5. Reboot
As always, let us know how/if this works for you.