mirror of
https://github.com/cirosantilli/linux-kernel-module-cheat.git
synced 2026-01-25 03:01:36 +01:00
prebuilt
This commit is contained in:
Ciro Santilli
193
README.adoc
193
README.adoc
@@ -15,7 +15,7 @@ toc::[]
|
||||
|
||||
== Getting started
|
||||
|
||||
=== Getting started Ubuntu
|
||||
=== Getting started with Ubuntu
|
||||
|
||||
This is the most native setup, and therefore the best one if you are on one of the supported Ubuntu: 16.04 or 18.04.
|
||||
|
||||
@@ -29,7 +29,12 @@ cd linux-kernel-module-cheat
|
||||
|
||||
The first configure will take a while (30 minutes to 2 hours) to clone and build, see <<benchmark-builds>> for more details.
|
||||
|
||||
If you don't want to wait, you could also try to compile the examples and run them on your host computer as explained on at <<run-kernel-modules-on-host>>, but as explained on that section, that is dangerous, limited, and will likely not work.
|
||||
If you don't want to wait, you could also try the following faster but much more limited methods:
|
||||
|
||||
* <<prebuilt>>
|
||||
* <<host>>
|
||||
|
||||
but you will soon find that they are simply not enough if you anywhere near serious about systems programming.
|
||||
|
||||
After QEMU opens up, you can start playing with the kernel modules:
|
||||
|
||||
@@ -103,7 +108,7 @@ We will try to support the following Ubuntu versions at least:
|
||||
* otherwise, support both latest LTS and the latest non-LTS
|
||||
|
||||
[[docker]]
|
||||
=== Getting started Docker
|
||||
=== Getting started with Docker
|
||||
|
||||
This is a good option if you are on a Linux host, but the <<getting-started-ubuntu,native build>> failed due to your weird host distribution.
|
||||
|
||||
@@ -209,6 +214,103 @@ After this, to start using Docker again will you need another:
|
||||
./rundocker setup
|
||||
....
|
||||
|
||||
[[prebuilt]]
|
||||
=== Getting started with prebuilts
|
||||
|
||||
We don't currently provide a full prebuilt because it would be too big to host freely.
|
||||
|
||||
However, if you just want to quickly try out running our kernel modules, try this:
|
||||
|
||||
.
|
||||
+
|
||||
....
|
||||
sudo apt-get install qemu-system-x86
|
||||
git clone https://github.com/cirosantilli/linux-kernel-module-cheat
|
||||
cd linux-kernel-module-cheat
|
||||
....
|
||||
. go to the latest release https://github.com/cirosantilli/linux-kernel-module-cheat/releases
|
||||
. download the `images-*.zip` file
|
||||
|
||||
Limitations of this method:
|
||||
|
||||
* can't GDB step debug the kernel, since the source is not available, and no cross toolchain, and Buildroot cannot easily use a host one: <<prebuilt-toolchain>>.
|
||||
+
|
||||
Maybe we could work around this by just downloading the kernel source somehow, and using a host prebuilt GDB, but we felt that it would be too messy and unreliable.
|
||||
* can't create new modules or modify the existing ones, since no cross toolchain
|
||||
* can't use things that rely on our QEMU fork, e.g. in-fork <<hardware-models>> or <<tracing>>
|
||||
* you won't get the latest version of this repository. Our <<travis>> attempt failed.
|
||||
|
||||
[[host]]
|
||||
=== Getting started with host
|
||||
|
||||
This method runs the kernel modules directly on your host computer without a VM, and saves you the compilation time and disk usage of the virtual machine method.
|
||||
|
||||
It has however severe limitations, and you will soon see that the compilation time and disk usage are well worth it:
|
||||
|
||||
* can't control which kernel version and build options to use. So some of the modules will likely not compile because of kernel API changes, since https://stackoverflow.com/questions/37098482/how-to-build-a-linux-kernel-module-so-that-it-is-compatible-with-all-kernel-rele/45429681#45429681[the Linux kernel does not have a stable kernel module API].
|
||||
* bugs can easily break you system. E.g.:
|
||||
** segfaults can trivially lead to a kernel crash, and require a reboot
|
||||
** your disk could get erased. Yes, this can also happen with `sudo` from userland. But you should not use `sudo` when developing newbie programs. And for the kernel you don't have the choice not to use `sudo`.
|
||||
** even more subtle system corruption such as https://unix.stackexchange.com/questions/78858/cannot-remove-or-reinsert-kernel-module-after-error-while-inserting-it-without-r[not being able to rmmod]
|
||||
* can't control which hardware is used, notably the CPU architecture
|
||||
* can't step debug it with <<gdb,GDB>> easily. The alternatives are JTAG or <<kgdb>>, but those are less reliable, and JTAG requires extra hardware.
|
||||
|
||||
Still interested?
|
||||
|
||||
....
|
||||
cd kernel_module
|
||||
./make-host.sh
|
||||
....
|
||||
|
||||
If the compilation of any of the C files fails because of kernel or toolchain differences that we don't control on the host, just rename it to remove the `.c` extension and try again:
|
||||
|
||||
....
|
||||
mv broken.c broken.c~
|
||||
./build_host
|
||||
....
|
||||
|
||||
Once you manage to compile, and have come to terms with the fact that this may blow up your host, try it out with:
|
||||
|
||||
....
|
||||
sudo insmod hello.ko
|
||||
|
||||
# Our module is there.
|
||||
sudo lsmod | grep hello
|
||||
|
||||
# Last message should be: hello init
|
||||
dmest -T
|
||||
|
||||
sudo rmmod hello
|
||||
|
||||
# Last message should be: hello exit
|
||||
dmesg -T
|
||||
|
||||
# Not present anymore
|
||||
sudo lsmod | grep hello
|
||||
....
|
||||
|
||||
Once you are done with this method, you must clean up the in-tree build objects before you decide to do the right thing and move on to the superior `./build` Buildroot method:
|
||||
|
||||
....
|
||||
cd "kernel_module"
|
||||
./make-host.sh clean
|
||||
....
|
||||
|
||||
otherwise they will cause problems.
|
||||
|
||||
==== Hello host
|
||||
|
||||
Minimal host build system example:
|
||||
|
||||
....
|
||||
cd hello_host
|
||||
make
|
||||
insmod hello.ko
|
||||
dmesg
|
||||
rmmod hello.ko
|
||||
dmesg
|
||||
....
|
||||
|
||||
=== Text mode
|
||||
|
||||
By default, we show the serial console directly on the current terminal, without opening a QEMU window.
|
||||
@@ -5864,9 +5966,7 @@ Finally, do a clone of the relevant repository out of tree and reproduce the bug
|
||||
|
||||
In this section document how benchmark builds and runs of this repo, and how to investigate what the bottleneck is.
|
||||
|
||||
Ideally, we should setup an automated build server that benchmarks those things continuously for us.
|
||||
|
||||
We tried to automate it on Travis with link:.travis.yml[] but it hits the current 50 minute job timeout: https://travis-ci.org/cirosantilli/linux-kernel-module-cheat/builds/296454523 And I bet it would likely hit a disk maxout either way if it went on.
|
||||
Ideally, we should setup an automated build server that benchmarks those things continuously for us, but our <<travis>> attempt failed.
|
||||
|
||||
So currently, we are running benchmarks manually when it seems reasonable and uploading them to: https://github.com/cirosantilli/linux-kernel-module-cheat-regression
|
||||
|
||||
@@ -5878,6 +5978,10 @@ Run all benchmarks and upload the results:
|
||||
./bench-all -A
|
||||
....
|
||||
|
||||
=== Travis
|
||||
|
||||
We tried to automate it on Travis with link:.travis.yml[] but it hits the current 50 minute job timeout: https://travis-ci.org/cirosantilli/linux-kernel-module-cheat/builds/296454523 And I bet it would likely hit a disk maxout either way if it went on.
|
||||
|
||||
=== Benchmark this repo benchmarks
|
||||
|
||||
==== Benchmark Linux kernel boot
|
||||
@@ -5997,8 +6101,6 @@ Our philosophy is:
|
||||
* try to keep the toolchain (GCC, Binutils) unchanged, otherwise a full rebuild is required.
|
||||
+
|
||||
So we generally just enable all toolchain options by default, even though this adds a bit of time to the build.
|
||||
+
|
||||
The biggest build time hog is always GCC, and it does not look like we can use a precompiled one: https://stackoverflow.com/questions/10833672/buildroot-environment-with-host-toolchain
|
||||
* if something is very valuable, we just add it by default even if it increases the Build time, notably GDB and QEMU
|
||||
* runtime is sacred.
|
||||
+
|
||||
@@ -6011,6 +6113,11 @@ We do our best to reduce the instruction and feature count to the bare minimum n
|
||||
+
|
||||
One possibility we could play with is to build loadable modules instead of built-in modules to reduce runtime, but make it easier to get started with the modules.
|
||||
|
||||
[[prebuilt-toolchain]]
|
||||
====== Buildroot use prebuilt host toolchain
|
||||
|
||||
The biggest build time hog is always GCC, and it does not look like we can use a precompiled one: https://stackoverflow.com/questions/10833672/buildroot-environment-with-host-toolchain
|
||||
|
||||
===== Benchmark Buildroot build baseline
|
||||
|
||||
This is the minimal build we could expect to get away with.
|
||||
@@ -6095,76 +6202,6 @@ gem5:
|
||||
** https://stackoverflow.com/questions/47997565/gem5-system-requirements-for-decent-performance/48941793#48941793
|
||||
** https://github.com/gem5/gem5/issues/25
|
||||
|
||||
== Run kernel modules on host
|
||||
|
||||
This method runs the kernel modules directly on your host computer without a VM, and saves you the compilation time and disk usage of the virtual machine method.
|
||||
|
||||
It has however severe limitations, and you will soon see that the compilation time and disk usage are well worth it:
|
||||
|
||||
* can't control which kernel version and build options to use. So some of the modules will likely not compile because of kernel API changes, since https://stackoverflow.com/questions/37098482/how-to-build-a-linux-kernel-module-so-that-it-is-compatible-with-all-kernel-rele/45429681#45429681[the Linux kernel does not have a stable kernel module API].
|
||||
* bugs can easily break you system. E.g.:
|
||||
** segfaults can trivially lead to a kernel crash, and require a reboot
|
||||
** your disk could get erased. Yes, this can also happen with `sudo` from userland. But you should not use `sudo` when developing newbie programs. And for the kernel you don't have the choice not to use `sudo`.
|
||||
** even more subtle system corruption such as https://unix.stackexchange.com/questions/78858/cannot-remove-or-reinsert-kernel-module-after-error-while-inserting-it-without-r[not being able to rmmod]
|
||||
* can't control which hardware is used, notably the CPU architecture
|
||||
* can't step debug it with <<gdb,GDB>> easily. The alternatives are JTAG or <<kgdb>>, but those are less reliable, and JTAG requires extra hardware.
|
||||
|
||||
Still interested?
|
||||
|
||||
....
|
||||
cd kernel_module
|
||||
./make-host.sh
|
||||
....
|
||||
|
||||
If the compilation of any of the C files fails because of kernel or toolchain differences that we don't control on the host, just rename it to remove the `.c` extension and try again:
|
||||
|
||||
....
|
||||
mv broken.c broken.c~
|
||||
./build_host
|
||||
....
|
||||
|
||||
Once you manage to compile, and have come to terms with the fact that this may blow up your host, try it out with:
|
||||
|
||||
....
|
||||
sudo insmod hello.ko
|
||||
|
||||
# Our module is there.
|
||||
sudo lsmod | grep hello
|
||||
|
||||
# Last message should be: hello init
|
||||
dmest -T
|
||||
|
||||
sudo rmmod hello
|
||||
|
||||
# Last message should be: hello exit
|
||||
dmesg -T
|
||||
|
||||
# Not present anymore
|
||||
sudo lsmod | grep hello
|
||||
....
|
||||
|
||||
Once you are done with this method, you must clean up the in-tree build objects before you decide to do the right thing and move on to the superior `./build` Buildroot method:
|
||||
|
||||
....
|
||||
cd "kernel_module"
|
||||
./make-host.sh clean
|
||||
....
|
||||
|
||||
otherwise they will cause problems.
|
||||
|
||||
=== Hello host
|
||||
|
||||
Minimal host build system sanity check example.
|
||||
|
||||
....
|
||||
cd hello_host
|
||||
make
|
||||
insmod hello.ko
|
||||
dmesg
|
||||
rmmod hello.ko
|
||||
dmesg
|
||||
....
|
||||
|
||||
== Conversation
|
||||
|
||||
=== kmod
|
||||
|
||||
3
run
3
run
@@ -208,7 +208,7 @@ else
|
||||
extra_flags="${extra_flags_qemu} ${extra_flags}"
|
||||
qemu_common="\
|
||||
${debug_vm} \
|
||||
'${buildroot_out_dir}/host/usr/bin/qemu-system-${arch}' \\
|
||||
'qemu-system-${arch}' \\
|
||||
-device rtl8139,netdev=net0 \\
|
||||
-gdb tcp::1234 \\
|
||||
-m '${memory}' \\
|
||||
@@ -245,7 +245,6 @@ ${qemu_common} \
|
||||
-M pc \\
|
||||
-append '${root} nopat ${extra_append}' \\
|
||||
-device edu \\
|
||||
-device lkmc_pci_min \\
|
||||
-kernel '${images_dir}/bzImage' \\
|
||||
${extra_flags} \
|
||||
"
|
||||
|
||||
6
zip-img
6
zip-img
@@ -1,10 +1,6 @@
|
||||
#!/usr/bin/env bash
|
||||
set -eu
|
||||
# Ideally should be obtained from common.
|
||||
# But the paths there are absolute, and get recorded by in zip.
|
||||
# and generating relative paths seems hard:
|
||||
# https://stackoverflow.com/questions/2564634/convert-absolute-path-into-relative-path-given-a-current-directory-using-bash
|
||||
outfile="out/images-$(git log -1 --format="%H").zip"
|
||||
outfile="out/out.zip"
|
||||
rm -f "$outfile"
|
||||
for arch in x86_64 arm aarch64; do
|
||||
img_dir="out/${arch}/buildroot/images"
|
||||
|
||||
Reference in New Issue
Block a user