# Linux Kernel Module Cheat

#### 2.2.1. QEMU Buildroot setup getting started

This setup has been tested on Ubuntu 20.04.

The Buildroot build is already broken on Ubuntu 21.04 onwards: https://github.com/cirosantilli/linux-kernel-module-cheat/issues/155, just use the Docker host setup setup in that case. We could fix it on Ubuntu 21.04, but it will break again inevitably later on.

For other host operating systems see: Section 38.1, “Supported hosts”.

Reserve 12Gb of disk and run:

    git clone https://github.com/cirosantilli/linux-kernel-module-cheat
    cd linux-kernel-module-cheat
    ./setup
    ./build --download-dependencies qemu-buildroot
    ./run

You don’t need to clone recursively even though we have `.git` submodules: `download-dependencies` fetches just the submodules that you need for this build to save time.

If something goes wrong, see: Section 38.2, “Common build issues” and use our issue tracker: https://github.com/cirosantilli/linux-kernel-module-cheat/issues

The initial build will take a while (30 minutes to 2 hours) to clone and build, see Benchmark builds for more details.

If you don’t want to wait, you could also try the following faster but much more limited methods:

- Prebuilt setup

- Host kernel module setup

but you will soon find that they are simply not enough if you anywhere near serious about systems programming.

After `./run`, QEMU opens up leaving you in the `/lkmc/` directory, and you can start playing with the kernel modules inside the simulated system:

    insmod hello.ko
    insmod hello2.ko
    rmmod hello
    rmmod hello2

This should print to the screen:

    hello init
    hello2 init
    hello cleanup
    hello2 cleanup

which are `printk` messages from `init` and `cleanup` methods of those modules.

Sources:

- kernel_modules/hello.c

- kernel_modules/hello2.c

Quit QEMU with:

    Ctrl-A X

See also: Section 14.1.1, “Quit QEMU from text mode”.

All available modules can be found in the kernel_modules directory.

It is super easy to build for different CPU architectures, just use the `--arch` option:

    ./setup
    ./build --arch aarch64 --download-dependencies qemu-buildroot
    ./run --arch aarch64

To avoid typing `--arch aarch64` many times, you can set the default arch as explained at: Section 38.4, “Default command line arguments”

I now urge you to read the following sections which contain widely applicable information:

- Run command after boot

- Clean the build

- Build the documentation

- Linux kernel

  - printk

  - Kernel command line parameters

Once you use GDB step debug and tmux, your terminal will look a bit like this:

    [    1.451857] input: AT Translated Set 2 keyboard as /devices/platform/i8042/s1│loading @0xffffffffc0000000: ../kernel_modules-1.0//timer.ko
    [    1.454310] ledtrig-cpu: registered to indicate activity on CPUs             │(gdb) b lkmc_timer_callback
    [    1.455621] usbcore: registered new interface driver usbhid                  │Breakpoint 1 at 0xffffffffc0000000: file /home/ciro/bak/git/linux-kernel-module
    [    1.455811] usbhid: USB HID core driver                                      │-cheat/out/x86_64/buildroot/build/kernel_modules-1.0/./timer.c, line 28.
    [    1.462044] NET: Registered protocol family 10                               │(gdb) c
    [    1.467911] Segment Routing with IPv6                                        │Continuing.
    [    1.468407] sit: IPv6, IPv4 and MPLS over IPv4 tunneling driver              │
    [    1.470859] NET: Registered protocol family 17                               │Breakpoint 1, lkmc_timer_callback (data=0xffffffffc0002000 <mytimer>)
    [    1.472017] 9pnet: Installing 9P2000 support                                 │    at /linux-kernel-module-cheat//out/x86_64/buildroot/build/
    [    1.475461] sched_clock: Marking stable (1473574872, 0)->(1554017593, -80442)│kernel_modules-1.0/./timer.c:28
    [    1.479419] ALSA device list:                                                │28      {
    [    1.479567]   No soundcards found.                                           │(gdb) c
    [    1.619187] ata2.00: ATAPI: QEMU DVD-ROM, 2.5+, max UDMA/100                 │Continuing.
    [    1.622954] ata2.00: configured for MWDMA2                                   │
    [    1.644048] scsi 1:0:0:0: CD-ROM            QEMU     QEMU DVD-ROM     2.5+ P5│Breakpoint 1, lkmc_timer_callback (data=0xffffffffc0002000 <mytimer>)
    [    1.741966] tsc: Refined TSC clocksource calibration: 2904.010 MHz           │    at /linux-kernel-module-cheat//out/x86_64/buildroot/build/
    [    1.742796] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x29dc0f4s│kernel_modules-1.0/./timer.c:28
    [    1.743648] clocksource: Switched to clocksource tsc                         │28      {
    [    2.072945] input: ImExPS/2 Generic Explorer Mouse as /devices/platform/i8043│(gdb) bt
    [    2.078641] EXT4-fs (vda): couldn't mount as ext3 due to feature incompatibis│#0  lkmc_timer_callback (data=0xffffffffc0002000 <mytimer>)
    [    2.080350] EXT4-fs (vda): mounting ext2 file system using the ext4 subsystem│    at /linux-kernel-module-cheat//out/x86_64/buildroot/build/
    [    2.088978] EXT4-fs (vda): mounted filesystem without journal. Opts: (null)  │kernel_modules-1.0/./timer.c:28
    [    2.089872] VFS: Mounted root (ext2 filesystem) readonly on device 254:0.    │#1  0xffffffff810ab494 in call_timer_fn (timer=0xffffffffc0002000 <mytimer>,
    [    2.097168] devtmpfs: mounted                                                │    fn=0xffffffffc0000000 <lkmc_timer_callback>) at kernel/time/timer.c:1326
    [    2.126472] Freeing unused kernel memory: 1264K                              │#2  0xffffffff810ab71f in expire_timers (head=<optimized out>,
    [    2.126706] Write protecting the kernel read-only data: 16384k               │    base=<optimized out>) at kernel/time/timer.c:1363
    [    2.129388] Freeing unused kernel memory: 2024K                              │#3  __run_timers (base=<optimized out>) at kernel/time/timer.c:1666
    [    2.139370] Freeing unused kernel memory: 1284K                              │#4  run_timer_softirq (h=<optimized out>) at kernel/time/timer.c:1692
    [    2.246231] EXT4-fs (vda): warning: mounting unchecked fs, running e2fsck isd│#5  0xffffffff81a000cc in __do_softirq () at kernel/softirq.c:285
    [    2.259574] EXT4-fs (vda): re-mounted. Opts: block_validity,barrier,user_xatr│#6  0xffffffff810577cc in invoke_softirq () at kernel/softirq.c:365
    hello S98                                                                       │#7  irq_exit () at kernel/softirq.c:405
                                                                                    │#8  0xffffffff818021ba in exiting_irq () at ./arch/x86/include/asm/apic.h:541
    Apr 15 23:59:23 login[49]: root login on 'console'                              │#9  smp_apic_timer_interrupt (regs=<optimized out>)
    hello /root/.profile                                                            │    at arch/x86/kernel/apic/apic.c:1052
    # insmod /timer.ko                                                              │#10 0xffffffff8180190f in apic_timer_interrupt ()
    [    6.791945] timer: loading out-of-tree module taints kernel.                 │    at arch/x86/entry/entry_64.S:857
    # [    7.821621] 4294894248                                                     │#11 0xffffffff82003df8 in init_thread_union ()
    [    8.851385] 4294894504                                                       │#12 0x0000000000000000 in ?? ()
                                                                                    │(gdb)