From 23cb0409c9804c15315bdf551fb432e5dc0906d5 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ciro=20Santilli=20=E5=85=AD=E5=9B=9B=E4=BA=8B=E4=BB=B6=20?= =?UTF-8?q?=E6=B3=95=E8=BD=AE=E5=8A=9F?= Date: Fri, 31 Jul 2020 01:00:00 +0000 Subject: [PATCH] 6a9c478a8371bc19c04a56b69e62ae425d783d12 --- index.html | 357 +++++++++++++++++++++++++++-------------------------- 1 file changed, 184 insertions(+), 173 deletions(-) diff --git a/index.html b/index.html index 3ac4aeb..f4cec6e 100644 --- a/index.html +++ b/index.html @@ -5,7 +5,7 @@ - + Linux Kernel Module Cheat +
-

The perfect emulation setup to study and develop the Linux kernel v5.4.3, kernel modules, QEMU, gem5 and x86_64, ARMv7 and ARMv8 userland and baremetal assembly, ANSI C, C++ and POSIX. GDB step debug and KGDB just work. Powered by Buildroot and crosstool-NG. Highly automated. Thoroughly documented. Automated tests. "Tested" in an Ubuntu 19.10 host.

+

The perfect emulation setup to study and develop the Linux kernel v5.4.3, kernel modules, QEMU, gem5 and x86_64, ARMv7 and ARMv8 userland and baremetal assembly, ANSI C, C++ and POSIX. GDB step debug and KGDB just work. Powered by Buildroot and crosstool-NG. Highly automated. Thoroughly documented. Automated tests. "Tested" in an Ubuntu 20.04 host.

-

TL;DR: Section 1.1.1, “QEMU Buildroot setup getting started”

+

TL;DR: Section 1.2.1, “QEMU Buildroot setup getting started”

-

The source code for this page is located at: https://github.com/cirosantilli/linux-kernel-module-cheat. Due to a GitHub limitation, this README is too long and not fully rendered on github.com. Either use: README.adoc, https://cirosantilli.com/linux-kernel-module-cheat or build the docs yourself.

+

The source code for this page is located at: https://github.com/cirosantilli/linux-kernel-module-cheat. Due to a GitHub limitation, this README is too long and not fully rendered on github.com, so either use: https://cirosantilli.com/linux-kernel-module-cheat or build the docs yourself.

-

1.1. QEMU Buildroot setup

+

1.1. Should you waste your life with systems programming?

+
+

Being the hardcore person who fully understands an important complex system such as a computer, it does have a nice ring to it doesn’t it?

+
+
+

But before you dedicate your life to this nonsense, do consider the following points:

+
+
+
    +
  • +

    almost all contributions to the kernel are done by large companies, and if you are not an employee in one of them, you are likely not going to be able to do much.

    +
    +

    This can be inferred by the fact that the devices/ directory is by far the largest in the kernel.

    +
    +
    +

    The kernel is of course just an interface to hardware, and the hardware developers start developing their kernel stuff even before specs are publicly released, both to help with hardware development and to have things working when the announcement is made.

    +
    +
    +

    Furthermore, I believe that there are in-tree devices which have never been properly publicly documented. Linus is of course fine with this, since code == documentation for him, but it is not as easy for mere mortals.

    +
    +
    +

    There are some less hardware bound higher level layers in the kernel which might not require being in a hardware company, and a few people must be living off it.

    +
    +
    +

    But of course, those are heavily motivated by the underlying hardware characteristics, and it is very likely that most of the people working there were previously at a hardware company.

    +
    +
    +

    In that sense, therefore, the kernel is not as open as one might want to believe.

    +
    +
    +

    Of course, if there is some super useful and undocumented hardware that is just waiting there to be reverse engineered, then that’s a much juicier target :-)

    +
    +
    • +
  • +

    it is impossible to become rich with this knowledge.

    +
    +

    This is partly implied by the fact that you need to be in a big company to make useful low level things, and therefore you will only be a tiny cog in the engine.

    +
    +
    +

    The key problem is that the entry cost of hardware design is just too insanely high for startups in general.

    +
    +
    • +
  • +

    Is learning this the most useful thing that you think can do for society?

    +
    +

    Or are you just learning it for job security and having a nice sounding title?

    +
    +
    +

    I’m not a huge fan of the person, but I think Jobs said it right: https://www.youtube.com/watch?v=FF-tKLISfPE

    +
    +
    +

    First determine the useful goal, and then backtrack down to the most efficient thing you can do to reach it.

    +
    +
    • +
  • +

    there are two things that sadden me compared to physics-based engineering:

    +
    +
    +
    +
      +
    • +

      you will never become eternally famous. All tech disappears sooner or later, while laws of nature, at least as useful approximations, stay unchanged.

      +
      • +
    • +

      every problem that you face is caused by imperfections introduced by other humans.

      +
      +

      It is much easier to accept limitations of physics, and even natural selection in biology, which is are produced by a sentient being (?).

      +
      +
      • +
    +
    +
    +
    +
    +

    Physics-based engineering, just like low level hardware, is of course completely closed source however, since wrestling against the laws of physics is about the most expensive thing humans can do.

    +
    +
    • +
+
+
+

Are you fine with those points, and ready to continue wasting your life with this crap?

+
+
+

Good. In that case, read on, and let’s have some fun together ;-)

+
+
+

Related: Soft topics.

+
+
+
+

1.2. QEMU Buildroot setup

-

1.1.1. QEMU Buildroot setup getting started

+

1.2.1. QEMU Buildroot setup getting started

This setup has been mostly tested on Ubuntu. For other host operating systems see: Section 33.1, “Supported hosts”. For greater stability, consider using the latest release instead of master: https://github.com/cirosantilli/linux-kernel-module-cheat/releases

@@ -2469,12 +2567,12 @@ hello /root/.profile
-

1.1.2. How to hack stuff

+

1.2.2. How to hack stuff

Besides a seamless initial build, this project also aims to make it effortless to modify and rebuild several major components of the system, to serve as an awesome development setup.

-
1.1.2.1. Your first Linux kernel hack
+
1.2.2.1. Your first Linux kernel hack

Let’s hack up the Linux kernel entry point, which is an easy place to start.

@@ -2546,7 +2644,7 @@ hello /root/.profile
-
1.1.2.2. Your first kernel module hack
+
1.2.2.2. Your first kernel module hack

Edit kernel_modules/hello.c to contain:

@@ -2634,7 +2732,7 @@ hello /root/.profile
-
1.1.2.3. Your first QEMU hack
+
1.2.2.3. Your first QEMU hack

Not satisfied with mere software? OK then, let’s hack up the QEMU x86 CPU identification:

@@ -2700,14 +2798,14 @@ hello /root/.profile

If you really want to develop semiconductors, your only choice is to join an university or a semiconductor company that has the EDA licenses.

-

See also: Section 33.21.2, “Should you waste your life with systems programming?”.

+

See also: Section 1.1, “Should you waste your life with systems programming?”.

While hacking QEMU, you will likely want to GDB step its source. That is trivial since QEMU is just another userland program like any other, but our setup has a shortcut to make it even more convenient, see: Section 18.8, “Debug the emulator”.

-
1.1.2.4. Your first glibc hack
+
1.2.2.4. Your first glibc hack

We use glibc as our default libc now, and it is tracked as an unmodified submodule at submodules/glibc, at the exact same version that Buildroot has it, which can be found at: package/glibc/glibc.mk. Buildroot 2018.05 applies no patches.

@@ -2795,7 +2893,7 @@ index 706b20b492..23185948f3 100644
-
1.1.2.5. Your first Binutils hack
+
1.2.2.5. Your first Binutils hack

Have you ever felt that a single inc instruction was not enough? Really? Me too!

@@ -2881,7 +2979,7 @@ index af583ce578..3cc341f303 100644
-
1.1.2.6. Your first GCC hack
+
1.2.2.6. Your first GCC hack

OK, now time to hack GCC.

@@ -2984,7 +3082,7 @@ j = 0
-

1.1.3. About the QEMU Buildroot setup

+

1.2.3. About the QEMU Buildroot setup

This is our reference setup, and the best supported one, use it unless you have good reason not to.

@@ -3007,7 +3105,7 @@ j = 0
-

1.2. Dry run to get commands for your project

+

1.3. Dry run to get commands for your project

One of the major features of this repository is that we try to support the --dry-run option really well for all scripts.

@@ -3093,9 +3191,9 @@ j = 0
-

1.3. gem5 Buildroot setup

+

1.4. gem5 Buildroot setup

-

1.3.1. About the gem5 Buildroot setup

+

1.4.1. About the gem5 Buildroot setup

This setup is like the QEMU Buildroot setup, but it uses gem5 instead of QEMU as a system simulator.

@@ -3132,7 +3230,7 @@ j = 0
-

1.3.2. gem5 Buildroot setup getting started

+

1.4.2. gem5 Buildroot setup getting started

For the most part, if you just add the --emulator gem5 option or *-gem5 suffix to all commands and everything should magically work.

@@ -3226,7 +3324,7 @@ j = 0
-

1.4. Docker host setup

+

1.5. Docker host setup

This repository has been tested inside clean Docker containers.

@@ -3375,9 +3473,9 @@ j = 0
-

1.5. Prebuilt setup

+

1.6. Prebuilt setup

-

1.5.1. About the prebuilt setup

+

1.6.1. About the prebuilt setup

This setup uses prebuilt binaries that we upload to GitHub from time to time.

@@ -3447,7 +3545,7 @@ j = 0
-

1.5.2. Prebuilt setup getting started

+

1.6.2. Prebuilt setup getting started

Checkout to the latest tag and use the Ubuntu packaged QEMU to boot Linux:

@@ -3535,7 +3633,7 @@ unzip lkmc-*.zip
-

1.6. Host kernel module setup

+

1.7. Host kernel module setup

THIS IS DANGEROUS (AND FUN), YOU HAVE BEEN WARNED

@@ -3640,7 +3738,7 @@ sudo lsmod | grep hello
-

1.6.1. Hello host

+

1.7.1. Hello host

Minimal host build system example:

@@ -3657,9 +3755,9 @@ dmesg
-

1.7. Userland setup

+

1.8. Userland setup

-

1.7.1. About the userland setup

+

1.8.1. About the userland setup

In order to test the kernel and emulators, userland content in the form of executables and scripts is of course required, and we store it mostly under:

@@ -3709,14 +3807,14 @@ dmesg
-

1.7.2. Userland setup getting started

+

1.8.2. Userland setup getting started

There are several ways to run our Userland content, notably:

-
1.7.2.1. Userland setup getting started natively
+
1.8.2.1. Userland setup getting started natively

With this setup, we will use the host toolchain and execute executables directly on the host.

@@ -3908,7 +4006,7 @@ cd userland
-
1.7.2.2. Userland setup getting started with prebuilt toolchain and QEMU user mode
+
1.8.2.2. Userland setup getting started with prebuilt toolchain and QEMU user mode

If you are lazy to built the Buildroot toolchain and QEMU, but want to run e.g. ARM Userland assembly in User mode simulation, you can get away on Ubuntu 18.04 with just:

@@ -3946,7 +4044,7 @@ cd userland
-

This present the usual trade-offs of using prebuilts as mentioned at: Section 1.5, “Prebuilt setup”.

+

This present the usual trade-offs of using prebuilts as mentioned at: Section 1.6, “Prebuilt setup”.

Other functionality are analogous, e.g. testing:

@@ -3979,7 +4077,7 @@ cd userland
-
1.7.2.3. Userland setup getting started full system
+
1.8.2.3. Userland setup getting started full system

First ensure that QEMU Buildroot setup is working.

@@ -3987,7 +4085,7 @@ cd userland

After doing that setup, you can already execute your userland programs from inside QEMU: the only missing step is how to rebuild executables and run them.

-

And the answer is exactly analogous to what is shown at: Section 1.1.2.2, “Your first kernel module hack”

+

And the answer is exactly analogous to what is shown at: Section 1.2.2.2, “Your first kernel module hack”

For example, if we modify userland/c/hello.c to print out something different, we can just rebuild it with:

@@ -4028,9 +4126,9 @@ cd userland
-

1.8. Baremetal setup

+

1.9. Baremetal setup

-

1.8.1. About the baremetal setup

+

1.9.1. About the baremetal setup

This setup does not use the Linux kernel nor Buildroot at all: it just runs your very own minimal OS.

@@ -4051,7 +4149,7 @@ cd userland
-

1.8.2. Baremetal setup getting started

+

1.9.2. Baremetal setup getting started

Every .c file inside baremetal/ and .S file inside baremetal/arch/<arch>/ generates a separate baremetal image.

@@ -4262,7 +4360,7 @@ echo "$(./getvar --arch aarch64 --baremetal userland/c/hello.c --emulator gem5 -
-

1.9. Build the documentation

+

1.10. Build the documentation

You don’t need to depend on GitHub.

@@ -7874,7 +7972,7 @@ qw er

./run --userland path resolution is analogous to that of ./run --baremetal.

-

./build user-mode-qemu first builds Buildroot, and then runs ./build-userland, which is further documented at: Section 1.7, “Userland setup”. It also builds QEMU. If you ahve already done a QEMU Buildroot setup previously, this will be very fast.

+

./build user-mode-qemu first builds Buildroot, and then runs ./build-userland, which is further documented at: Section 1.8, “Userland setup”. It also builds QEMU. If you ahve already done a QEMU Buildroot setup previously, this will be very fast.

If you modify the userland programs, rebuild simply with:

@@ -8849,7 +8947,7 @@ Program aborted at tick 0
  • -

    we would have to think how to not have to include the kernel modules twice in the root filesystem, but still have 9P working for fast development as described at: Section 1.1.2.2, “Your first kernel module hack”

    +

    we would have to think how to not have to include the kernel modules twice in the root filesystem, but still have 9P working for fast development as described at: Section 1.2.2.2, “Your first kernel module hack”

    • @@ -18654,7 +18752,7 @@ root

    19. gem5

    -

    Getting started at: Section 1.3, “gem5 Buildroot setup”.

    +

    Getting started at: Section 1.4, “gem5 Buildroot setup”.

    gem5 has a bunch of crappiness, mostly described at: gem5 vs QEMU, but it does deserve some credit on the following points:

    @@ -22491,8 +22589,8 @@ wbWidth=8
    -
    19.16.1.3.2. gem5 util/o3-pipeview.py O3 pipeline viewer
    -
    +
    19.16.1.3.2. gem5 util/o3-pipeview.py O3 pipeline viewer
    +

    Mentioned at: http://www.m5sim.org/Visualization

    @@ -22533,7 +22631,7 @@ less o3pipeview.tmp.log

    http://learning.gem5.org/tutorial/presentations/vis-o3-gem5.pdf

    -

    Appears to be browser based, so you can zoom in and out, rather than the forced wrapping as for [gem5-util-o3-pipeview-py-o3-pipeline-viewer].

    +

    Appears to be browser based, so you can zoom in and out, rather than the forced wrapping as for gem5 util/o3-pipeview.py O3 pipeline viewer.

    Uses the same data source as util/o3-pipeview.py.

    @@ -25736,7 +25834,7 @@ non-atomic 19

    Let’s have a look at the arguably simplest example userland/arch/aarch64/freestanding/linux/hazardless.S.

    -

    First let’s start with a [gem5-util-o3-pipeview-py-o3-pipeline-viewer] visualization:

    +

    First let’s start with a gem5 util/o3-pipeview.py O3 pipeline viewer visualization:

    @@ -29430,7 +29528,7 @@ cd ../..

    This section documents our test and educational userland content, such as C, C++ and POSIX examples, present mostly under userland/.

    -

    Getting started at: Section 1.7, “Userland setup”

    +

    Getting started at: Section 1.8, “Userland setup”

    Userland assembly content is located at: Section 22, “Userland assembly”. It was split from this section basically because we were hitting the HTML h6 limit, stupid web :-)

    @@ -29439,7 +29537,7 @@ cd ../..

    This content makes up the bulk of the userland/ directory.

    -

    The quickest way to run the arch agnostic examples, which comprise the majority of the examples, is natively as shown at: Section 1.7.2.1, “Userland setup getting started natively”

    +

    The quickest way to run the arch agnostic examples, which comprise the majority of the examples, is natively as shown at: Section 1.8.2.1, “Userland setup getting started natively”

    This section was originally moved in here from: https://github.com/cirosantilli/cpp-cheat

    @@ -32651,7 +32749,7 @@ Solution Validates: avg error less than 1.000000e-13 on all three arrays
    -

    Like other userland programs, these programs can be run as explained at: Section 1.7, “Userland setup”.

    +

    Like other userland programs, these programs can be run as explained at: Section 1.8, “Userland setup”.

    As a quick reminder, the fastest setups to get started are:

    @@ -33184,7 +33282,7 @@ When instructions do not interpret this operand encoding as the zero register, u

    Userland assembly is generally simpler, and a pre-requisite for Baremetal setup.

    -

    System-land assembly cheats will be put under: Section 1.8, “Baremetal setup”.

    +

    System-land assembly cheats will be put under: Section 1.9, “Baremetal setup”.

    @@ -38029,7 +38127,7 @@ AArch64, see Procedure Call Standard for the ARM 64-bit Architecture.

    27. Baremetal

    -

    Getting started at: Section 1.8, “Baremetal setup”

    +

    Getting started at: Section 1.9, “Baremetal setup”

    27.1. Baremetal GDB step debug

    @@ -43213,7 +43311,7 @@ CACHE2 S nyy

    33.5. Documentation

    -

    To learn how to build the documentation see: Section 1.9, “Build the documentation”.

    +

    To learn how to build the documentation see: Section 1.10, “Build the documentation”.

    33.5.1. Documentation verification

    @@ -44274,7 +44372,7 @@ git -C "$(./getvar buildroot_source_dir)" checkout -

    Source: copy-overlay

    -

    Build Buildroot is required for the same reason as described at: Section 1.1.2.2, “Your first kernel module hack”.

    +

    Build Buildroot is required for the same reason as described at: Section 1.2.2.2, “Your first kernel module hack”.

    However, since the rootfs_overlay directory does not require compilation, unlike say kernel modules, we also make it 9P available to the guest directly even without ./copy-overlay at:

    @@ -45391,93 +45489,6 @@ git push --follow-tags
    -
    -

    33.21.2. Should you waste your life with systems programming?

    -
    -

    Being the hardcore person who fully understands an important complex system such as a computer, it does have a nice ring to it doesn’t it?

    -
    -
    -

    But before you dedicate your life to this nonsense, do consider the following points:

    -
    -
    -
      -
    • -

      almost all contributions to the kernel are done by large companies, and if you are not an employee in one of them, you are likely not going to be able to do much.

      -
      -

      This can be inferred by the fact that the devices/ directory is by far the largest in the kernel.

      -
      -
      -

      The kernel is of course just an interface to hardware, and the hardware developers start developing their kernel stuff even before specs are publicly released, both to help with hardware development and to have things working when the announcement is made.

      -
      -
      -

      Furthermore, I believe that there are in-tree devices which have never been properly publicly documented. Linus is of course fine with this, since code == documentation for him, but it is not as easy for mere mortals.

      -
      -
      -

      There are some less hardware bound higher level layers in the kernel which might not require being in a hardware company, and a few people must be living off it.

      -
      -
      -

      But of course, those are heavily motivated by the underlying hardware characteristics, and it is very likely that most of the people working there were previously at a hardware company.

      -
      -
      -

      In that sense, therefore, the kernel is not as open as one might want to believe.

      -
      -
      -

      Of course, if there is some super useful and undocumented hardware that is just waiting there to be reverse engineered, then that’s a much juicier target :-)

      -
      -
      • -
    • -

      it is impossible to become rich with this knowledge.

      -
      -

      This is partly implied by the fact that you need to be in a big company to make useful low level things, and therefore you will only be a tiny cog in the engine.

      -
      -
      -

      The key problem is that the entry cost of hardware design is just too insanely high for startups in general.

      -
      -
      • -
    • -

      Is learning this the most useful thing that you think can do for society?

      -
      -

      Or are you just learning it for job security and having a nice sounding title?

      -
      -
      -

      I’m not a huge fan of the person, but I think Jobs said it right: https://www.youtube.com/watch?v=FF-tKLISfPE

      -
      -
      -

      First determine the useful goal, and then backtrack down to the most efficient thing you can do to reach it.

      -
      -
      • -
    • -

      there are two things that sadden me compared to physics-based engineering:

      -
      -
      -
      -
        -
      • -

        you will never become eternally famous. All tech disappears sooner or later, while laws of nature, at least as useful approximations, stay unchanged.

        -
        • -
      • -

        every problem that you face is caused by imperfections introduced by other humans.

        -
        -

        It is much easier to accept limitations of physics, and even natural selection in biology, which is are produced by a sentient being (?).

        -
        -
        • -
      -
      -
      -
      -
      -

      Physics-based engineering, just like low level hardware, is of course completely closed source however, since wrestling against the laws of physics is about the most expensive thing humans can do.

      -
      -
      • -
    -
    -
    -

    Are you fine with those points, and ready to continue wasting your life with this crap?

    -
    -
    -

    Good. In that case, read on, and let’s have some fun together ;-)

    -
    -

    33.22. Bibliography