From eeff0d0f4e73e8dbf4208c1c606fac29842f34e6 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ciro=20Santilli=20=E5=85=AD=E5=9B=9B=E4=BA=8B=E4=BB=B6=20?= =?UTF-8?q?=E6=B3=95=E8=BD=AE=E5=8A=9F?= Date: Mon, 15 Jul 2019 00:00:00 +0000 Subject: [PATCH] README xrefstyle: full try it out a bit, not complete --- README.adoc | 1361 ++++++++++++++++++++++++++------------------------- 1 file changed, 681 insertions(+), 680 deletions(-) diff --git a/README.adoc b/README.adoc index 9c6aee7..83a472f 100644 --- a/README.adoc +++ b/README.adoc @@ -1,5 +1,5 @@ = Linux Kernel Module Cheat -:description: The perfect emulation setup to study and develop the <> v5.1, kernel modules, <>, <> and x86_64, ARMv7 and ARMv8 <> and <> assembly, <>, <> and <>. <> and <> just work. Powered by <> and <>. Highly automated. Thoroughly documented. Automated <>. "Tested" in an Ubuntu 18.04 host. +:description: The perfect emulation setup to study and develop the xref:linux-kernel[Linux kernel] v5.1, kernel modules, xref:qemu-buildroot-setup[QEMU], xref:gem5-buildroot-setup[gem5] and x86_64, ARMv7 and ARMv8 xref:userland-assembly[userland] and xref:baremetal-setup[baremetal] assembly, xref:c[ANSI C], xref:cpp[C++] and xref:posix[POSIX]. xref:gdb[GDB] and xref:kgdb[KGDB] just work. Powered by xref:about-the-qemu-buildroot-setup[Buildroot] and xref:about-the-baremetal-setup[crosstool-NG]. Highly automated. Thoroughly documented. Automated xref:test-this-repo[tests]. "Tested" in an Ubuntu 18.04 host. :idprefix: :idseparator: - :nofooter: @@ -10,12 +10,13 @@ :toc-title: :toc: macro :toclevels: 6 +:xrefstyle: full {description} -TL;DR: <> +TL;DR: xref:qemu-buildroot-setup-getting-started[] -The source code for this page is located at: https://github.com/cirosantilli/linux-kernel-module-cheat[]. Due to https://github.com/isaacs/github/issues/1610[a GitHub limitation], this README is too long and not fully rendered on github.com. Either use: https://cirosantilli.com/linux-kernel-module-cheat or <>. +The source code for this page is located at: https://github.com/cirosantilli/linux-kernel-module-cheat[]. Due to https://github.com/isaacs/github/issues/1610[a GitHub limitation], this README is too long and not fully rendered on github.com. Either use: https://cirosantilli.com/linux-kernel-module-cheat or xref:build-the-documentation[build the docs yourself]. toc::[] @@ -23,15 +24,15 @@ toc::[] Each child section describes a possible different setup for this repo. -If you don't know which one to go for, start with <>. +If you don't know which one to go for, start with xref:qemu-buildroot-setup-getting-started[]. -Design goals of this project are documented at: <>. +Design goals of this project are documented at: xref:design-goals[]. === QEMU Buildroot setup ==== QEMU Buildroot setup getting started -This setup has been mostly tested on Ubuntu. For other host operating systems see: <>. For greater stability, consider using the <> instead of master: https://github.com/cirosantilli/linux-kernel-module-cheat/releases +This setup has been mostly tested on Ubuntu. For other host operating systems see: xref:supported-hosts[]. For greater stability, consider using the xref:release-procedure[latest release] instead of master: https://github.com/cirosantilli/linux-kernel-module-cheat/releases Reserve 12Gb of disk and run: @@ -44,18 +45,18 @@ cd linux-kernel-module-cheat You don't need to clone recursively even though we have `.git` submodules: `download-dependencies` fetches just the submodules that you need for this build to save time. -If something goes wrong, see: <> and use our issue tracker: https://github.com/cirosantilli/linux-kernel-module-cheat/issues +If something goes wrong, see: xref:common-build-issues[] and use our issue tracker: https://github.com/cirosantilli/linux-kernel-module-cheat/issues -The initial build will take a while (30 minutes to 2 hours) to clone and build, see <> for more details. +The initial build will take a while (30 minutes to 2 hours) to clone and build, see xref:benchmark-builds[] for more details. If you don't want to wait, you could also try the following faster but much more limited methods: -* <> -* <> +* xref:prebuilt[] +* xref:host[] but you will soon find that they are simply not enough if you anywhere near serious about systems programming. -After `./run`, QEMU opens up leaving you in the <>, and you can start playing with the kernel modules inside the simulated system: +After `./run`, QEMU opens up leaving you in the xref:lkmc_home[`/lkmc/` directory], and you can start playing with the kernel modules inside the simulated system: .... insmod hello.ko @@ -86,29 +87,29 @@ Quit QEMU with: Ctrl-A X .... -See also: <>. +See also: xref:quit-qemu-from-text-mode[]. All available modules can be found in the link:kernel_modules[] directory. -It is super easy to build for different <>, just use the `--arch` option: +It is super easy to build for different xref:cpu-architecture[CPU architectures], just use the `--arch` option: .... ./build --arch aarch64 --download-dependencies qemu-buildroot ./run --arch aarch64 .... -To avoid typing `--arch aarch64` many times, you can set the default arch as explained at: <> +To avoid typing `--arch aarch64` many times, you can set the default arch as explained at: xref:default-command-line-arguments[] I now urge you to read the following sections which contain widely applicable information: -* <> -* <> -* <> +* xref:run-command-after-boot[] +* xref:clean-the-build[] +* xref:build-the-documentation[] * Linux kernel -** <> -** <> +** xref:printk[] +** xref:kernel-command-line-parameters[] -Once you use <> and <>, your terminal will look a bit like this: +Once you use xref:gdb[GDB] and xref:tmux[tmux], your terminal will look a bit like this: .... [ 1.451857] input: AT Translated Set 2 keyboard as /devices/platform/i8042/s1│loading @0xffffffffc0000000: ../kernel_modules-1.0//timer.ko @@ -154,11 +155,11 @@ hello /root/.profile ==== How to hack stuff -Besides a seamless <>, this project also aims to make it effortless to modify and rebuild several major components of the system, to serve as an awesome development setup. +Besides a seamless xref:qemu-buildroot-setup-getting-started[initial build], this project also aims to make it effortless to modify and rebuild several major components of the system, to serve as an awesome development setup. ===== Your first Linux kernel hack -Let's hack up the <>, which is an easy place to start. +Let's hack up the xref:linux-kernel-entry-point[ Linux kernel entry point], which is an easy place to start. Open the file: @@ -187,7 +188,7 @@ and, surely enough, your message has appeared at the beginning of the boot: So you are now officially a Linux kernel hacker, way to go! -We could have used just link:build[] to rebuild the kernel as in the <> instead of link:build-linux[], but building just the required individual components is preferred during development: +We could have used just link:build[] to rebuild the kernel as in the xref:qemu-buildroot-setup-getting-started[initial build] instead of link:build-linux[], but building just the required individual components is preferred during development: * saves a few seconds from parsing Make scripts and reading timestamps * makes it easier to understand what is being done in more detail @@ -199,7 +200,7 @@ The link:build[] script is just a lightweight wrapper that calls the smaller bui ./build --dry-run .... -When you reach difficulties, QEMU makes it possible to easily GDB step debug the Linux kernel source code, see: <>. +When you reach difficulties, QEMU makes it possible to easily GDB step debug the Linux kernel source code, see: xref:gdb[]. ===== Your first kernel module hack @@ -225,7 +226,7 @@ insmod /mnt/9p/out_rootfs_overlay/lkmc/hello.ko and the new `pr_info` message should now show on the terminal at the end of the boot. -This works because we have a <<9p>> mount there setup by default, which mounts the host directory that contains the build outputs on the guest: +This works because we have a xref:9p[9P] mount there setup by default, which mounts the host directory that contains the build outputs on the guest: .... ls "$(./getvar out_rootfs_overlay_dir)" @@ -235,7 +236,7 @@ The fast method is slightly risky because your previously insmodded buggy kernel Such failures are however unlikely, and you should be fine if you don't see anything weird happening. -The safe way, is to fist <>, rebuild the modules, put them in the root filesystem, and then reboot: +The safe way, is to fist xref:rebuild-buildroot-while-running[quit QEMU], rebuild the modules, put them in the root filesystem, and then reboot: .... ./build-modules @@ -253,7 +254,7 @@ You can see that `./build` does that as well, by running: `--eval-after` is optional: you could just type `insmod hello.ko` in the terminal, but this makes it run automatically at the end of boot, and then drops you into a shell. -If the guest and host are the same arch, typically x86_64, you can speed up boot further with <>: +If the guest and host are the same arch, typically x86_64, you can speed up boot further with xref:kvm[KVM]: .... ./run --kvm @@ -261,7 +262,7 @@ If the guest and host are the same arch, typically x86_64, you can speed up boot All of this put together makes the safe procedure acceptably fast for regular development as well. -It is also easy to GDB step debug kernel modules with our setup, see: <>. +It is also easy to GDB step debug kernel modules with our setup, see: xref:gdb-step-debug-kernel-module[]. ===== Your first QEMU hack @@ -305,13 +306,13 @@ The only thing you can do with open source is purely functional designs with htt If you really want to develop semiconductors, your only choice is to join an university or a semiconductor company that has the EDA licenses. -See also: <>. +See also: xref:should-you-waste-your-life-with-systems-programming[]. -While hacking QEMU, you will likely want to GDB step its source. That is trivial since QEMU is just another userland program like any other, but our setup has a shortcut to make it even more convenient, see: <>. +While hacking QEMU, you will likely want to GDB step its source. That is trivial since QEMU is just another userland program like any other, but our setup has a shortcut to make it even more convenient, see: xref:debug-the-emulator[]. ===== Your first glibc hack -We use <>, and it is tracked as an unmodified submodule at link:submodules/glibc[], at the exact same version that Buildroot has it, which can be found at: https://github.com/buildroot/buildroot/blob/2018.05/package/glibc/glibc.mk#L13[package/glibc/glibc.mk]. Buildroot 2018.05 applies no patches. +We use xref:libc-choice[glibc as our default libc now], and it is tracked as an unmodified submodule at link:submodules/glibc[], at the exact same version that Buildroot has it, which can be found at: https://github.com/buildroot/buildroot/blob/2018.05/package/glibc/glibc.mk#L13[package/glibc/glibc.mk]. Buildroot 2018.05 applies no patches. Let's hack up the `puts` function: @@ -353,7 +354,7 @@ hello hacked Lol! -We can also test our hacked glibc on <> with: +We can also test our hacked glibc on xref:user-mode-simulation[user mode simulation] with: .... ./run --userland userland/c/hello.c @@ -375,9 +376,9 @@ Tested on a30ed0f047523ff2368d421ee2cce0800682c44e + 1. Have you ever felt that a single `inc` instruction was not enough? Really? Me too! -So let's hack the <>, which is part of https://en.wikipedia.org/wiki/GNU_Binutils[GNU Binutils], to add a new shiny version of `inc` called... `myinc`! +So let's hack the xref:gnu-gas-assembler[GNU GAS assembler], which is part of https://en.wikipedia.org/wiki/GNU_Binutils[GNU Binutils], to add a new shiny version of `inc` called... `myinc`! -GCC uses GNU GAS as its backend, so we will test out new mnemonic with an <> test program: link:userland/arch/x86_64/binutils_hack.c[], which is just a copy of link:userland/arch/x86_64/binutils_nohack.c[] but with `myinc` instead of `inc`. +GCC uses GNU GAS as its backend, so we will test out new mnemonic with a xref:gcc-inline-assembly[GCC inline assembly] test program: link:userland/arch/x86_64/binutils_hack.c[], which is just a copy of link:userland/arch/x86_64/binutils_nohack.c[] but with `myinc` instead of `inc`. The inline assembly is disabled with an `#ifdef`, so first modify the source to enable that. @@ -428,7 +429,7 @@ index af583ce578..3cc341f303 100644 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, .... -Finally, rebuild Binutils, userland and test our program with <>: +Finally, rebuild Binutils, userland and test our program with xref:user-mode-simulation[user mode simulation]: .... ./build-buildroot -- host-binutils-rebuild @@ -444,7 +445,7 @@ Tested on b60784d59bee993bf0de5cde6c6380dd69420dda + 1. OK, now time to hack GCC. -For convenience, let's use the <>. +For convenience, let's use xref:user-mode-simulation[user mode simulation]. If we run the program link:userland/c/gcc_hack.c[]: @@ -525,14 +526,14 @@ It was historically the first one we did, and all sections have been tested with Read the following sections for further introductory material: -* <> -* <> +* xref:introduction-to-qemu[] +* xref:introduction-to-buildroot[] === gem5 Buildroot setup ==== About the gem5 Buildroot setup -This setup is like the <>, but it uses http://gem5.org/[gem5] instead of QEMU as a system simulator. +This setup is like the xref:qemu-buildroot-setup[], but it uses http://gem5.org/[gem5] instead of QEMU as a system simulator. QEMU tries to run as fast as possible and give correct results at the end, but it does not tell us how many CPU cycles it takes to do something, just the number of instructions it ran. This kind of simulation is known as functional simulation. @@ -544,17 +545,17 @@ gem5 on the other hand, can simulate the system in more detail than QEMU, includ * caches * DRAM timing -and can therefore be used to estimate system performance, see: <> for an example. +and can therefore be used to estimate system performance, see: xref:gem5-run-benchmark[] for an example. The downside of gem5 much slower than QEMU because of the greater simulation detail. -See <> for a more thorough comparison. +See xref:gem5-vs-qemu[] for a more thorough comparison. ==== gem5 Buildroot setup getting started For the most part, if you just add the `--emulator gem5` option or `*-gem5` suffix to all commands and everything should magically work. -If you haven't built Buildroot yet for <>, you can build from the beginning with: +If you haven't built Buildroot yet for xref:qemu-buildroot-setup[], you can build from the beginning with: .... ./build --download-dependencies gem5-buildroot @@ -563,7 +564,7 @@ If you haven't built Buildroot yet for <>, you can build f If you have already built previously, don't be afraid: gem5 and QEMU use almost the same root filesystem and kernel, so `./build` will be fast. -Remember that the gem5 boot is <> than QEMU since the simulation is more detailed. +Remember that the gem5 boot is xref:benchmark-linux-kernel-boot[considerably slower] than QEMU since the simulation is more detailed. To get a terminal, either open a new shell and run: @@ -577,15 +578,15 @@ You can quit the shell without killing gem5 by typing tilde followed by a period ~. .... -If you are inside <>, which I highly recommend, you can both run gem5 stdout and open the guest terminal on a split window with: +If you are inside xref:tmux[tmux], which I highly recommend, you can both run gem5 stdout and open the guest terminal on a split window with: .... ./run --emulator gem5 --tmux .... -See also: <>. +See also: xref:tmux-gem5[]. -At the end of boot, it might not be very clear that you have the shell since some <> messages may appear in front of the prompt like this: +At the end of boot, it might not be very clear that you have the shell since some xref:printk[printk] messages may appear in front of the prompt like this: .... # <6>[ 1.215329] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x1cd486fa865, max_idle_ns: 440795259574 ns @@ -600,22 +601,22 @@ If you forgot to open the shell and gem5 exit, you can inspect the terminal outp less "$(./getvar --emulator gem5 m5out_dir)/system.pc.com_1.device" .... -More gem5 information is present at: <> +More gem5 information is present at: xref:gem5[] Good next steps are: -* <> -* <> -* <> +* xref:gem5-run-benchmark[] +* xref:m5out-directory[] +* xref:m5ops[] [[docker]] === Docker host setup This repository has been tested inside clean https://en.wikipedia.org/wiki/Docker_(software)[Docker] containers. -This is a good option if you are on a Linux host, but the native setup failed due to your weird host distribution, and you have better things to do with your life than to debug it. See also: <>. +This is a good option if you are on a Linux host, but the native setup failed due to your weird host distribution, and you have better things to do with your life than to debug it. See also: xref:supported-hosts[]. -For example, to do a <> inside Docker, run: +For example, to do a xref:qemu-buildroot-setup[QEMU buildroot setup] inside Docker, run: .... sudo apt-get install docker @@ -673,13 +674,13 @@ To actually delete the Docker build, run on host: # sudo rm -rf out.docker .... -To use <> from inside Docker, you need a second shell inside the container. You can either do that from another shell with: +To use xref:gdb[GDB] from inside Docker, you need a second shell inside the container. You can either do that from another shell with: .... ./run-docker sh .... -or even better, by starting a <> session inside the container. We install `tmux` by default in the container. +or even better, by starting a xref:tmux[tmux] session inside the container. We install `tmux` by default in the container. You can also start a second shell and run a command in it at the same time with: @@ -687,7 +688,7 @@ You can also start a second shell and run a command in it at the same time with: ./run-docker sh -- ./run-gdb start_kernel .... -To use <> from Docker, run: +To use xref:qemu-graphic-mode[QEMU graphic mode] from Docker, run: .... ./run --graphic --vnc @@ -717,26 +718,26 @@ We don't currently provide a full prebuilt because it would be too big to host f Our prebuilts currently include: -* <> binaries +* xref:qemu-buildroot-setup[] binaries ** Linux kernel ** root filesystem -* <> binaries for QEMU +* xref:baremetal-setup[] binaries for QEMU -For more details, see our our <>. +For more details, see our our xref:release[release procedure]. Advantage of this setup: saves time and disk space on the initial install, which is expensive in largely due to building the toolchain. The limitations are severe however: -* can't <>, since the source and cross toolchain with GDB are not available. Buildroot cannot easily use a host toolchain: <>. +* can't xref:gdb[GDB step debug the kernel], since the source and cross toolchain with GDB are not available. Buildroot cannot easily use a host toolchain: xref:prebuilt-toolchain[]. + Maybe we could work around this by just downloading the kernel source somehow, and using a host prebuilt GDB, but we felt that it would be too messy and unreliable. -* you won't get the latest version of this repository. Our <> attempt to automate builds failed, and storing a release for every commit would likely make GitHub mad at us anyways. -* <> is not currently supported. The major blocking point is how to avoid distributing the kernel images twice: once for gem5 which uses `vmlinux`, and once for QEMU which uses `arch/*` images, see also: +* you won't get the latest version of this repository. Our xref:travis[Travis] attempt to automate builds failed, and storing a release for every commit would likely make GitHub mad at us anyways. +* xref:gem5[] is not currently supported. The major blocking point is how to avoid distributing the kernel images twice: once for gem5 which uses `vmlinux`, and once for QEMU which uses `arch/*` images, see also: ** https://github.com/cirosantilli/linux-kernel-module-cheat/issues/79 -** <>. +** xref:vmlinux-vs-bzimage-vs-zimage-vs-image[]. -This setup might be good enough for those developing simulators, as that requires less image modification. But once again, if you are serious about this, why not just let your computer build the <> while you take a coffee or a nap? :-) +This setup might be good enough for those developing simulators, as that requires less image modification. But once again, if you are serious about this, why not just let your computer build the xref:qemu-buildroot-setup[full featured setup] while you take a coffee or a nap? :-) ==== Prebuilt setup getting started @@ -773,9 +774,9 @@ Be saner and use our custom built QEMU instead: ./run .... -This also allows you to <> if you're into that sort of thing. +This also allows you to xref:your-first-qemu-hack[modify QEMU] if you're into that sort of thing. -To build the kernel modules as in <> do: +To build the kernel modules as in xref:your-first-kernel-module-hack[] do: .... git submodule update --depth 1 --init --recursive "$(./getvar linux_source_dir)" @@ -784,7 +785,7 @@ git submodule update --depth 1 --init --recursive "$(./getvar linux_source_dir)" ./run .... -TODO: for now the only way to test those modules out without <> is with 9p, since we currently rely on Buildroot to manipulate the root filesystem. +TODO: for now the only way to test those modules out without xref:qemu-buildroot-setup-getting-started[building Buildroot] is with 9p, since we currently rely on Buildroot to manipulate the root filesystem. Command explanation: @@ -810,7 +811,7 @@ sudo apt-get install qemu-utils ./run --emulator gem5 --qemu-which host .... -`qemu-utils` is required because we currently distribute `.qcow2` files which <>, so we need `qemu-img` to extract them first. +`qemu-utils` is required because we currently distribute `.qcow2` files which xref:gem5-qcow2[gem5 can't handle], so we need `qemu-img` to extract them first. The Linux kernel is required for `extract-vmlinux` to convert the compressed kernel image which QEMU understands into the raw vmlinux that gem5 understands: https://superuser.com/questions/298826/how-do-i-uncompress-vmlinuz-to-vmlinux //// @@ -821,9 +822,9 @@ The Linux kernel is required for `extract-vmlinux` to convert the compressed ker ==== About the Ubuntu guest setup -This setup is similar to <>, but instead of using Buildroot for the root filesystem, it downloads an Ubuntu image with Docker, and uses that as the root filesystem. +This setup is similar to xref:prebuilt[], but instead of using Buildroot for the root filesystem, it downloads an Ubuntu image with Docker, and uses that as the root filesystem. -The rationale for choice of Ubuntu as a second distribution in addition to Buildroot can be found at: <> +The rationale for choice of Ubuntu as a second distribution in addition to Buildroot can be found at: xref:linux-distro-choice[] Advantages over Buildroot: @@ -870,7 +871,7 @@ It has however severe limitations: ** your disk could get erased. Yes, this can also happen with `sudo` from userland. But you should not use `sudo` when developing newbie programs. And for the kernel you don't have the choice not to use `sudo`. ** even more subtle system corruption such as https://unix.stackexchange.com/questions/78858/cannot-remove-or-reinsert-kernel-module-after-error-while-inserting-it-without-r[not being able to rmmod] * can't control which hardware is used, notably the CPU architecture -* can't step debug it with <> easily. The alternatives are https://en.wikipedia.org/wiki/JTAG[JTAG] or <>, but those are less reliable, and require extra hardware. +* can't step debug it with xref:gdb[GDB] easily. The alternatives are https://en.wikipedia.org/wiki/JTAG[JTAG] or xref:kgdb[KGDB], but those are less reliable, and require extra hardware. Still interested? @@ -946,8 +947,8 @@ dmesg In order to test the kernel and emulators, userland content in the form of executables and scripts is of course required, and we store it mostly under: * link:userland/[] -* <> -* <> +* xref:rootfs_overlay[] +* xref:add-new-buildroot-packages[] When we started this repository, it only contained content that interacted very closely with the kernel, or that had required performance analysis. @@ -957,33 +958,33 @@ Therefore, we decided to consolidate other userland tutorials that we had scatte Notable userland content included / moving into this repository includes: -* <> -* <> -* <> -* <> -* https://github.com/cirosantilli/algorithm-cheat TODO will be good to move here for performance analysis <> +* xref:userland-assembly[] +* xref:c[] +* xref:cpp[] +* xref:posix[] +* https://github.com/cirosantilli/algorithm-cheat TODO will be good to move here for performance analysis xref:gem5-run-benchmark[with gem5] ==== Userland setup getting started There are several ways to run our userland content, notably: -* natively on the host as shown at: <> +* natively on the host as shown at: xref:userland-setup-getting-started-natively[] + Can only run examples compatible with your host CPU architecture and OS, but has the fastest setup and runtimes. * from user mode simulation with: + -- -** the host prebuilt toolchain: <> -** the Buildroot toolchain you built yourself: <> +** the host prebuilt toolchain: xref:userland-setup-getting-started-with-prebuilt-toolchain-and-qemu-user-mode[] +** the Buildroot toolchain you built yourself: xref:qemu-user-mode-getting-started[] -- + This setup: + -- ** can run most examples, including those for other CPU architectures, with the notable exception of examples that rely on kernel modules -** can run reproducible approximate performance experiments with gem5, see e.g. <> +** can run reproducible approximate performance experiments with gem5, see e.g. xref:bst-vs-heap-vs-hashmap[] -- -* from full system simulation as shown at: <>. +* from full system simulation as shown at: xref:qemu-buildroot-setup-getting-started[]. + This is the most reproducible and controlled environment, and all examples work there. But also the slower one to setup. @@ -1021,7 +1022,7 @@ cd userland/c ./test .... -As mentioned at <>, tests under link:userland/libs[] require certain optional libraries to be installed, and are not built or tested by default. +As mentioned at xref:user-mode-tests[], tests under link:userland/libs[] require certain optional libraries to be installed, and are not built or tested by default. You can install those libraries with: @@ -1051,7 +1052,7 @@ Some CLI options have more specialized flags, e.g. `-O` optimization level: ./build --optimization-level 3 --force-rebuild .... -See also <> for `--static`. +See also xref:user-mode-static-executables[] for `--static`. The `build` scripts inside link:userland/[] are just symlinks to link:build-userland-in-tree[] which you can also use from toplevel as: @@ -1069,7 +1070,7 @@ The `build` scripts inside link:userland/[] are just symlinks to link:build-user So you can use any option supported by `build-userland` script freely with `build-userland-in-tree` and `build`. -The situation is analogous for link:userland/test[], link:test-executables-in-tree[] and link:test-executables[], which are further documented at: <>. +The situation is analogous for link:userland/test[], link:test-executables-in-tree[] and link:test-executables[], which are further documented at: xref:user-mode-tests[]. Do a more clean out-of-tree build instead and run the program: @@ -1080,7 +1081,7 @@ Do a more clean out-of-tree build instead and run the program: Here we: -* put the host executables in a separate <> to avoid conflict with Buildroot builds. +* put the host executables in a separate xref:build-variants[build-variant] to avoid conflict with Buildroot builds. * ran with the `--emulator native` option to run the program natively In this case you can debub the program with: @@ -1089,11 +1090,11 @@ In this case you can debub the program with: ./run --debug-vm --emulator native --userland userland/c/hello.c --userland-build-id host .... -as shown at: <>, although direct GDB host usage works as well of course. +as shown at: xref:debug-the-emulator[], although direct GDB host usage works as well of course. ===== Userland setup getting started with prebuilt toolchain and QEMU user mode -If you are lazy to built the Buildroot toolchain and QEMU, but want to run e.g. ARM <> in <>, you can get away on Ubuntu 18.04 with just: +If you are lazy to built the Buildroot toolchain and QEMU, but want to run e.g. ARM xref:userland-assembly[userland assembly] in xref:user-mode-simulation[user mode simulation], you can get away on Ubuntu 18.04 with just: .... sudo apt-get install gcc-aarch64-linux-gnu qemu-system-aarch64 @@ -1115,10 +1116,10 @@ where: * `--gcc-which host`: use the host toolchain. + -We must pass this to `./run` as well because QEMU must know which dynamic libraries to use. See also: <>. -* `--userland-build-id host`: put the host built into a <> +We must pass this to `./run` as well because QEMU must know which dynamic libraries to use. See also: xref:user-mode-static-executables[]. +* `--userland-build-id host`: put the host built into a xref:build-variants[build variant] -This present the usual trade-offs of using prebuilts as mentioned at: <>. +This present the usual trade-offs of using prebuilts as mentioned at: xref:prebuilt[]. Other functionality are analogous, e.g. testing: @@ -1131,7 +1132,7 @@ Other functionality are analogous, e.g. testing: ; .... -and <>: +and xref:user-mode-gdb[user mode GDB]: .... ./run \ @@ -1147,11 +1148,11 @@ and <>: ===== Userland setup getting started full system -First ensure that <> is working. +First ensure that xref:qemu-buildroot-setup[] is working. After doing that setup, you can already execute your userland programs from inside QEMU: the only missing step is how to rebuild executables and run them. -And the answer is exactly analogous to what is shown at: <> +And the answer is exactly analogous to what is shown at: xref:your-first-kernel-module-hack[] For example, if we modify link:userland/c/hello.c[] to print out something different, we can just rebuild it with: @@ -1161,7 +1162,7 @@ For example, if we modify link:userland/c/hello.c[] to print out something diffe Source: link:build-userland[]. `./build` calls that script automatically for us when doing the initial full build. -Now, run the program either without rebooting use the <<9p>> mount: +Now, run the program either without rebooting use the xref:9p[9P] mount: .... /mnt/9p/out_rootfs_overlay/c/hello.out @@ -1193,7 +1194,7 @@ This setup allows you to make a tiny OS and that runs just a few instructions, u You can also use C and a subset of the C standard library because we enable https://en.wikipedia.org/wiki/Newlib[Newlib] by default. See also: https://electronics.stackexchange.com/questions/223929/c-standard-libraries-on-bare-metal/400077#400077 -Our C bare-metal compiler is built with https://github.com/crosstool-ng/crosstool-ng[crosstool-NG]. If you have already built <> previously, you will end up with two GCCs installed. Unfortunately I don't see a solution for this, since we need separate toolchains for Newlib on baremetal and glibc on Linux: https://stackoverflow.com/questions/38956680/difference-between-arm-none-eabi-and-arm-linux-gnueabi/38989869#38989869 +Our C bare-metal compiler is built with https://github.com/crosstool-ng/crosstool-ng[crosstool-NG]. If you have already built xref:qemu-buildroot-setup[Buildroot] previously, you will end up with two GCCs installed. Unfortunately I don't see a solution for this, since we need separate toolchains for Newlib on baremetal and glibc on Linux: https://stackoverflow.com/questions/38956680/difference-between-arm-none-eabi-and-arm-linux-gnueabi/38989869#38989869 ==== Baremetal setup getting started @@ -1206,13 +1207,13 @@ For example, to run link:baremetal/arch/aarch64/dump_regs.c[] in QEMU do: ./run --arch aarch64 --baremetal baremetal/arch/aarch64/dump_regs.c .... -And the terminal prints the values of certain system registers. This example prints registers that are only accessible from <> or higher, and thus could not be run in userland. +And the terminal prints the values of certain system registers. This example prints registers that are only accessible from xref:arm-exception-levels[EL1] or higher, and thus could not be run in userland. -In addition to the examples under link:baremetal/[], several of the <> can also be run in baremetal! This is largely due to the <>. +In addition to the examples under link:baremetal/[], several of the xref:userland-content[userland examples] can also be run in baremetal! This is largely due to the xref:about-the-baremetal-setup[awesomeness of Newlib]. -The examples that work include most <> that don't rely on complicated syscalls such as threads, and almost all the <> examples. +The examples that work include most xref:c[C examples] that don't rely on complicated syscalls such as threads, and almost all the xref:userland-assembly[] examples. -The exact list of userland programs that work in baremetal is specified in <> with the `baremetal` property, but you can also easily find it out with a <>: +The exact list of userland programs that work in baremetal is specified in xref:path-properties[] with the `baremetal` property, but you can also easily find it out with a xref:baremetal-tests[baremetal test dry run]: .... ./test-executables --arch aarch64 --dry-run --mode baremetal @@ -1289,19 +1290,19 @@ To use gem5 instead of QEMU do: ./run --arch aarch64 --baremetal userland/c/hello.c --emulator gem5 .... -and then <> open a shell with: +and then xref:qemu-buildroot-setup[as usual] open a shell with: .... ./gem5-shell .... -Or as usual, <> users can do both in one go with: +Or as usual, xref:tmux[] users can do both in one go with: .... ./run --arch aarch64 --baremetal userland/c/hello.c --emulator gem5 --tmux .... -TODO: the carriage returns are a bit different than in QEMU, see: <>. +TODO: the carriage returns are a bit different than in QEMU, see: xref:gem5-baremetal-carriage-return[]. Note that `./build-baremetal` requires the `--emulator gem5` option, and generates separate executable images for both, as can be seen from: @@ -1317,7 +1318,7 @@ echo "$(./getvar --arch aarch64 --emulator qemu image)" echo "$(./getvar --arch aarch64 --emulator gem5 image)" .... -The reason for that is that on baremetal we don't parse the <> from memory like the Linux kernel does, which tells the kernel for example the UART address, and many other system parameters. +The reason for that is that on baremetal we don't parse the xref:device-tree[device tress] from memory like the Linux kernel does, which tells the kernel for example the UART address, and many other system parameters. `gem5` also supports the `RealViewPBX` machine, which represents an older hardware compared to the default `VExpress_GEM5_V1`: @@ -1335,14 +1336,14 @@ echo "$(./getvar --arch aarch64 --baremetal userland/c/hello.c --emulator gem5 - But just stick to newer and better `VExpress_GEM5_V1` unless you have a good reason to use `RealViewPBX`. -When doing baremetal programming, it is likely that you will want to learn userland assembly first, see: <>. +When doing baremetal programming, it is likely that you will want to learn userland assembly first, see: xref:userland-assembly[]. -For more information on baremetal, see the section: <>. +For more information on baremetal, see the section: xref:baremetal[]. The following subjects are particularly important: -* <> -* <> +* xref:tracing[] +* xref:baremetal-gdb-step-debug[] [[gdb]] == GDB step debug @@ -1454,7 +1455,7 @@ When you hit `Ctrl-C`, if we happen to be inside kernel code at that point, whic tmux just makes things even more fun by allowing us to see both the terminal for: * emulator stdout -* <> +* xref:gdb[] at once without dragging windows around! @@ -1511,7 +1512,7 @@ This is equivalent to: ./run-gdb start_kernel .... -Due to Python's CLI parsing quicks, if the link:run-gdb[] arguments start with a dash `-`, you have to use the `=` sign, e.g. to <>: +Due to Python's CLI parsing quicks, if the link:run-gdb[] arguments start with a dash `-`, you have to use the `=` sign, e.g. to xref:gdb-step-debug-early-boot[]: .... ./run --gdb --tmux-args=--no-continue @@ -1704,7 +1705,7 @@ For the next time, you can also put a breakpoint there directly: ./run-gdb init/main.c:833 .... -How we found this out: first we got <> working, and then we did a `bt`. AKA cheating :-) +How we found this out: first we got xref:gdb-module_init-calculate-entry-address[] working, and then we did a `bt`. AKA cheating :-) ===== GDB module_init calculate entry address @@ -1784,7 +1785,7 @@ then hitting: n .... -does not break, and insertion happens, likely because of optimizations? <> +does not break, and insertion happens, likely because of optimizations? xref:kernel-o0[] Then we try: @@ -1800,7 +1801,7 @@ fin also fails to break! -Finally, in despair we notice that <> prints the kernel load address as explained at <>. +Finally, in despair we notice that xref:pr_debug[] prints the kernel load address as explained at xref:bypass-lx-symbols[]. So, if we set a breakpoint just after that message is printed by searching where that happens on the Linux source code, we must be able to get the correct load address before `init_module` happens. @@ -1818,7 +1819,7 @@ And just adding: asm( " int $3"); .... -directly gives an <> as I'd expect. +directly gives an xref:oops[oops] as I'd expect. ==== Bypass lx-symbols @@ -1854,7 +1855,7 @@ add-symbol-file ../../../rootfs_overlay/x86_64/timer.ko 0xffffffffc0000000 0xffffffffc0000000 .... -Alternatively, if the module panics before you can read `/proc/modules`, there is a <> which shows the load address: +Alternatively, if the module panics before you can read `/proc/modules`, there is a xref:pr_debug[] which shows the load address: .... echo 8 > /proc/sys/kernel/printk @@ -1891,9 +1892,9 @@ Maybe it is because they are being copied around at specific locations instead o See also: https://stackoverflow.com/questions/2589845/what-are-the-first-operations-that-the-linux-kernel-executes-on-boot -<> with `--debug-flags=Exec` does show the right symbols however! So in the worst case, we can just read their source. Amazing. +xref:gem5-tracing[] with `--debug-flags=Exec` does show the right symbols however! So in the worst case, we can just read their source. Amazing. -v4.19 also added a `CONFIG_HAVE_KERNEL_UNCOMPRESSED=y` option for having the kernel uncompressed which could make following the startup easier, but it is only available on s390. `aarch64` however is already uncompressed by default, so might be the easiest one. See also: <>. +v4.19 also added a `CONFIG_HAVE_KERNEL_UNCOMPRESSED=y` option for having the kernel uncompressed which could make following the startup easier, but it is only available on s390. `aarch64` however is already uncompressed by default, so might be the easiest one. See also: xref:vmlinux-vs-bzimage-vs-zimage-vs-image[]. ==== GDB step debug early boot by address @@ -1941,9 +1942,9 @@ QEMU's `-gdb` GDB breakpoints are set on virtual addresses, so you can in theory * https://stackoverflow.com/questions/26271901/is-it-possible-to-use-gdb-and-qemu-to-debug-linux-user-space-programs-and-kernel * https://stackoverflow.com/questions/16273614/debug-init-on-qemu-using-gdb -You will generally want to use <> for this as it is more reliable, but this method can overcome the following limitations of `gdbserver`: +You will generally want to use xref:gdbserver[] for this as it is more reliable, but this method can overcome the following limitations of `gdbserver`: -* the emulator does not support host to guest networking. This seems to be the case for gem5: <> +* the emulator does not support host to guest networking. This seems to be the case for gem5: xref:gem5-host-to-guest-networking[] * cannot see the start of the `init` process easily * `gdbserver` alters the working of the kernel, and makes your run less representative @@ -1951,7 +1952,7 @@ Known limitations of direct userland debugging: * the kernel might switch context to another process or to the kernel itself e.g. on a system call, and then TODO confirm the PIC would go to weird places and source code would be missing. + -Solutions to this are being researched at: <>. +Solutions to this are being researched at: xref:lx-ps[]. * TODO step into shared libraries. If I attempt to load them explicitly: + .... @@ -1984,7 +1985,7 @@ Alternatively, we could also pass the full path to the executable: ./run-gdb --userland "$(./getvar userland_build_dir)/posix/count.out" main .... + -Path resolution is analogous to <>. +Path resolution is analogous to xref:baremetal-setup-getting-started[that of `./run --baremetal`]. Then, as soon as boot ends, we are left inside a debug session that looks just like what `gdbserver` would produce. @@ -2018,7 +2019,7 @@ BusyBox default init process: ./run-gdb --userland "$(./getvar buildroot_build_build_dir)"/busybox-*/busybox init_main .... -`init` cannot be debugged with <> without modifying the source, or else `/sbin/init` exits early with: +`init` cannot be debugged with xref:gdbserver[] without modifying the source, or else `/sbin/init` exits early with: .... "must be run as PID 1" @@ -2049,7 +2050,7 @@ This is the least reliable setup as there might be other processes that use the [[gdb-step-debug-userland-non-init-without-gdb-wait]] ===== GDB step debug userland non-init without --gdb-wait -TODO: if I try <> without `--gdb-wait` and the `break main` that we do inside `./run-gdb` says: +TODO: if I try xref:gdb-step-debug-userland-non-init[] without `--gdb-wait` and the `break main` that we do inside `./run-gdb` says: .... Cannot access memory at address 0x10604 @@ -2153,7 +2154,7 @@ The implementation is described at: https://stackoverflow.com/questions/46415059 === GDB step debug multicore userland -For a more minimal baremetal multicore setup, see: <>. +For a more minimal baremetal multicore setup, see: xref:arm-multicore[]. We can set and get which cores the Linux kernel allows a program to run on with `sched_getaffinity` and `sched_setaffinity`: @@ -2179,7 +2180,7 @@ Which shows us that: ** the process was randomly assigned to run on core 1 (the second one) as shown by `sched_getcpu = 1`. If we run this several times, it will also run on core 0 sometimes. * then we restrict the affinity to just core 0, and we see that the program was actually moved to core 0 -The number of cores is modified as explained at: <> +The number of cores is modified as explained at: xref:number-of-cores[] `taskset` from the util-linux package sets the initial core affinity of a program: @@ -2202,7 +2203,7 @@ sched_getcpu = 0 so we see that the affinity was restricted to the second core from the start. -Let's do a QEMU observation to justify this example being in the repository with <>. +Let's do a QEMU observation to justify this example being in the repository with xref:gdb-step-debug-userland-non-init[userland breakpoints]. We will run our `./linux/sched_getaffinity.out` infinitely many times, on core 0 and core 1 alternatively: @@ -2278,7 +2279,7 @@ Show dmesg: lx-dmesg .... -Show the <>: +Show the xref:kernel-command-line-parameters[]: .... lx-cmdline @@ -2417,7 +2418,7 @@ Entering kdb (current=0x(____ptrval____), pid 1) on processor 0 due to Keyboard KGDB expects the connection at `ttyS1`, our second serial port after `ttyS0` which contains the terminal. -The last line is the KDB prompt, and is covered at: <>. Typing now shows nothing because that prompt is expecting input from `ttyS1`. +The last line is the KDB prompt, and is covered at: xref:kdb[]. Typing now shows nothing because that prompt is expecting input from `ttyS1`. Instead, we connect to the serial port `ttyS1` with GDB: @@ -2467,7 +2468,7 @@ See also: TODO: we would need a second serial for KGDB to work, but it is not currently supported on `arm` and `aarch64` with `-M virt` that we use: https://unix.stackexchange.com/questions/479085/can-qemu-m-virt-on-arm-aarch64-have-multiple-serial-ttys-like-such-as-pl011-t/479340#479340 -One possible workaround for this would be to use <>. +One possible workaround for this would be to use xref:kdb-arm[]. Main more generic question: https://stackoverflow.com/questions/14155577/how-to-use-kgdb-on-arm @@ -2542,7 +2543,7 @@ The other KDB commands allow you to step instructions, view memory, registers an ==== KDB graphic -You can also use KDB directly from the <> window with: +You can also use KDB directly from the xref:graphics[graphic] window with: .... ./run --graphic --kdb @@ -2618,7 +2619,7 @@ Bibliography: https://reverseengineering.stackexchange.com/questions/8829/cross- === gdbserver BusyBox -Analogous to <>: +Analogous to xref:gdb-step-debug-userland-processes[]: .... ./gdbserver.sh ls @@ -2803,13 +2804,13 @@ BusyBox provides its own minimalistic init implementation which Buildroot, and t The `init` program can be either an executable shell text file, or a compiled ELF file. It becomes easy to accept this once you see that the `exec` system call handles both cases equally: https://unix.stackexchange.com/questions/174062/can-the-init-process-be-a-shell-script-in-linux/395375#395375 -The `init` executable is searched for in a list of paths in the root filesystem, including `/init`, `/sbin/init` and a few others. For more details see: <> +The `init` executable is searched for in a list of paths in the root filesystem, including `/init`, `/sbin/init` and a few others. For more details see: xref:path-to-init[] === Replace init To have more control over the system, you can replace BusyBox's init with your own. -The most direct way to replace `init` with our own is to just use the `init=` <> directly: +The most direct way to replace `init` with our own is to just use the `init=` xref:kernel-command-line-parameters[command line parameter] directly: .... ./run --kernel-cli 'init=/lkmc/count.sh' @@ -2817,7 +2818,7 @@ The most direct way to replace `init` with our own is to just use the `init=` << This just counts every second forever and does not give you a shell. -This method is not very flexible however, as it is hard to reliably pass multiple commands and command line arguments to the init with it, as explained at: <>. +This method is not very flexible however, as it is hard to reliably pass multiple commands and command line arguments to the init with it, as explained at: xref:init-environment[]. For this reason, we have created a more robust helper method with the `--eval` option: @@ -2833,9 +2834,9 @@ It is basically a shortcut for: Source: link:rootfs_overlay/lkmc/eval_base64.sh[]. -This allows quoting and newlines by base64 encoding on host, and decoding on guest, see: <>. +This allows quoting and newlines by base64 encoding on host, and decoding on guest, see: xref:kernel-command-line-parameters-escaping[]. -It also automatically chooses between `init=` and `rcinit=` for you, see: <> +It also automatically chooses between `init=` and `rcinit=` for you, see: xref:path-to-init[] `--eval` replaces BusyBox' init completely, which makes things more minimal, but also has has the following consequences: @@ -2850,7 +2851,7 @@ It also automatically chooses between `init=` and `rcinit=` for you, see: <> +The best way to overcome those limitations is to use: xref:init-busybox[] If the script is large, you can add it to a gitignored file and pass that to `--eval` as in: @@ -2979,7 +2980,7 @@ Scripts under `/etc/init.d` are run by `/etc/init.d/rcS`, which gets called by t The init is selected at: -* initrd or initramfs system: `/init`, a custom one can be set with the `rdinit=` <> +* initrd or initramfs system: `/init`, a custom one can be set with the `rdinit=` xref:kernel-command-line-parameters[kernel command line parameter] * otherwise: default is `/sbin/init`, followed by some other paths, a custom one can be set with `init=` More details: https://unix.stackexchange.com/questions/30414/what-can-make-passing-init-path-to-program-to-the-kernel-not-start-program-as-i/430614#430614 @@ -3101,7 +3102,7 @@ The shell knows that it is a login shell if the first character of `argv[0]` is When we use just `init=/bin/sh`, the Linux kernel sets `argv[0]` to `/bin/sh`, which does not start with `-`. -However, if you use `::respawn:-/bin/sh` on inttab described at <>, BusyBox' init sets `argv[0][0]` to `-`, and so does `getty`. This can be observed with: +However, if you use `::respawn:-/bin/sh` on inttab described at xref:tty[], BusyBox' init sets `argv[0][0]` to `-`, and so does `getty`. This can be observed with: .... cat /proc/$$/cmdline @@ -3138,7 +3139,7 @@ Instead, we used the QEMU `-initrd` option to point to the `.cpio` filesystem th Try removing that `-initrd` option to watch the kernel panic without rootfs at the end of boot. -When using `.cpio`, there can be no <> across boots, since all file operations happen in memory in a tmpfs: +When using `.cpio`, there can be no xref:disk-persistency[filesystem persistency] across boots, since all file operations happen in memory in a tmpfs: .... date >f @@ -3149,7 +3150,7 @@ cat f which can be good for automated tests, as it ensures that you are using a pristine unmodified system image every time. -Not however that we already disable disk persistency by default on ext2 filesystems even without `--initrd`: <>. +Not however that we already disable disk persistency by default on ext2 filesystems even without `--initrd`: xref:disk-persistency[]. One downside of this method is that it has to put the entire filesystem into memory, and could lead to a panic: @@ -3193,7 +3194,7 @@ Related: https://stackoverflow.com/questions/6405083/initrd-and-booting-the-linu === initramfs -initramfs is just like <>, but you also glue the image directly to the kernel image itself using the kernel's build system. +initramfs is just like xref:initrd[], but you also glue the image directly to the kernel image itself using the kernel's build system. Try it out with: @@ -3226,7 +3227,7 @@ Don't forget that to stop using initramfs, you must rebuild the kernel without ` ./run .... -Alternatively, consider using <> if you need to switch between initramfs and non initramfs often: +Alternatively, consider using xref:linux-kernel-build-variants[] if you need to switch between initramfs and non initramfs often: .... ./build-buildroot --initramfs @@ -3253,7 +3254,7 @@ TODO: understand `/dev/root` better: ==== /dev/root -See: <> +See: xref:rootfs[] === gem5 initrd @@ -3277,9 +3278,9 @@ However, it didn't: boot fails at the end because it does not see the initramfs, VFS: Cannot open root device "sda" or unknown-block(8,0): error -5 .... -We think that this might be because gem5 boots directly `vmlinux`, and not from the final compressed images that contain the attached rootfs such as `bzImage`, which is what QEMU does, see also: <>. +We think that this might be because gem5 boots directly `vmlinux`, and not from the final compressed images that contain the attached rootfs such as `bzImage`, which is what QEMU does, see also: xref:vmlinux-vs-bzimage-vs-zimage-vs-image[]. -To do this failed test, we automatically pass a dummy disk image as of gem5 7fa4c946386e7207ad5859e8ade0bbfc14000d91 since the scripts don't handle a missing `--disk-image` well, much like is currently done for <>. +To do this failed test, we automatically pass a dummy disk image as of gem5 7fa4c946386e7207ad5859e8ade0bbfc14000d91 since the scripts don't handle a missing `--disk-image` well, much like is currently done for xref:baremetal[]. Interestingly, using initramfs significantly slows down the gem5 boot, even though it did not work. For example, we've observed a 4x slowdown of as 17062a2e8b6e7888a14c3506e9415989362c58bf for aarch64. This must be because expanding the large attached CPIO must be expensive. We can clearly see from the kernel logs that the kernel just hangs at a point after the message `PCI: CLS 0 bytes, default 64` for a long time before proceeding further. @@ -3287,7 +3288,7 @@ Interestingly, using initramfs significantly slows down the gem5 boot, even thou The device tree is a Linux kernel defined data structure that serves to inform the kernel how the hardware is setup. -<> contains a minimal runnable example of device tree manipulation. +xref:platform_device[] contains a minimal runnable example of device tree manipulation. Device trees serve to reduce the need for hardware vendors to patch the kernel: they just provide a device tree file instead, which is much simpler. @@ -3428,7 +3429,7 @@ Since emulators know everything about the hardware, they can automatically gener This is the case for both QEMU and gem5. -For example, if we increase the <> to 2: +For example, if we increase the xref:number-of-cores[number of cores] to 2: .... ./run --arch aarch64 --cpus 2 @@ -3451,13 +3452,13 @@ You can dump the DTB QEMU generated with: as mentioned at: https://lists.gnu.org/archive/html/qemu-discuss/2017-02/msg00051.html -<> 2a9573f5942b5416fb0570cf5cb6cdecba733392 can also generate its own DTB. +xref:gem5-fs_biglittle[] 2a9573f5942b5416fb0570cf5cb6cdecba733392 can also generate its own DTB. -gem5 can generate DTBs on ARM with `--generate-dtb`. The generated DTB is placed in the <> named as `system.dtb`. +gem5 can generate DTBs on ARM with `--generate-dtb`. The generated DTB is placed in the xref:m5out-directory[] named as `system.dtb`. == KVM -https://en.wikipedia.org/wiki/Kernel-based_Virtual_Machine[KVM] is Linux kernel interface that <> execution of virtual machines. +https://en.wikipedia.org/wiki/Kernel-based_Virtual_Machine[KVM] is Linux kernel interface that xref:benchmark-linux-kernel-boot[greatly speeds up] execution of virtual machines. You can make QEMU or gem5 by passing enabling KVM with: @@ -3478,12 +3479,12 @@ Therefore, KVM only works if you the host architecture is the same as the guest We don't enable KVM by default because: * it limits visibility, since more things are running natively: -** can't use <> -** can't do <> -** on gem5, you lose <> and therefor any notion of performance -* QEMU kernel boots are already <> for most purposes without it +** can't use xref:gdb[GDB] +** can't do xref:tracing[instruction tracing] +** on gem5, you lose xref:gem5-run-benchmark[cycle counts] and therefor any notion of performance +* QEMU kernel boots are already xref:benchmark-linux-kernel-boot[fast enough] for most purposes without it -One important use case for KVM is to fast forward gem5 execution, often to skip boot, take a <>, and then move on to a more detailed and slow simulation +One important use case for KVM is to fast forward gem5 execution, often to skip boot, take a xref:gem5-checkpoint[], and then move on to a more detailed and slow simulation === KVM arm @@ -3504,7 +3505,7 @@ cd linux-kernel-module-cheat sudo ./setup -y .... -and then proceed exactly as in <>. +and then proceed exactly as in xref:prebuilt[]. We don't want to build the full Buildroot image inside the VM as that would be way too slow, thus the recommendation for the prebuilt setup. @@ -3514,7 +3515,7 @@ TODO: do the right thing and cross compile QEMU and gem5. gem5's Python parts mi Both QEMU and gem5 have an user mode simulation mode in addition to full system simulation that we consider elsewhere in this project. -In QEMU, it is called just <>, and in gem5 it is called <>. +In QEMU, it is called just xref:qemu-user-mode-getting-started["user mode"], and in gem5 it is called xref:gem5-syscall-emulation-mode[syscall emulation mode]. In both, the basic idea is the same. @@ -3524,7 +3525,7 @@ Instead of simulating the full system, it translates normal instructions like in Advantages over full system simulation: -* the simulation may <> since you don't have to simulate the Linux kernel and several device models +* the simulation may xref:user-mode-vs-full-system-benchmark[run faster] since you don't have to simulate the Linux kernel and several device models * you don't need to build your own kernel or root filesystem, which saves time. You still need a toolchain however, but the pre-packaged ones may work fine. Disadvantages: @@ -3538,7 +3539,7 @@ It may still work even if that is not the case, but could fail is a missing syst The target Linux kernel of the executable is a GCC toolchain build-time configuration. ** emulator implementers have to keep up with libc changes, some of which break even a C hello world due setup code executed before main. + -See also: <> +See also: xref:user-mode-simulation-with-glibc[] * cannot be used to test the Linux kernel or any devices, and results are less representative of a real system since we are faking more === QEMU user mode getting started @@ -3561,9 +3562,9 @@ asdf qw er .... -`./run --userland` path resolution is analogous to <>. +`./run --userland` path resolution is analogous to xref:baremetal-setup-getting-started[that of `./run --baremetal`]. -`./build user-mode-qemu` first builds Buildroot, and then runs `./build-userland`, which is further documented at: <>. It also builds QEMU. If you ahve already done a <> previously, this will be very fast. +`./build user-mode-qemu` first builds Buildroot, and then runs `./build-userland`, which is further documented at: xref:userland-setup[]. It also builds QEMU. If you ahve already done a xref:qemu-buildroot-setup[] previously, this will be very fast. If you modify the userland programs, rebuild simply with: @@ -3573,7 +3574,7 @@ If you modify the userland programs, rebuild simply with: ==== User mode GDB -It's nice when <> just works, right? +It's nice when xref:gdb[the obvious] just works, right? .... ./run \ @@ -3594,7 +3595,7 @@ and on another shell: ; .... -Or alternatively, if you are using <>, do everything in one go with: +Or alternatively, if you are using xref:tmux[], do everything in one go with: .... ./run \ @@ -3605,7 +3606,7 @@ Or alternatively, if you are using <>, do everything in one go with: ; .... -To stop at the very first instruction of a freestanding program, just use `--no-continue`. A good example of this is shown at: <>. +To stop at the very first instruction of a freestanding program, just use `--no-continue`. A good example of this is shown at: xref:freestanding-programs[]. === User mode tests @@ -3632,15 +3633,15 @@ This script skips a manually configured list of tests, notably: * tests that take perceptible ammounts of time * known bugs we didn't have time to fix ;-) -Tests under link:userland/libs/[] depend on certain libraries being available on the target, e.g. <> for link:userland/libs/openblas[]. They are not run by default, but can be enabled with `--package` and `--package-all`. +Tests under link:userland/libs/[] depend on certain libraries being available on the target, e.g. xref:blas[] for link:userland/libs/openblas[]. They are not run by default, but can be enabled with `--package` and `--package-all`. -The gem5 tests require building statically with build id `static`, see also: <>. TODO automate this better. +The gem5 tests require building statically with build id `static`, see also: xref:gem5-syscall-emulation-mode[]. TODO automate this better. -See: <> for more useful testing tips. +See: xref:test-this-repo[] for more useful testing tips. === User mode Buildroot executables -If you followed <>, you can now run the executables created by Buildroot directly as: +If you followed xref:qemu-buildroot-setup[], you can now run the executables created by Buildroot directly as: .... ./run \ @@ -3669,11 +3670,11 @@ or: ; .... -Here is an interesting examples of this: <> +Here is an interesting examples of this: xref:linux-test-project[] === User mode simulation with glibc -At 125d14805f769104f93c510bedaa685a52ec025d we <>, and caused some user mode pain, which we document here. +At 125d14805f769104f93c510bedaa685a52ec025d we xref:libc-choice[moved Buildroot from uClibc to glibc], and caused some user mode pain, which we document here. ==== FATAL: kernel too old @@ -3705,7 +3706,7 @@ The ID is just hardcoded on the source: For some reason QEMU / glibc x86_64 picks up the host libc, which breaks things. -Other archs work as they different host libc is skipped. <> also work. +Other archs work as they different host libc is skipped. xref:user-mode-static-executables[] also work. We have worked around this with with https://bugs.launchpad.net/qemu/+bug/1701798/comments/12 from the thread: https://bugs.launchpad.net/qemu/+bug/1701798 by creating the file: link:rootfs_overlay/etc/ld.so.cache[] which is a symlink to a file that cannot exist: `/dev/null/nonexistent`. @@ -3763,8 +3764,8 @@ We pass `-L` by default, so everything just works. However, in case something goes wrong, you can also try statically linked executables, since this mechanism tends to be a bit more stable, for example: -* gem5 user mode currently only supports static executables: <> -* QEMU x86_64 guest on x86_64 host was failing with <>, but we found a workaround +* gem5 user mode currently only supports static executables: xref:gem5-syscall-emulation-mode[] +* QEMU x86_64 guest on x86_64 host was failing with xref:stack-smashing-detected[], but we found a workaround ==== User mode static executables with dynamic libraries @@ -3772,7 +3773,7 @@ One limitation of static executables is that Buildroot mostly only builds dynami So programs that rely on those libraries might not compile as GCC can't find the `.a` version of the library. -For example, if we try to build <> statically: +For example, if we try to build xref:blas[] statically: .... ./build-userland --package openblas --static -- userland/libs/openblas/hello.c @@ -3820,7 +3821,7 @@ So let's just play with some static ones: TODO: how to escape spaces on the command line arguments? -<> also works normally on gem5: +xref:user-mode-gdb[GDB step debug] also works normally on gem5: .... ./run \ @@ -3858,7 +3859,7 @@ echo $? and the output is always `0`. -Instead, it just outputs a message to stdout just like for <>: +Instead, it just outputs a message to stdout just like for xref:m5-fail[]: .... Simulated exit code not 0! Exit code is 1 @@ -3891,7 +3892,7 @@ Source: link:userland/c/getchar.c[] Let's see if user mode runs considerably faster than full system or not. -First we build Dhrystone manually statically since dynamic linking is broken in gem5: <>. +First we build Dhrystone manually statically since dynamic linking is broken in gem5: xref:gem5-syscall-emulation-mode[]. gem5 user mode: @@ -3940,7 +3941,7 @@ time \ ; .... -Result on <> at bad30f513c46c1b0995d3a10c0d9bc2a33dc4fa0: +Result on xref:p51[] at bad30f513c46c1b0995d3a10c0d9bc2a33dc4fa0: * gem5 user: 33 seconds * gem5 full system: 51 seconds @@ -3973,7 +3974,7 @@ TODO: investigate further and then possibly post on QEMU mailing list. ===== QEMU user mode does not show errors -Similarly to <>, QEMU error messages do not show at all through pipes. +Similarly to xref:qemu-user-mode-does-not-show-stdout-immediately[], QEMU error messages do not show at all through pipes. In particular, it does not say anything if you pass it a non-existing executable: @@ -4049,15 +4050,15 @@ This is the default install path for `CONFIG_SOME_MOD=m` modules built with `mak Currently, there are only two kinds of kernel modules that you can try out with `modprobe`: -* modules built with Buildroot, see: <> -* modules built from the kernel tree itself, see: <> +* modules built with Buildroot, see: xref:kernel_modules-buildroot-package[] +* modules built from the kernel tree itself, see: xref:dummy-irq[] We are not installing out custom `./build-modules` modules there, because: * we don't know the right way. Why is there no `install` or `install_modules` target for kernel modules? + This can of course be solved by running Buildroot in verbose mode, and copying whatever it is doing, initial exploration at: https://stackoverflow.com/questions/22783793/how-to-install-kernel-modules-from-source-code-error-while-make-process/53169078#53169078 -* we would have to think how to not have to include the kernel modules twice in the root filesystem, but still have <<9p>> working for fast development as described at: <> +* we would have to think how to not have to include the kernel modules twice in the root filesystem, but still have xref:9p[] working for fast development as described at: xref:your-first-kernel-module-hack[] === kmod @@ -4087,7 +4088,7 @@ contains: ii kmod 22-1ubuntu5 amd64 tools for managing Linux kernel modules .... -BusyBox also implements its own version of those executables, see e.g. <>. Here we will only describe features that differ from kmod to the BusyBox implementation. +BusyBox also implements its own version of those executables, see e.g. xref:modprobe[]. Here we will only describe features that differ from kmod to the BusyBox implementation. ==== module-init-tools @@ -4117,14 +4118,14 @@ We are very interested in this filesystem because we are looking for a way to ma This would have several advantages: * makes it faster to test modified guest programs -** not rebooting is fundamental for <>, where the reboot is very costly. +** not rebooting is fundamental for xref:gem5[], where the reboot is very costly. ** no need to regenerate the root filesystem at all and reboot -** overcomes the `check_bin_arch` problem: <> +** overcomes the `check_bin_arch` problem: xref:rpath[] * we could keep the base root filesystem very small, which implies: ** less host disk usage, no need to copy the entire `./getvar out_rootfs_overlay_dir` to the image again -** no need to worry about <> +** no need to worry about xref:br2_target_rootfs_ext2_size[] -We can already make host files appear on the guest with <<9p>>, but they appear on a subdirectory instead of the root. +We can already make host files appear on the guest with xref:9p[], but they appear on a subdirectory instead of the root. If they would appear on the root instead, that would be even more awesome, because you would just use the exact same paths relative to the root transparently. @@ -4161,7 +4162,7 @@ Even more awesome than `chroot` would be to `pivot_root`, but I couldn't get tha === Secondary disk -A simpler and possibly less overhead alternative to <<9P>> would be to generate a secondary disk image with the benchmark you want to rebuild. +A simpler and possibly less overhead alternative to xref:9P[] would be to generate a secondary disk image with the benchmark you want to rebuild. Then you can `umount` and re-mount on guest without reboot. @@ -4179,13 +4180,13 @@ https://unix.stackexchange.com/questions/307390/what-is-the-difference-between-t Text mode is the default mode for QEMU. -The opposite of text mode is <> +The opposite of text mode is xref:qemu-graphic-mode[] In text mode, we just show the serial console directly on the current terminal, without opening a QEMU GUI window. You cannot see any graphics from text mode, but text operations in this mode, including: -* scrolling up: <> +* scrolling up: xref:scroll-up-in-graphic-mode[] * copy paste to and from the terminal making this a good default, unless you really need to use with graphics. @@ -4197,7 +4198,7 @@ This is different from a display screen, where each character is a bunch of pixe For more details, see: * https://unix.stackexchange.com/questions/307390/what-is-the-difference-between-ttys0-ttyusb0-and-ttyama0-in-linux -* <> +* xref:tty[] Note that you can still see an image even in text mode with the VNC: @@ -4211,7 +4212,7 @@ and on another terminal: ./vnc .... -but there is not terminal on the VNC window, just the <> penguin. +but there is not terminal on the VNC window, just the xref:config_logo[] penguin. ==== Quit QEMU from text mode @@ -4227,7 +4228,7 @@ Ctrl-A X Alternative methods include: -* `quit` command on the <> +* `quit` command on the xref:qemu-monitor[] * `pkill qemu` === QEMU graphic mode @@ -4238,9 +4239,9 @@ Enable graphic mode with: ./run --graphic .... -Outcome: you see a penguin due to <>. +Outcome: you see a penguin due to xref:config_logo[]. -For a more exciting GUI experience, see: <> +For a more exciting GUI experience, see: xref:x11[] Text mode is the default due to the following considerable advantages: @@ -4252,7 +4253,7 @@ Text mode is the default due to the following considerable advantages: Text mode has the following limitations over graphics mode: -* you can't see graphics such as those produced by <> +* you can't see graphics such as those produced by xref:x11[] * very early kernel messages such as `early console in extract_kernel` only show on the GUI, since at such early stages, not even the serial has been setup. `x86_64` has a VGA device enabled by default, as can be seen as: @@ -4271,7 +4272,7 @@ flooding the screen with colors. See also: https://superuser.com/questions/22309 ==== Scroll up in graphic mode -Scroll up in <>: +Scroll up in xref:qemu-graphic-mode[]: .... Shift-PgUp @@ -4282,7 +4283,7 @@ but I never managed to increase that buffer: * https://askubuntu.com/questions/709697/how-to-increase-scrollback-lines-in-ubuntu14-04-2-server-edition * https://unix.stackexchange.com/questions/346018/how-to-increase-the-scrollback-buffer-size-for-tty -The superior alternative is to use text mode and GNU screen or <>. +The superior alternative is to use text mode and GNU screen or xref:tmux[]. ==== QEMU Graphic mode arm @@ -4294,7 +4295,7 @@ TODO: on arm, we see the penguin and some boot messages, but don't get a shell a ./run --arch aarch64 --graphic .... -I think it does not work because the graphic window is <> only, i.e.: +I think it does not work because the graphic window is xref:drm[] only, i.e.: .... cat /dev/urandom > /dev/fb0 @@ -4306,9 +4307,9 @@ fails with: cat: write error: No space left on device .... -and has no effect, and the Linux kernel does not appear to have a built-in DRM console as it does for fbdev with <>. +and has no effect, and the Linux kernel does not appear to have a built-in DRM console as it does for fbdev with xref:fbcon[fbcon]. -There is however one out-of-tree implementation: <>. +There is however one out-of-tree implementation: xref:kmscon[]. ===== QEMU graphic mode arm terminal implementation @@ -4355,7 +4356,7 @@ TODO could not get it working on `x86_64`, only ARM. Overview: https://stackoverflow.com/questions/50364863/how-to-get-graphical-gui-output-and-user-touch-keyboard-mouse-input-in-a-ful/50364864#50364864 -More concretely, first build the kernel with the <>, and then run: +More concretely, first build the kernel with the xref:gem5-arm-linux-kernel-patches[], and then run: .... ./build-linux \ @@ -4372,7 +4373,7 @@ and then on another shell: vinagre localhost:5900 .... -The <> penguin only appears after several seconds, together with kernel messages of type: +The xref:config_logo[] penguin only appears after several seconds, together with kernel messages of type: .... [ 0.152755] [drm] found ARM HDLCD version r0p0 @@ -4407,7 +4408,7 @@ Alternatively, you can also dump each new frame to an image file with `--frame-c ; .... -This creates on compressed PNG whenever the screen image changes inside the <> with filename of type: +This creates on compressed PNG whenever the screen image changes inside the xref:m5out-directory[] with filename of type: .... frames_system.vncserver/fb...png.gz @@ -4417,7 +4418,7 @@ It is fun to see how we get one new frame whenever the white underscore cursor a The last frame is always available uncompressed at: `system.framebuffer.png`. -TODO <> failed on `aarch64` with: +TODO xref:kmscube[] failed on `aarch64` with: .... kmscube[706]: unhandled level 2 translation fault (11) at 0x00000000, esr 0x92000006, in libgbm.so.1.0.0[7fbf6a6000+e000] @@ -4452,7 +4453,7 @@ The DP650 is a newer display hardware than HDLCD. TODO is its interface publicly The key option to enable support in Linux is `DRM_MALI_DISPLAY=y` which we enable at link:linux_config/display[]. -Build the kernel exactly as for <> and then run with: +Build the kernel exactly as for xref:graphic-mode-gem5-aarch64[] and then run with: .... ./run --arch aarch64 --dp650 --emulator gem5 --linux-build-id gem5-v4.15 @@ -4460,11 +4461,11 @@ Build the kernel exactly as for <> and then run with: ==== Graphic mode gem5 internals -We cannot use mainline Linux because the <> are required at least to provide the `CONFIG_DRM_VIRT_ENCODER` option. +We cannot use mainline Linux because the xref:gem5-arm-linux-kernel-patches[] are required at least to provide the `CONFIG_DRM_VIRT_ENCODER` option. gem5 emulates the http://infocenter.arm.com/help/index.jsp?topic=/com.arm.doc.dui0541c/CHDBAIDI.html[HDLCD] ARM Holdings hardware for `arm` and `aarch64`. -The kernel uses HDLCD to implement the <> interface, the required kernel config options are present at: link:linux_config/display[]. +The kernel uses HDLCD to implement the xref:drm[] interface, the required kernel config options are present at: link:linux_config/display[]. TODO: minimize out the `--custom-config-file`. If we just remove it on `arm`: it does not work with a failing dmesg: @@ -4539,7 +4540,7 @@ xcalc xeyes .... -Outcome: <> +Outcome: xref:image-x11[] [[image-x11]] .X11 Buildroot graphical user interface screenshot @@ -4560,7 +4561,7 @@ To x11 packages have an `xserver` prefix as in: the easiest way to find them out is to just list `"$(./getvar buildroot_build_build_dir)/x*`. -TODO as of: c2696c978d6ca88e8b8599c92b1beeda80eb62b2 I noticed that `startx` leads to a <>: +TODO as of: c2696c978d6ca88e8b8599c92b1beeda80eb62b2 I noticed that `startx` leads to a xref:bug_on[]: .... [ 2.809104] WARNING: CPU: 0 PID: 51 at drivers/gpu/drm/ttm/ttm_bo_vm.c:304 ttm_bo_vm_open+0x37/0x40 @@ -4637,7 +4638,7 @@ A friend told me this but I haven't tried it yet: === Enable networking -We disable networking by default because it starts an userland process, and we want to keep the number of userland processes to a minimum to make the system more understandable: <> +We disable networking by default because it starts an userland process, and we want to keep the number of userland processes to a minimum to make the system more understandable: xref:resource-tradeoff-guidelines[] To enable networking on Buildroot, simply run: @@ -4660,7 +4661,7 @@ Disable networking with: ifdown -a .... -To enable networking by default after boot, use the methods documented at <>. +To enable networking by default after boot, use the methods documented at xref:init-busybox[]. === ping @@ -4682,7 +4683,7 @@ https://unix.stackexchange.com/questions/473448/how-to-ping-from-the-qemu-guest- In this section we discuss how to interact between the guest and the host through networking. -First ensure that you can access the external network since that is easier to get working: <>. +First ensure that you can access the external network since that is easier to get working: xref:networking[]. ==== Host to guest networking @@ -4711,7 +4712,7 @@ This uses: Only this specific port works by default since we have forwarded it on the QEMU command line. -We us this exact procedure to connect to <>. +We us this exact procedure to connect to xref:gdbserver[]. ===== ssh into guest @@ -4743,7 +4744,7 @@ Could not do port forwarding from host to guest, and therefore could not use `gd ==== Guest to host networking -First <>. +First xref:enable-networking[]. Then in the host, start a server: @@ -4776,7 +4777,7 @@ Bibliography: https://serverfault.com/questions/769874/how-to-forward-a-port-fro The https://en.wikipedia.org/wiki/9P_(protocol)[9p protocol] allows the guest to mount a host directory. -Both QEMU and <<9p-gem5>> support 9P. +Both QEMU and xref:9p-gem5[] support 9P. ==== 9P vs NFS @@ -4785,7 +4786,7 @@ All of 9P and NFS (and sshfs) allow sharing directories between guest and host. Advantages of 9P * requires `sudo` on the host to mount -* we could share a guest directory to the host, but this would require running a server on the guest, which adds <> +* we could share a guest directory to the host, but this would require running a server on the guest, which adds xref:resource-tradeoff-guidelines[simulation overhead] + Furthermore, this would be inconvenient, since what we usually want to do is to share host cross built files with the guest, and to do that we would have to copy the files over after the guest starts the server. * QEMU implements 9P natively, which makes it very stable and convenient, and must mean it is a simpler protocol than NFS as one would expect. @@ -4822,7 +4823,7 @@ cat guest The main ingredients for this are: -* `9P` settings in our <> +* `9P` settings in our xref:kernel-configs-about[kernel configs] * `9p` entry on our link:rootfs_overlay/etc/fstab[] + Alternatively, you could also mount your own with: @@ -4857,9 +4858,9 @@ TODO seems possible! Lets do it: TODO: get working. -<<9p>> is better with emulation, but let's just get this working for fun. +xref:9p[] is better with emulation, but let's just get this working for fun. -First make sure that this works: <>. +First make sure that this works: xref:guest-to-host-networking[]. Then, build the kernel with NFS support: @@ -4912,7 +4913,7 @@ systemctl disable nfs-kernel-server ==== Modify kernel config -To modify a single option on top of our <>, do: +To modify a single option on top of our xref:kernel-configs-about[default kernel configs], do: .... ./build-linux --config 'CONFIG_FORTIFY_SOURCE=y' @@ -4925,7 +4926,7 @@ Kernel modules depend on certain kernel configs, and therefore in general you mi ./build-modules .... -and then proceed as in <>. +and then proceed as in xref:your-first-kernel-module-hack[]. You might often get way without rebuilding the kernel modules however. @@ -4945,7 +4946,7 @@ To use just your own exact `.config` instead of our defaults ones, use: ./build-linux --custom-config-file data/myconfig .... -There is also a shortcut `--custom-config-file` to use the <>. +There is also a shortcut `--custom-config-file` to use the xref:gem5-arm-linux-kernel-patches[]. The following options can all be used together, sorted by decreasing config setting power precedence: @@ -4974,7 +4975,7 @@ cp "$(./getvar linux_build_dir)/defconfig" data/myconfig ./build-linux --custom-config-file data/myconfig .... -You can also use other config generating targets such as `defconfig` with the same method as shown at: <>. +You can also use other config generating targets such as `defconfig` with the same method as shown at: xref:linux-kernel-defconfig[]. ==== Find the kernel config @@ -5024,7 +5025,7 @@ although this can be useful when someone gives you a random image. By default, link:build-linux[] generates a `.config` that is a mixture of: -* a base config extracted from Buildroot's minimal per machine `.config`, which has the minimal options needed to boot: <>. +* a base config extracted from Buildroot's minimal per machine `.config`, which has the minimal options needed to boot: xref:buildroot-kernel-config[]. * small overlays put top of that To find out which kernel configs are being used exactly, simply run: @@ -5049,10 +5050,10 @@ Note that Buildroot can `sed` override some of the configurations, e.g. it force On top of those, we add the following by default: -* link:linux_config/min[]: see: <> +* link:linux_config/min[]: see: xref:linux-kernel-min-config[] * link:linux_config/default[]: other optional configs that we enable by default because they increase visibility, or expose some cool feature, and don't significantly increase build time nor add significant runtime overhead + -We have since observed that the kernel size itself is very bloated compared to `defconfig`: <>. +We have since observed that the kernel size itself is very bloated compared to `defconfig`: xref:linux-kernel-defconfig[]. [[buildroot-kernel-config]] ===== About Buildroot's kernel configs @@ -5095,7 +5096,7 @@ Output: Brutal. Where did we go wrong? -The extra virtio options are not needed if we use <>: +The extra virtio options are not needed if we use xref:initrd[]: .... ./build-linux \ @@ -5140,7 +5141,7 @@ Tested on 1e2b7f1e5e9e3073863dc17e25b2455c8ebdeadd + 1. link:linux_config/min[] contains minimal tweaks required to boot gem5 or for using our slightly different QEMU command line options than Buildroot on all archs. -It is one of the default config fragments we use, as explained at: <>>. +It is one of the default config fragments we use, as explained at: xref:kernel-configs-about[]>. Having the same config working for both QEMU and gem5 (oh, the hours of bisection) means that you can deal with functional matters in QEMU, which runs much faster, and switch to gem5 only for performance issues. @@ -5163,7 +5164,7 @@ Tested on 649d06d6758cefd080d04dc47fd6a5a26a620874 + 1. Other configs which we had previously tested at 4e0d9af81fcce2ce4e777cb82a1990d7c2ca7c1e are: -* `arm` and `aarch64` configs present in the official ARM gem5 Linux kernel fork: <>. Some of the configs present there are added by the patches. +* `arm` and `aarch64` configs present in the official ARM gem5 Linux kernel fork: xref:gem5-arm-linux-kernel-patches[]. Some of the configs present there are added by the patches. * Jason's magic `x86_64` config: http://web.archive.org/web/20171229121642/http://www.lowepower.com/jason/files/config which is referenced at: http://web.archive.org/web/20171229121525/http://www.lowepower.com/jason/setting-up-gem5-full-system.html[]. QEMU boots with that by removing `# CONFIG_VIRTIO_PCI is not set`. === Kernel version @@ -5197,7 +5198,7 @@ This backwards compatibility is just awesome, it makes getting and running the l This also makes this repo the perfect setup to develop the Linux kernel. -In case something breaks while updating the Linux kernel, you can try to bisect it to understand the root cause: <>. +In case something breaks while updating the Linux kernel, you can try to bisect it to understand the root cause: xref:bisection[]. ==== Downgrade the Linux kernel @@ -5333,7 +5334,7 @@ which uses default mount `rw` flags. We have however removed those setups init setups to keep things more minimal, and replaced them with the `rw` kernel boot parameter makes the root mounted as writable. -To observe the default readonly behaviour, hack the link:run[] script to remove <>, and then run on a raw shell: +To observe the default readonly behaviour, hack the link:run[] script to remove xref:replace-init[replace init], and then run on a raw shell: .... ./run --kernel-cli 'init=/bin/sh' @@ -5368,7 +5369,7 @@ and so it is Read Only as shown by `ro`. ==== norandmaps -Disable userland address space randomization. Test it out by running <> twice: +Disable userland address space randomization. Test it out by running xref:rand_check-out[] twice: .... ./run --eval-after './linux/rand_check.out;./linux/poweroff.out' @@ -5403,7 +5404,7 @@ echo 1 > /proc/sys/kernel/printk See also: https://superuser.com/questions/351387/how-to-stop-kernel-messages-from-flooding-my-console -Do it with a <> to affect the boot itself: +Do it with a xref:kernel-command-line-parameters[] to affect the boot itself: .... ./run --kernel-cli 'loglevel=5' @@ -5433,7 +5434,7 @@ This format is selected by the following boot options: * `console_msg_format=syslog`: add the `` part. Added in v4.16. * `printk.time=y`: add the `[TIMESTAMP]` part -The debug highest level is a bit more magic, see: <> for more info. +The debug highest level is a bit more magic, see: xref:pr_debug[] for more info. ==== ignore_loglevel @@ -5569,7 +5570,7 @@ This likely comes from the ifdef split at `init/main.c`: ==== Kernel module parameters -The Linux kernel allows passing module parameters at insertion time <>. +The Linux kernel allows passing module parameters at insertion time xref:myinsmod[through the `init_module` and `finit_module` system calls]. The `insmod` tool exposes that as: @@ -5603,7 +5604,7 @@ Sources: * link:kernel_modules/params.c[] * link:rootfs_overlay/lkmc/params.sh[] -As shown in the example, module parameters can also be read and modified at runtime from <>. +As shown in the example, module parameters can also be read and modified at runtime from xref:sysfs[]. We can obtain the help text of the parameters with: @@ -5620,7 +5621,7 @@ parm: i:my favorite int ===== modprobe.conf -<> insertion can also set default parameters via the link:rootfs_overlay/etc/modprobe.conf[`/etc/modprobe.conf`] file: +xref:modprobe[] insertion can also set default parameters via the link:rootfs_overlay/etc/modprobe.conf[`/etc/modprobe.conf`] file: .... modprobe params @@ -5633,7 +5634,7 @@ Output: 12 34 .... -This is specially important when loading modules with <> or else we would have no opportunity of passing those. +This is specially important when loading modules with xref:kernel-module-dependencies[] or else we would have no opportunity of passing those. `modprobe.conf` doesn't actually insmod anything for us: https://superuser.com/questions/397842/automatically-load-kernel-module-at-boot-angstrom/1267464#1267464 @@ -5677,7 +5678,7 @@ ffffffffc0002300 B lkmc_dep [dep] This requires `CONFIG_KALLSYMS_ALL=y`. -Dependency information is stored by the kernel module build system in the `.ko` files' <>, e.g.: +Dependency information is stored by the kernel module build system in the `.ko` files' xref:module_info[], e.g.: .... modinfo dep2.ko @@ -5719,9 +5720,9 @@ TODO: what for, and at which point point does Buildroot / BusyBox generate that ===== Kernel module dependencies with modprobe -Unlike `insmod`, <> deals with kernel module dependencies for us. +Unlike `insmod`, xref:modprobe[] deals with kernel module dependencies for us. -First get <> working. +First get xref:kernel_modules-buildroot-package[] working. Then, for example: @@ -5860,7 +5861,7 @@ Bibliography: ==== vermagic -Vermagic is a magic string present in the kernel and on <> of kernel modules. It is used to verify that the kernel module was compiled against a compatible kernel version and relevant configuration: +Vermagic is a magic string present in the kernel and on xref:module_info[] of kernel modules. It is used to verify that the kernel module was compiled against a compatible kernel version and relevant configuration: .... insmod vermagic.ko @@ -5910,7 +5911,7 @@ The `SMP` part of the string for example is defined on the same file based on th TODO how to get the vermagic from running kernel from userland? https://lists.kernelnewbies.org/pipermail/kernelnewbies/2012-October/006306.html -<> has a flag to skip the vermagic check: +xref:kmod-modprobe[] has a flag to skip the vermagic check: .... --force-modversion @@ -6011,7 +6012,7 @@ How to generate them: * https://unix.stackexchange.com/questions/66197/how-to-cause-kernel-panic-with-a-single-command * https://stackoverflow.com/questions/23484147/generate-kernel-oops-or-crash-in-the-code -When a panic happens, <> does not work as it normally does, and it is hard to get the logs if on are on <>: +When a panic happens, xref:linux-kernel-magic-keys[`Shift-PgUp`] does not work as it normally does, and it is hard to get the logs if on are on xref:qemu-graphic-mode[]: * https://superuser.com/questions/848412/scrolling-up-the-failed-screen-with-kernel-panic * https://superuser.com/questions/269228/write-qemu-booting-virtual-machine-output-to-a-file @@ -6119,14 +6120,14 @@ For testing purposes, it is very useful to quit the emulator automatically with Enabled by default with: -* `panic=-1` command line option which reboots the kernel immediately on panic, see: <> +* `panic=-1` command line option which reboots the kernel immediately on panic, see: xref:reboot-on-panic[] * QEMU `-no-reboot`, which makes QEMU exit when the guest tries to reboot Also asked at https://unix.stackexchange.com/questions/443017/can-i-make-qemu-exit-with-failure-on-kernel-panic which also mentions the x86_64 `-device pvpanic`, but I don't see much advantage to it. TODO neither method exits with exit status different from 0, so for now we are just grepping the logs for panic messages, which sucks. -One possibility that gets close would be to use <> to break at the `panic` function, and then send a <> `quit` command if that happens, but I don't see a way to exit with non-zero status to indicate error. +One possibility that gets close would be to use xref:gdb[] to break at the `panic` function, and then send a xref:qemu-monitor-from-gdb[] `quit` command if that happens, but I don't see a way to exit with non-zero status to indicate error. ====== Exit gem5 on panic @@ -6300,10 +6301,10 @@ which gives us the correct line: Line 7 of "/root/linux-kernel-module-cheat/out/kernel_modules/x86_64/kernel_modules/panic.c" starts at address 0xbf00001c and ends at 0xbf00002c . .... -This-did not work on `arm` due to <> so we need to either: +This-did not work on `arm` due to xref:gdb-step-debug-kernel-module-arm[] so we need to either: -* <> -* <> post-mortem method +* xref:gdb-module_init[] +* xref:kernel-module-stack-trace-to-source-line[] post-mortem method ==== dump_stack @@ -6317,7 +6318,7 @@ Source: link:kernel_modules/dump_stack.c[] ==== WARN_ON -The `WARN_ON` macro basically just calls <>. +The `WARN_ON` macro basically just calls xref:dump_stack[dump_stack]. One extra side effect is that we can make it also panic with: @@ -6334,7 +6335,7 @@ Can also be activated with the `panic_on_warn` boot parameter. Pseudo filesystems are filesystems that don't represent actual files in a hard disk, but rather allow us to do special operations on filesystem-related system calls. -What each pseudo-file does for each related system call does is defined by its <>. +What each pseudo-file does for each related system call does is defined by its xref:file-operations[]. Bibliography: @@ -6361,7 +6362,7 @@ Sources: * link:kernel_modules/debugfs.c[] * link:rootfs_overlay/lkmc/debugfs.sh[] -Debugfs is made specifically to help test kernel stuff. Just mount, set <>, and we are done. +Debugfs is made specifically to help test kernel stuff. Just mount, set xref:file-operations[], and we are done. For this reason, it is the filesystem that we use whenever possible in our tests. @@ -6393,9 +6394,9 @@ Outcome: the test passes: 0 .... -Procfs is a little less convenient than <>, but is more used in serious applications. +Procfs is a little less convenient than xref:debugfs[], but is more used in serious applications. -Procfs can run all system calls, including ones that debugfs can't, e.g. <>. +Procfs can run all system calls, including ones that debugfs can't, e.g. xref:mmap[]. Sources: @@ -6410,7 +6411,7 @@ Bibliography: [[proc-version]] ===== /proc/version -Its data is shared with `uname()`, which is a <> function and has a Linux syscall to back it up. +Its data is shared with `uname()`, which is a xref:posix[POSIX C] function and has a Linux syscall to back it up. Where the data comes from and how to modify it: @@ -6430,7 +6431,7 @@ Linux version 4.19.0-dirty (lkmc@84df9525b0c27f3ebc2ebb1864fa62a97fdedb7d) (gcc ==== sysfs -Sysfs is more restricted than <>, as it does not take an arbitrary `file_operations`: +Sysfs is more restricted than xref:procfs[], as it does not take an arbitrary `file_operations`: .... ./sysfs.sh @@ -6456,7 +6457,7 @@ Vs procfs: You basically can only do `open`, `close`, `read`, `write`, and `lseek` on sysfs files. -It is similar to a <> file operation, except that write is also implemented. +It is similar to a xref:seq_file[] file operation, except that write is also implemented. TODO: what are those `kobject` structs? Make a more complex example that shows what they can do. @@ -6470,7 +6471,7 @@ Bibliography: ==== Character devices -Character devices can have arbitrary <> associated to them: +Character devices can have arbitrary xref:file-operations[] associated to them: .... ./character_device.sh @@ -6489,7 +6490,7 @@ Sources: * link:rootfs_overlay/lkmc/mknoddev.sh[] * link:kernel_modules/character_device.c[] -Unlike <> entires, character device files are created with userland `mknod` or `mknodat` syscalls: +Unlike xref:procfs[] entires, character device files are created with userland `mknod` or `mknodat` syscalls: .... mknod c @@ -6549,7 +6550,7 @@ Bibliography: https://stackoverflow.com/questions/5970595/how-to-create-a-device ==== File operations -File operations are the main method of userland driver communication. `struct file_operations` determines what the kernel will do on filesystem system calls of <>. +File operations are the main method of userland driver communication. `struct file_operations` determines what the kernel will do on filesystem system calls of xref:pseudo-filesystems[]. This example illustrates the most basic system calls: `open`, `read`, `write`, `close` and `lseek`: @@ -6581,7 +6582,7 @@ No, there no official documentation: https://stackoverflow.com/questions/1521393 ==== seq_file -Writing trivial read <> is repetitive and error prone. The `seq_file` API makes the process much easier for those trivial cases: +Writing trivial read xref:file-operations[] is repetitive and error prone. The `seq_file` API makes the process much easier for those trivial cases: .... ./seq_file.sh @@ -6618,7 +6619,7 @@ Bibliography: ===== seq_file single_open -If you have the entire read output upfront, `single_open` is an even more convenient version of <>: +If you have the entire read output upfront, `single_open` is an even more convenient version of xref:seq_file[]: .... ./seq_file.sh @@ -6662,7 +6663,7 @@ Typically, we are waiting for some hardware to make some piece of data available The hardware notifies the kernel that the data is ready with an interrupt. -To simplify this example, we just fake the hardware interrupts with a <> that sleeps for a second in an infinite loop. +To simplify this example, we just fake the hardware interrupts with a xref:kthread[] that sleeps for a second in an infinite loop. Bibliography: https://stackoverflow.com/questions/30035776/how-to-add-poll-function-to-the-kernel-module-code/44645336#44645336 @@ -6738,7 +6739,7 @@ Sources: In this example, we make a tiny 4 byte kernel buffer available to user-space, and we then modify it on userspace, and check that the kernel can see the modification. -`mmap`, like most more complex <>, does not work with <> as of 4.9, so we use a <> file for it. +`mmap`, like most more complex xref:file-operations[], does not work with xref:debugfs[] as of 4.9, so we use a xref:procfs[] file for it. Example adapted from: https://coherentmusings.wordpress.com/2014/06/10/implementing-mmap-for-transferring-data-from-user-space-to-kernel-space/ @@ -6773,7 +6774,7 @@ Sources: * link:userland/kernel_modules/anonymous_inode.c[] * link:rootfs_overlay/lkmc/anonymous_inode.sh[] -This example gets an anonymous inode via <> from a debugfs entry by using `anon_inode_getfd`. +This example gets an anonymous inode via xref:ioctl[] from a debugfs entry by using `anon_inode_getfd`. Reads to that inode return the sequence: `1`, `10`, `100`, ... `10000000`, `1`, `100`, ... @@ -6846,7 +6847,7 @@ The count stops when we `rmmod`: rmmod kthread .... -The sleep is done with `usleep_range`, see: <>. +The sleep is done with `usleep_range`, see: xref:sleep[]. Bibliography: @@ -6906,7 +6907,7 @@ Bibliography: ==== Workqueues -A more convenient front-end for <>: +A more convenient front-end for xref:kthread[]: .... insmod workqueue_cheat.ko @@ -6958,7 +6959,7 @@ Outcome: the system hangs, the only way out is to kill the VM. Source: link:kernel_modules/schedule.c[] -kthreads only allow interrupting if you call `schedule()`, and the `schedule=0` <> turns it off. +kthreads only allow interrupting if you call `schedule()`, and the `schedule=0` xref:kernel-module-parameters[kernel module parameter] turns it off. Sleep functions like `usleep_range` also end up calling schedule. @@ -6975,7 +6976,7 @@ and we can observe the counting with: dmesg -w .... -The system also responds if we <>: +The system also responds if we xref:number-of-cores[add another core]: .... ./run --cpus 2 --eval-after 'dmesg -n 1;insmod schedule.ko schedule=0' @@ -7113,7 +7114,7 @@ which shows: so only `1` has `myirqhandler0` attached but not `0`. -The <> also has some interrupt statistics for x86_64: +The xref:qemu-monitor[] also has some interrupt statistics for x86_64: .... ./qemu-monitor info irq @@ -7163,11 +7164,11 @@ However, this module is intended to fire only once as can be seen from its sourc and furthermore interrupt `1` and `12` happen immediately TODO why, were they somehow pending? -So so see something interesting, you need to monitor an interrupt that is more rare than the keyboard, e.g. <>. +So so see something interesting, you need to monitor an interrupt that is more rare than the keyboard, e.g. xref:platform_device[]. ==== /proc/interrupts -In the guest with <>: +In the guest with xref:qemu-graphic-mode[]: .... watch -n 1 cat /proc/interrupts @@ -7180,7 +7181,7 @@ This confirms that: * 1: keyboard * 12: mouse click and drags -The module also shows which handlers are registered for each IRQ, as we have observed at <> +The module also shows which handlers are registered for each IRQ, as we have observed at xref:irq-ko[] When in text mode, we can also observe interrupt line 4 with handler `ttyS0` increase continuously as IO goes through the UART. @@ -7307,8 +7308,8 @@ Source: link:userland/linux/virt_to_phys_user.c[] Now we can verify that `linux/virt_to_phys_user.out` gave the correct physical address in the following ways: -* <> -* <> +* xref:qemu-xp[] +* xref:dev-mem[] Bibliography: @@ -7317,9 +7318,9 @@ Bibliography: ====== QEMU xp -The `xp` <> command reads memory at a given physical address. +The `xp` xref:qemu-monitor[] command reads memory at a given physical address. -First launch `linux/virt_to_phys_user.out` as described at <>. +First launch `linux/virt_to_phys_user.out` as described at xref:userland-physical-address-experiments[]. On a second terminal, use QEMU to read the physical address: @@ -7342,7 +7343,7 @@ We could not find however to write to memory from the QEMU monitor, boring. `/dev/mem` exposes access to physical addresses, and we use it through the convenient `devmem` BusyBox utility. -First launch `linux/virt_to_phys_user.out` as described at <>. +First launch `linux/virt_to_phys_user.out` as described at xref:userland-physical-address-experiments[]. Next, read from the physical address: @@ -7393,7 +7394,7 @@ Bibliography: https://stackoverflow.com/questions/11891979/how-to-access-mmaped- Dump the physical address of all pages mapped to a given process using `/proc//maps` and `/proc//pagemap`. -First launch `linux/virt_to_phys_user.out` as described at <>. Suppose that the output was: +First launch `linux/virt_to_phys_user.out` as described at xref:userland-physical-address-experiments[]. Suppose that the output was: .... # ./posix/virt_to_phys_test.out & @@ -7754,7 +7755,7 @@ TODO: didn't port during refactor after 3b0a343647bed577586989fb702b760bd280844a * `qemu/docs/tracing.txt` and `qemu/docs/replay.txt` * https://stackoverflow.com/questions/39149446/how-to-use-qemus-simple-trace-backend/46497873#46497873 -Results (boot not excluded): <> +Results (boot not excluded): xref:table-boot-instruction-counts[] [[table-boot-instruction-counts]] .Boot instruction counts for various setups @@ -7967,7 +7968,7 @@ it does not work and outputs: setenforce: SELinux is disabled .... -SELinux requires glibc: <>. +SELinux requires glibc: xref:libc-choice[]. === User mode Linux @@ -8030,7 +8031,7 @@ Bibliography: [[fbcon]] ==== Linux kernel console fun -Requires <>. +Requires xref:graphics[]. You can also try those on the `Ctrl-Alt-F3` of your Ubuntu host, but it is much more fun inside a VM! @@ -8057,7 +8058,7 @@ TODO: font and keymap. Mentioned at: https://cmcenroe.me/2017/05/05/linux-consol ==== Linux kernel magic keys -Requires <>. +Requires xref:graphics[]. Let's have some fun. @@ -8097,7 +8098,7 @@ Enabled from our link:rootfs_overlay/etc/inittab[]: ::ctrlaltdel:/sbin/reboot .... -Linux tries to reboot, and QEMU shutdowns due to the `-no-reboot` option which we set by default for: <>. +Linux tries to reboot, and QEMU shutdowns due to the `-no-reboot` option which we set by default for: xref:exit-emulator-on-panic[]. Under the hood, behaviour is controlled by the `reboot` syscall: @@ -8411,7 +8412,7 @@ We can get some visibility into it to try and solve the problem with: ===== console kernel boot parameter -Take the command described at <> and try adding the following: +Take the command described at xref:tty[] and try adding the following: * `-e 'console=tty7'`: boot messages still show on `/dev/tty1` (TODO how to change that?), but we don't get a shell at the end of boot there. + @@ -8424,7 +8425,7 @@ Instead, the shell appears on `/dev/tty7`. ==== CONFIG_LOGO -If you run in <>, then you get a Penguin image for <> above the console! https://askubuntu.com/questions/80938/is-it-possible-to-get-the-tux-logo-on-the-text-based-boot +If you run in xref:graphics[], then you get a Penguin image for xref:number-of-cores[every core] above the console! https://askubuntu.com/questions/80938/is-it-possible-to-get-the-tux-logo-on-the-text-based-boot This is due to the https://github.com/torvalds/linux/blob/v4.17/drivers/video/logo/Kconfig#L5[`CONFIG_LOGO=y`] option which we enable by default. @@ -8467,7 +8468,7 @@ TODO not working for `aarch64`, it takes over the screen for a few seconds and t ./run --eval-after './libs/libdrm/modeset.out' --graphic .... -<> however worked, which means that it must be a bug with this demo? +xref:kmscube[] however worked, which means that it must be a bug with this demo? We set `CONFIG_DRM=y` on our default kernel configuration, and it creates one device file for each display: @@ -8554,9 +8555,9 @@ Tested on: https://github.com/cirosantilli/linux-kernel-module-cheat/commit/2903 TODO get working. -Implements a console for <>. +Implements a console for xref:drm[]. -The Linux kernel has a built-in fbdev console: <> but not for <> it seems. +The Linux kernel has a built-in fbdev console: xref:fbcon[fbcon] but not for xref:drm[] it seems. The upstream project seems dead with last commit in 2014: https://www.freedesktop.org/wiki/Software/kmscon/ @@ -8613,7 +8614,7 @@ exit01 1 TPASS : exit() test PASSED and has source code at: https://github.com/linux-test-project/ltp/blob/20190115/testcases/kernel/syscalls/exit/exit01.c -Besides testing any kernel modifications you make, LTP can also be used to the system call implementation of <> as shown at <>: +Besides testing any kernel modifications you make, LTP can also be used to the system call implementation of xref:user-mode-simulation[] as shown at xref:user-mode-buildroot-executables[]: .... ./run --userland "$(./getvar buildroot_target_dir)/usr/lib/ltp-testsuite/testcases/bin/exit01" @@ -8623,7 +8624,7 @@ Tested at: 287c83f3f99db8c1ff9bbc85a79576da6a78e986 + 1. ==== stress -<> userland stress. Two versions: +xref:posix[] userland stress. Two versions: .... ./build-buildroot \ @@ -8632,7 +8633,7 @@ Tested at: 287c83f3f99db8c1ff9bbc85a79576da6a78e986 + 1. ; .... -`STRESS_NG` is likely the best, but it requires glibc: <>. +`STRESS_NG` is likely the best, but it requires glibc: xref:libc-choice[]. Websites: @@ -8707,7 +8708,7 @@ Maybe it is also possible to run Xen directly like this: QEMU can already load m -kernel file1.elf -device loader,file=file2.elf .... -so as long as we craft the correct DTB and feed it into Xen so that it can see the kernel, it should work. TODO does QEMU support patching the auto-generated DTB with pre-generated options? In the worst case we can just dump it hand hack it up though with `-machine dumpdtb`: <>. +so as long as we craft the correct DTB and feed it into Xen so that it can see the kernel, it should work. TODO does QEMU support patching the auto-generated DTB with pre-generated options? In the worst case we can just dump it hand hack it up though with `-machine dumpdtb`: xref:device-tree-emulator-generation[]. Bibliography: @@ -8726,11 +8727,11 @@ https://en.wikipedia.org/wiki/QEMU[QEMU] is a system simulator: it simulates a C If you are familiar with https://en.wikipedia.org/wiki/VirtualBox[VirtualBox], then QEMU then basically does the same thing: it opens a "window" inside your desktop that can run an operating system inside your operating system. -Also both can use very similar techniques: either https://en.wikipedia.org/wiki/Binary_translation[binary translation] or <>. VirtualBox' binary translator is / was based on QEMU's it seems: https://en.wikipedia.org/wiki/VirtualBox#Software-based_virtualization +Also both can use very similar techniques: either https://en.wikipedia.org/wiki/Binary_translation[binary translation] or xref:KVM[]. VirtualBox' binary translator is / was based on QEMU's it seems: https://en.wikipedia.org/wiki/VirtualBox#Software-based_virtualization The huge advantage of QEMU over VirtualBox is that is supports cross arch simulation, e.g. simulate an ARM guest on an x86 host. -QEMU is likely the leading cross arch system simulator as of 2018. It is even the default <> simulator that developers get with Android Studio 3 to develop apps without real hardware. +QEMU is likely the leading cross arch system simulator as of 2018. It is even the default xref:android[] simulator that developers get with Android Studio 3 to develop apps without real hardware. Another advantage of QEMU over virtual box is that it doesn't have Oracle' hands all all over it, more like RedHat + ARM. @@ -8777,7 +8778,7 @@ the disk image gets overwritten by a fresh filesystem and you lose all changes. Remember that if you forcibly turn QEMU off without `sync` or `poweroff` from inside the VM, e.g. by closing the QEMU window, disk changes may not be saved. -Persistency is also turned off when booting from <> with a CPIO instead of with a disk. +Persistency is also turned off when booting from xref:initrd[] with a CPIO instead of with a disk. Disk persistency is useful to re-run shell commands from the history of a previous session with `Ctrl-R`, but we felt that the loss of determinism was not worth it. @@ -8785,7 +8786,7 @@ Disk persistency is useful to re-run shell commands from the history of a previo TODO how to make gem5 disk writes persistent? -As of cadb92f2df916dbb47f428fd1ec4932a2e1f0f48 there are some `read_only` entries in the <> under cow sections, but hacking them to true did not work: +As of cadb92f2df916dbb47f428fd1ec4932a2e1f0f48 there are some `read_only` entries in the xref:config-ini[] under cow sections, but hacking them to true did not work: .... diff --git a/configs/common/FSConfig.py b/configs/common/FSConfig.py @@ -8841,7 +8842,7 @@ Restore the snapshot: and the counting goes back to where we saved. This shows that CPU and memory states were reverted. -The `umount` is needed because snapshotting conflicts with <<9p>>, which we felt is a more valuable default. If you forget to unmount, the following error appears on the QEMU monitor: +The `umount` is needed because snapshotting conflicts with xref:9p[], which we felt is a more valuable default. If you forget to unmount, the following error appears on the QEMU monitor: ..... Migration is disabled when VirtFS export path '/linux-kernel-module-cheat/out/x86_64/buildroot/build' is mounted in the guest using mount_tag 'host_out' @@ -8879,7 +8880,7 @@ cat f And the output is `0`. -Our setup does not allow for snapshotting while using <>. +Our setup does not allow for snapshotting while using xref:initrd[]. Bibliography: https://stackoverflow.com/questions/40227651/does-qemu-emulator-have-checkpoint-function/48724371#48724371 @@ -8915,7 +8916,7 @@ Format specific information: As a consequence: * it is possible to restore snapshots across boots, since they stay on the same image the entire time -* it is not possible to use snapshots with <> in our setup, since we don't pass `-drive` at all when initrd is enabled +* it is not possible to use snapshots with xref:initrd[] in our setup, since we don't pass `-drive` at all when initrd is enabled === Device models @@ -8927,10 +8928,10 @@ This section documents: For the more complex interfaces, we focus on simplified educational devices, either: * present in the QEMU upstream: -** <> +** xref:qemu-edu[] * added in https://github.com/cirosantilli/qemu[our fork of QEMU]: -** <> -** <> +** xref:pci_min[] +** xref:platform_device[] ==== PCI @@ -8972,7 +8973,7 @@ What happened: * our hardware model is coded such that it generates an interrupt when written to * the Linux kernel interrupt handler write to another register, which tells the hardware to stop sending interrupts -Kernel messages and printks from inside QEMU are shown all together, to see that more clearly, run in <> instead. +Kernel messages and printks from inside QEMU are shown all together, to see that more clearly, run in xref:qemu-graphic-mode[] instead. We don't enable the device by default because it does not work for vanilla QEMU, which we often want to test with this repository. @@ -9096,13 +9097,13 @@ as: Memory at feb54000 .... -Then you can try messing with that address with <>: +Then you can try messing with that address with xref:dev-mem[]: .... devmem 0xfeb54000 w 0x12345678 .... -which writes to the first register of our <> device. +which writes to the first register of our xref:pci_min[] device. The device then fires an interrupt at irq 11, which is unhandled, which leads the kernel to say you are a bad boy: @@ -9114,14 +9115,14 @@ lkmc_pci_min mmio_write addr = 0 val = 12345678 size = 4 followed by a trace. -Next, also try using our <> IRQ monitoring module before triggering the interrupt: +Next, also try using our xref:irq-ko[] IRQ monitoring module before triggering the interrupt: .... insmod irq.ko devmem 0xfeb54000 w 0x12345678 .... -Our kernel module handles the interrupt, but does not acknowledge it like our proper <> kernel module, and so it keeps firing, which leads to infinitely many messages being printed: +Our kernel module handles the interrupt, but does not acknowledge it like our proper xref:pci_min[] kernel module, and so it keeps firing, which leads to infinitely many messages being printed: .... handler irq = 11 dev = 251 @@ -9249,7 +9250,7 @@ Buildroot's Linux tools package provides some GPIO CLI tools: `lsgpio`, `gpio-ev ==== LEDs -TODO: broken when `arm` moved to `-M virt`, same as <>. +TODO: broken when `arm` moved to `-M virt`, same as xref:gpio[]. Hack QEMU's `hw/misc/arm_sysctl.c` with a printf: @@ -9319,13 +9320,13 @@ Expected outcome after insmod: * QEMU reports MMIO with printfs * IRQs are generated and handled by this module, which logs to dmesg -Without insmoding this module, try writing to the register with <>: +Without insmoding this module, try writing to the register with xref:dev-mem[]: .... devmem 0x101e9000 w 0x12345678 .... -We can also observe the interrupt with <>: +We can also observe the interrupt with xref:dummy-irq[]: .... modprobe dummy-irq irq=34 @@ -9372,7 +9373,7 @@ Source: link:qemu-monitor[] `qemu-monitor` uses the `-monitor` QEMU command line option, which makes the monitor listen from a socket. -Alternatively, we can also enter the QEMU monitor from inside `-nographics` <> with: +Alternatively, we can also enter the QEMU monitor from inside `-nographics` xref:qemu-text-mode[] with: .... Ctrl-A C @@ -9395,7 +9396,7 @@ Ctrl-Alt ? where `?` is a digit `1`, or `2`, or, `3`, etc. depending on what else is available on the GUI: serial, parallel and frame buffer. -Finally, we can also access QEMU monitor commands directly from <> with the `monitor` command: +Finally, we can also access QEMU monitor commands directly from xref:gdb[] with the `monitor` command: .... ./run-gdb @@ -9434,7 +9435,7 @@ See also: https://superuser.com/questions/930588/how-to-pass-commands-noninterac ==== QEMU monitor from GDB -When doing <> it is possible to send QEMU monitor commands through the GDB `monitor` command, which saves you the trouble of opening yet another shell. +When doing xref:gdb[] it is possible to send QEMU monitor commands through the GDB `monitor` command, which saves you the trouble of opening yet another shell. Try for example: @@ -9472,7 +9473,7 @@ Or for a faster development loop: ./run --debug-vm-args '-ex "break edu_mmio_read" -ex "run"' .... -When in <>, using `--debug-vm` makes Ctrl-C not get passed to the QEMU guest anymore: it is instead captured by GDB itself, so allow breaking. So e.g. you won't be able to easily quit from a guest program like: +When in xref:qemu-text-mode[], using `--debug-vm` makes Ctrl-C not get passed to the QEMU guest anymore: it is instead captured by GDB itself, so allow breaking. So e.g. you won't be able to easily quit from a guest program like: .... sleep 10 @@ -9539,7 +9540,7 @@ Get the list of available trace events: ./run --trace help .... -TODO: any way to show the actualy disassembled instruction executed directly from there? Possible with <>. +TODO: any way to show the actualy disassembled instruction executed directly from there? Possible with xref:qemu-d-tracing[]. Enable other specific trace events: @@ -9620,7 +9621,7 @@ Seems impossible due to optimizations that QEMU does: PANDA can list memory addresses, so I bet it can also decode the instructions: https://github.com/panda-re/panda/blob/883c85fa35f35e84a323ed3d464ff40030f06bd6/panda/docs/LINE_Censorship.md I wonder why they don't just upstream those things to QEMU's tracing: https://github.com/panda-re/panda/issues/290 -gem5 can do it: <>. +gem5 can do it: xref:gem5-tracing[]. ==== Trace source lines @@ -9675,7 +9676,7 @@ A convenient shortcut to do both at once to test the feature is: By comparing the terminal output of both runs, we can see that they are the exact same, including things which normally differ across runs: * timestamps of dmesg output -* <> output +* xref:rand_check-out[] output The record and replay feature was revived around QEMU v3.0.0. It existed earlier but it rot completely. As of v3.0.0 it is still flaky: sometimes we get deadlocks, and only a limited number of command line arguments are supported. @@ -9711,7 +9712,7 @@ Solved on unmerged c42634d8e3428cfa60672c3ba89cabefc720cde9 from https://github. TODO `arm` and `aarch64` only seem to work with initrd since I cannot plug a working IDE disk device? See also: https://lists.gnu.org/archive/html/qemu-devel/2018-02/msg05245.html -Then, when I tried with <> and no disk: +Then, when I tried with xref:initrd[] and no disk: .... ./build-buildroot --arch aarch64 --initrd @@ -9800,7 +9801,7 @@ We also have a shortcut for `--trace ExecAll -trace-stdout` with `--trace-insts- ; .... -This would produce a lot of output however, so you will likely not want that when tracing a Linux kernel boot instructions. But it can be very convenient for smaller traces such as <>. +This would produce a lot of output however, so you will likely not want that when tracing a Linux kernel boot instructions. But it can be very convenient for smaller traces such as xref:baremetal[]. List all available debug flags: @@ -9823,7 +9824,7 @@ Be warned, the trace is humongous, at 16Gb. We can make the trace smaller by naming the trace file as `trace.txt.gz`, which enables GZIP compression, but that is not currently exposed on our scripts, since you usually just need something human readable to work on. -Enabling tracing made the runtime about 4x slower on the <>, with or without `.gz` compression. +Enabling tracing made the runtime about 4x slower on the xref:p51[], with or without `.gz` compression. The output format is of type: @@ -9853,9 +9854,9 @@ Breakdown: ** `D` stands for data, and represents the value that was written to memory or to a register ** `A` stands for address, and represents the address to which the value was written. It only shows when data is being written to memory, but not to registers. -The best way to verify all of this is to write some <> +The best way to verify all of this is to write some xref:baremetal[baremetal code] -Trace the source lines just like <> with: +Trace the source lines just like xref:trace-source-lines[for QEMU] with: .... ./trace-boot --arch aarch64 --emulator gem5 @@ -9883,7 +9884,7 @@ Using text mode is another workaround if you don't need GUI features. == gem5 -Getting started at: <>. +Getting started at: xref:gem5-buildroot-setup[]. === gem5 vs QEMU @@ -9909,9 +9910,9 @@ It is not of course truly cycle accurate, as that: but the approximation is reasonable. + It is used mostly for microarchitecture research purposes: when you are making a new chip technology, you don't really need to specialize enormously to an existing microarchitecture, but rather develop something that will work with a wide range of future architectures. -** runs are deterministic by default, unlike QEMU which has a special <> mode, that requires first playing the content once and then replaying -** gem5 ARM at least appears to implement more low level CPU functionality than QEMU, e.g. QEMU only added EL2 in 2018: https://stackoverflow.com/questions/42824706/qemu-system-aarch64-entering-el1-when-emulating-a53-power-up See also: <> -* disadvantage of gem5: slower than QEMU, see: <> +** runs are deterministic by default, unlike QEMU which has a special xref:qemu-record-and-replay[] mode, that requires first playing the content once and then replaying +** gem5 ARM at least appears to implement more low level CPU functionality than QEMU, e.g. QEMU only added EL2 in 2018: https://stackoverflow.com/questions/42824706/qemu-system-aarch64-entering-el1-when-emulating-a53-power-up See also: xref:arm-exception-levels[] +* disadvantage of gem5: slower than QEMU, see: xref:benchmark-linux-kernel-boot[] + This implies that the user base is much smaller, since no Android devs. + @@ -10021,7 +10022,7 @@ A more naive and simpler to understand approach would be a direct: ./run --arch aarch64 --emulator gem5 --eval 'm5 checkpoint;m5 resetstats;dhrystone 10000;m5 exit' .... -but the problem is that this method does not allow to easily run a different script without running the boot again. The `./gem5.sh` script works around that by using <> as explained further at: <>. +but the problem is that this method does not allow to easily run a different script without running the boot again. The `./gem5.sh` script works around that by using xref:m5-readfile[] as explained further at: xref:gem5-restore-new-script[]. Now you can play a fun little game with your friends: @@ -10029,18 +10030,18 @@ Now you can play a fun little game with your friends: * make a program that solves the computation problem, and outputs output to stdout * write the code that runs the correct computation in the smallest number of cycles possible -To find out why your program is slow, a good first step is to have a look at <> file. +To find out why your program is slow, a good first step is to have a look at xref:gem5-stats-txt[] file. ==== Skip extra benchmark instructions -A few imperfections of our <> are: +A few imperfections of our xref:gem5-run-benchmark[benchmarking method] are: * when we do `m5 resetstats` and `m5 exit`, there is some time passed before the `exec` system call returns and the actual benchmark starts and ends * the benchmark outputs to stdout, which means so extra cycles in addition to the actual computation. But TODO: how to get the output to check that it is correct without such IO cycles? Solutions to these problems include: -* modify benchmark code with instrumentation directly, see <> for an example. +* modify benchmark code with instrumentation directly, see xref:m5ops-instructions[] for an example. * monitor known addresses TODO possible? Create an example. Discussion at: https://stackoverflow.com/questions/48944587/how-to-count-the-number-of-cpu-clock-cycles-between-the-start-and-end-of-a-bench/48944588#48944588 @@ -10070,7 +10071,7 @@ getconf _NPROCESSORS_CONF https://stackoverflow.com/questions/50248067/how-to-run-a-gem5-arm-aarch64-full-system-simulation-with-fs-py-with-more-than-8 -Build the kernel with the <>, and then run: +Build the kernel with the xref:gem5-arm-linux-kernel-patches[], and then run: .... ./run \ @@ -10098,7 +10099,7 @@ A quick `+./run --emulator gem5 -- -h+` leads us to the options: --l3_size=1024 .... -But keep in mind that it only affects benchmark performance of the most detailed CPU types: <> +But keep in mind that it only affects benchmark performance of the most detailed CPU types: xref:table-gem5-cache-cpu-type[] [[table-gem5-cache-cpu-type]] .gem5 cache support in function of CPU type @@ -10276,14 +10277,14 @@ Buildroot built-in libraries, mostly under Libraries > Other: * libtommath * qhull -There are not yet enabled, but it should be easy to so, see: <> +There are not yet enabled, but it should be easy to so, see: xref:add-new-buildroot-packages[] ===== BST vs heap vs hashmap The following benchmark setup works both: * on host through timers + https://stackoverflow.com/questions/51952471/why-do-i-get-a-constant-instead-of-logarithmic-curve-for-an-insert-time-benchmar/51953081#51953081[granule] -* gem5 with <>, which can get more precise results with `granule == 1` +* gem5 with xref:m5ops-instructions[dumpstats], which can get more precise results with `granule == 1` It has been used to answer: @@ -10307,7 +10308,7 @@ xdg-open bst_vs_heap_vs_hashmap.tmp.png The parameters `heap_zoom_max` and `hashmap_zoom_max` are chosen manually interactively to best showcase the regions of interest in those plots. -First we build the benchmark with <> enabled, and then we run it and extract the stats: +First we build the benchmark with xref:m5ops-instructions[] enabled, and then we run it and extract the stats: .... ./build-userland \ @@ -10344,9 +10345,9 @@ xdg-open bst_vs_heap_vs_hashmap_gem5.tmp.png TODO: the gem5 simulation blows up on a tcmalloc allocation somewhere near 25k elements as of 3fdd83c2c58327d9714fa2347c724b78d7c05e2b + 1, likely linked to the extreme inefficiency of the stats collection? -The cache sizes were chosen to match the host <> to improve the comparison. Ideally we sould also use the same standard library. +The cache sizes were chosen to match the host xref:p51[] to improve the comparison. Ideally we sould also use the same standard library. -Note that this will take a long time, and will produce a humongous ~40Gb stats file due to: <> +Note that this will take a long time, and will produce a humongous ~40Gb stats file due to: xref:gem5-only-dump-selected-stats[] Sources: @@ -10428,8 +10429,8 @@ We have ported parts of the http://parsec.cs.princeton.edu[PARSEC benchmark] for There are two ways to run PARSEC with this repo: -* <>, most likely what you want -* <> +* xref:parsec-benchmark-without-parsecmgmt[without `pasecmgmt`], most likely what you want +* xref:parsec-benchmark-with-parsecmgmt[with `pasecmgmt`] ====== PARSEC benchmark without parsecmgmt @@ -10472,8 +10473,8 @@ Running a benchmark of a size different than `test`, e.g. `simsmall`, requires a Large input may also require tweaking: -* <> if the unpacked inputs are large -* <>, unless you want to meet the OOM killer, which is admittedly kind of fun +* xref:br2_target_rootfs_ext2_size[] if the unpacked inputs are large +* xref:memory-size[], unless you want to meet the OOM killer, which is admittedly kind of fun `test.sh` only contains the run commands for the `test` size, and cannot be used for `simsmall`. @@ -10539,7 +10540,7 @@ parsecmgmt -a run -p splash2x.fmm -i test ====== PARSEC uninstall -If you want to remove PARSEC later, Buildroot doesn't provide an automated package removal mechanism: <>, but the following procedure should be satisfactory: +If you want to remove PARSEC later, Buildroot doesn't provide an automated package removal mechanism: xref:remove-buildroot-packages[], but the following procedure should be satisfactory: .... rm -rf \ @@ -10587,7 +10588,7 @@ You may also want to test if your patches are still functionally correct inside === gem5 kernel command line parameters -Analogous <>: +Analogous xref:kernel-command-line-parameters[to QEMU]: .... ./run --arch arm --kernel-cli 'init=/lkmc/linux/poweroff.out' --emulator gem5 @@ -10613,7 +10614,7 @@ Kernel command line: === gem5 GDB step debug ==== gem5 GDB step debug kernel -Analogous <>, on the first shell: +Analogous xref:gdb[to QEMU], on the first shell: .... ./run --arch arm --emulator gem5 --gdb-wait @@ -10633,15 +10634,15 @@ On a third shell: When you want to break, just do a `Ctrl-C` on GDB shell, and then `continue`. -And we now see the boot messages, and then get a shell. Now try the `./count.sh` procedure described for QEMU: <>. +And we now see the boot messages, and then get a shell. Now try the `./count.sh` procedure described for QEMU: xref:gdb-step-debug-kernel-post-boot[]. ==== gem5 GDB step debug userland process -We are unable to use `gdbserver` because of networking: <> +We are unable to use `gdbserver` because of networking: xref:gem5-host-to-guest-networking[] -The alternative is to do as in <>. +The alternative is to do as in xref:gdb-step-debug-userland-processes[]. -Next, follow the exact same steps explained at <>, but passing `--emulator gem5` to every command as usual. +Next, follow the exact same steps explained at xref:gdb-step-debug-userland-non-init-without-gdb-wait[], but passing `--emulator gem5` to every command as usual. But then TODO (I'll still go crazy one of those days): for `arm`, while debugging `./linux/myinsmod.out hello.ko`, after then line: @@ -10662,7 +10663,7 @@ breaks when `m5` is run on guest, but does not show the source code. === gem5 checkpoint -Analogous to QEMU's <>, but better since it can be started from inside the guest, so we can easily checkpoint after a specific guest event, e.g. just before `init` is done. +Analogous to QEMU's xref:snapshot[], but better since it can be started from inside the guest, so we can easily checkpoint after a specific guest event, e.g. just before `init` is done. Documentation: http://gem5.org/Checkpoints @@ -10676,7 +10677,7 @@ In the guest, wait for the boot to end and run: m5 checkpoint .... -where <> is a guest utility present inside the gem5 tree which we cross-compiled and installed into the guest. +where xref:m5[] is a guest utility present inside the gem5 tree which we cross-compiled and installed into the guest. To restore the checkpoint, kill the VM and run: @@ -10709,7 +10710,7 @@ cat f contains the `date`. The file `f` wouldn't exist had we used the first checkpoint with `--gem5-restore 2`, which is the second most recent snapshot taken. -If you automate things with <> as in: +If you automate things with xref:kernel-command-line-parameters[] as in: .... ./run --arch arm --eval 'm5 checkpoint;m5 resetstats;dhrystone 1000;m5 exit' --emulator gem5 @@ -10725,7 +10726,7 @@ since boot has already happened, and the parameters are already in the RAM of th ==== gem5 checkpoint internals -Checkpoints are stored inside the <> at: +Checkpoints are stored inside the xref:m5out-directory[] at: .... "$(./getvar --emulator gem5 m5out_dir)/cpt." @@ -10751,7 +10752,7 @@ The problem is that boot takes forever, and after the checkpoint, the memory and * hack up an existing rc script, since the disk is fixed * inject new kernel boot command line options, since those have already been put into memory by the bootloader -There is however a few loopholes, <> being the simplest, as it reads whatever is present on the host. +There is however a few loopholes, xref:m5-readfile[] being the simplest, as it reads whatever is present on the host. So we can do it like: @@ -10778,12 +10779,12 @@ Since this is such a common setup, we provide the following helpers for this ope * link:rootfs_overlay/lkmc/gem5.sh[]. This script is analogous to gem5's in-tree https://github.com/gem5/gem5/blob/2b4b94d0556c2d03172ebff63f7fc502c3c26ff8/configs/boot/hack_back_ckpt.rcS[hack_back_ckpt.rcS], but with less noise. * `./run --gem5-readfile` is a convenient way to set the `m5 readfile` -Their usage us exemplified at <>. +Their usage us exemplified at xref:gem5-run-benchmark[]. Other loophole possibilities include: -* <<9p>> -* <> +* xref:9p[] +* xref:secondary-disk[] * `expect` as mentioned at: https://stackoverflow.com/questions/7013137/automating-telnet-session-using-bash-scripts + .... @@ -10855,7 +10856,7 @@ Quit the simulation after `1024` instructions: ./run --emulator gem5 -- -I 1024 .... -Can be nicely checked with <>. +Can be nicely checked with xref:gem5-tracing[]. Cycles instead of instructions: @@ -10873,25 +10874,25 @@ Documentation: http://gem5.org/M5ops There are two main ways to use m5ops: -* <> -* <> +* xref:m5[] +* xref:m5ops-instructions[] -`m5` is convenient if you only want to take snapshots before or after the benchmark, without altering its source code. It uses the <> as its backend. +`m5` is convenient if you only want to take snapshots before or after the benchmark, without altering its source code. It uses the xref:m5ops-instructions[] as its backend. `m5` cannot should / should not be used however: * in bare metal setups * when you want to call the instructions from inside interest points of your benchmark. Otherwise you add the syscall overhead to the benchmark, which is more intrusive and might affect results. + -Why not just hardcode some <> as in our example instead, since you are going to modify the source of the benchmark anyways? +Why not just hardcode some xref:m5ops-instructions[] as in our example instead, since you are going to modify the source of the benchmark anyways? ==== m5 -`m5` is a guest command line utility that is installed and run on the guest, that serves as a CLI front-end for the <> +`m5` is a guest command line utility that is installed and run on the guest, that serves as a CLI front-end for the xref:m5ops[] Its source is present in the gem5 tree: https://github.com/gem5/gem5/blob/6925bf55005c118dc2580ba83e0fa10b31839ef9/util/m5/m5.c -It is possible to guess what most tools do from the corresponding <>, but let's at least document the less obvious ones here. +It is possible to guess what most tools do from the corresponding xref:m5ops[], but let's at least document the less obvious ones here. ===== m5 exit @@ -10929,7 +10930,7 @@ as can be seen from the source: https://github.com/gem5/gem5/blob/50a57c0376c02c ===== m5 writefile -Send a guest file to the host. <<9p>> is a more advanced alternative. +Send a guest file to the host. xref:9p[] is a more advanced alternative. Guest: @@ -10972,7 +10973,7 @@ Outcome: date shows on guest. ===== m5 initparam -Ermm, just another <> that only takes integers and only from CLI options? Is this software so redundant? +Ermm, just another xref:m5-readfile[] that only takes integers and only from CLI options? Is this software so redundant? Host: @@ -11019,7 +11020,7 @@ adsf gem5 allocates some magic instructions on unused instruction encodings for convenient guest instrumentation. -Those instructions are exposed through the <> in tree executable. +Those instructions are exposed through the xref:m5[] in tree executable. To make things simpler to understand, you can play around with our own minimized educational `m5` subset link:userland/c/m5ops.c[]. @@ -11038,9 +11039,9 @@ To use that file, first rebuild `m5ops.out` with the m5ops instructions enabled ./build-buildroot --arch aarch64 .... -We don't enable `-DLKMC_M5OPS_ENABLE=1` by default on userland executables because we try to use a single image for both gem5, QEMU and <>, and those instructions would break the latter two. We enable it in the <> by default since we already have different images for QEMU and gem5 there. +We don't enable `-DLKMC_M5OPS_ENABLE=1` by default on userland executables because we try to use a single image for both gem5, QEMU and xref:userland-setup-getting-started-natively[native], and those instructions would break the latter two. We enable it in the xref:baremetal-setup[] by default since we already have different images for QEMU and gem5 there. -Then, from inside <>, test it out with: +Then, from inside xref:gem5-buildroot-setup[], test it out with: .... # checkpoint @@ -11070,7 +11071,7 @@ Bibliography: ===== m5ops instructions interface -Let's study how <> uses them: +Let's study how xref:m5[] uses them: * https://github.com/gem5/gem5/blob/05c4c2b566ce351ab217b2bd7035562aa7a76570/include/gem5/asm/generic/m5ops.h[`include/gem5/asm/generic/m5ops.h`]: defines the magic constants that represent the instructions * https://github.com/gem5/gem5/blob/05c4c2b566ce351ab217b2bd7035562aa7a76570/util/m5/m5op_arm_A64.S[`util/m5/m5op_arm_A64.S`]: use the magic constants that represent the instructions using C preprocessor magic @@ -11145,11 +11146,11 @@ https://gem5.googlesource.com/arm/linux/ contains an ARM Linux kernel forks with The patches are optional: the vanilla kernel does boot. But they add some interesting gem5-specific optimizations, instrumentations and device support. -The patches also <> that are known to work well with gem5. +The patches also xref:notable-alternate-gem5-kernel-configs[add defconfigs] that are known to work well with gem5. E.g. for arm v4.9 there is: https://gem5.googlesource.com/arm/linux/+/917e007a4150d26a0aa95e4f5353ba72753669c7/arch/arm/configs/gem5_defconfig[]. -In order to use those patches and their associated configs, and, we recommend using <> as: +In order to use those patches and their associated configs, and, we recommend using xref:linux-kernel-build-variants[] as: .... git -C "$(./getvar linux_source_dir)" fetch https://gem5.googlesource.com/arm/linux gem5/v4.15:gem5/v4.15 @@ -11182,16 +11183,16 @@ but glibc kernel version checks make init fail with: FATAL: kernel too old .... -because glibc was built to expect a newer Linux kernel: <>. Your choices to sole this are: +because glibc was built to expect a newer Linux kernel: xref:fatal-kernel-too-old[]. Your choices to sole this are: * see if there is a more recent gem5 kernel available, or port your patch of interest to the newest kernel -* modify this repo to use <>, which is not hard because of Buildroot +* modify this repo to use xref:libc-choice[uClibc], which is not hard because of Buildroot * patch glibc to remove that check, which is easy because glibc is in a submodule of this repo It is obviously not possible to understand what they actually do from their commit message, so let's explain them one by one here as we understand them: -* `drm: Add component-aware simple encoder` allows you to see images through VNC: <> -* `gem5: Add support for gem5's extended GIC mode` adds support for more than 8 cores: <> +* `drm: Add component-aware simple encoder` allows you to see images through VNC: xref:gem5-graphic-mode[] +* `gem5: Add support for gem5's extended GIC mode` adds support for more than 8 cores: xref:gem5-arm-more-than-8-cores[] Tested on 649d06d6758cefd080d04dc47fd6a5a26a620874 + 1. @@ -11223,7 +11224,7 @@ The files in that directory contains some very important information about the r Contains UART output, both from the Linux kernel or from the baremetal system. -Can also be seen live on <>. +Can also be seen live on xref:m5term[]. ==== gem5 stats.txt @@ -11294,7 +11295,7 @@ clock=500 Each node has: * a list of child nodes, e.g. `system` is a child of `root`, and both `cpu` and `cpu_clk_domain` are children of `system` -* a list of parameters, e.g. `system.semihosting` is `Null`, which means that <> was turned off +* a list of parameters, e.g. `system.semihosting` is `Null`, which means that xref:semihosting[] was turned off ** the `type` parameter shows is present on every node, and it maps to a `Python` object that inherits from `SimObject`. + For example, `AtomicSimpleCPU` maps is defined at https://github.com/gem5/gem5/blob/05c4c2b566ce351ab217b2bd7035562aa7a76570/src/cpu/simple/AtomicSimpleCPU.py#L45[src/cpu/simple/AtomicSimpleCPU.py]. @@ -11374,7 +11375,7 @@ Disadvantages over `fs.py`: * only works for ARM, not other archs * not as many configuration options as `fs.py`, many things are hardcoded -We setup 2 big and 2 small CPUs, but `cat /proc/cpuinfo` shows 4 identical CPUs instead of 2 of two different types, likely because gem5 does not expose some informational register much like the caches: https://www.mail-archive.com/gem5-users@gem5.org/msg15426.html <> does show that the two big ones are `DerivO3CPU` and the small ones are `MinorCPU`. +We setup 2 big and 2 small CPUs, but `cat /proc/cpuinfo` shows 4 identical CPUs instead of 2 of two different types, likely because gem5 does not expose some informational register much like the caches: https://www.mail-archive.com/gem5-users@gem5.org/msg15426.html xref:config-ini[] does show that the two big ones are `DerivO3CPU` and the small ones are `MinorCPU`. TODO: why is the `--dtb` required despite `fs_bigLITTLE.py` having a DTB generation capability? Without it, nothing shows on terminal, and the simulation terminates with `simulate() limit reached @ 18446744073709551615`. The magic `vmlinux.vexpress_gem5_v1.20170616` works however without a DTB. @@ -11452,7 +11453,7 @@ https://en.wikipedia.org/wiki/Buildroot[Buildroot] is a set of Make scripts that * GCC * Linux kernel -* C standard library: Buildroot supports several implementations, see: <> +* C standard library: Buildroot supports several implementations, see: xref:libc-choice[] * https://en.wikipedia.org/wiki/BusyBox[BusyBox]: provides the shell and basic command line utilities It therefore produces a pristine, blob-less, debuggable setup, where all moving parts are configured to work perfectly together. @@ -11487,7 +11488,7 @@ We provide the following mechanisms: * `./build-buildroot --config-fragment data/br2`: append the Buildroot configuration file `data/br2` to a single build. Must be passed every time you run `./build`. The format is the same as link:buildroot_config/default[]. * `./build-buildroot --config 'BR2_SOME_OPTION="myval"'`: append a single option to a single build. -For example, if you decide to <> after an initial build is finished, you must <> and rebuild: +For example, if you decide to xref:enable-buildroot-compiler-optimizations[] after an initial build is finished, you must xref:clean-the-build[] and rebuild: .... ./build-buildroot \ @@ -11503,7 +11504,7 @@ as explained at: https://buildroot.org/downloads/manual/manual.html#rebuild-pkg The clean is necessary because the source files didn't change, so `make` would just check the timestamps and not build anything. -You will then likely want to make those more permanent with: <> +You will then likely want to make those more permanent with: xref:default-command-line-arguments[] ==== Enable Buildroot compiler optimizations @@ -11523,11 +11524,11 @@ BR2_OPTIMIZE_3=y Our link:buildroot_packages/sample_package[] package correctly forwards the Buildroot options to the build with `$(TARGET_CONFIGURE_OPTS)`, so you don't have to do any extra work. -Don't forget to do that if you are <> with your own build system. +Don't forget to do that if you are xref:add-new-buildroot-packages[adding a new package] with your own build system. Then, you have two choices: -* if you already have a full `-O0` build, you can choose to rebuild just your package of interest to save some time as described at: <> +* if you already have a full `-O0` build, you can choose to rebuild just your package of interest to save some time as described at: xref:custom-buildroot-configs[] + .... ./build-buildroot \ @@ -11543,8 +11544,8 @@ However, this approach might not be representative since calls to an unoptimized + Maybe you can get away with rebuilding libc, but I'm not sure that it will work properly. + -Kernel-wise it should be fine though due to: <> -* <> and rebuild from scratch: +Kernel-wise it should be fine though due to: xref:kernel-o0[] +* xref:clean-the-build[clean the build] and rebuild from scratch: + .... mv out out~ @@ -11619,11 +11620,11 @@ with: First, see if you can't get away without actually adding a new package, for example: * if you have a standalone C file with no dependencies besides the C standard library to be compiled with GCC, just add a new file under link:buildroot_packages/sample_package[] and you are done -* if you have a dependency on a library, first check if Buildroot doesn't have a package for it already with `ls buildroot/package`. If yes, just enable that package as explained at: <> +* if you have a dependency on a library, first check if Buildroot doesn't have a package for it already with `ls buildroot/package`. If yes, just enable that package as explained at: xref:custom-buildroot-configs[] If none of those methods are flexible enough for you, you can just fork or hack up link:buildroot_packages/sample_package[] the sample package to do what you want. -For how to use that package, see: <>. +For how to use that package, see: xref:buildroot_packages-directory[]. Then iterate trying to do what you want and reading the manual until it works: https://buildroot.org/downloads/manual/manual.html @@ -11635,7 +11636,7 @@ Documented at: https://github.com/buildroot/buildroot/blob/2017.08/docs/manual/r Also mentioned at: https://stackoverflow.com/questions/47320800/how-to-clean-only-target-in-buildroot -See this for a sample manual workaround: <>. +See this for a sample manual workaround: xref:parsec-uninstall[]. === BR2_TARGET_ROOTFS_EXT2_SIZE @@ -11663,10 +11664,10 @@ du -hsx "$(./getvar --arch arm buildroot_target_dir)" Some promising ways to overcome this problem include: -* <> +* xref:squashfs[] TODO benchmark: would gem5 suffer a considerable disk read performance hit due to decompressing SquashFS? * libguestfs: https://serverfault.com/questions/246835/convert-directory-to-qemu-kvm-virtual-disk-image/916697#916697[], in particular http://libguestfs.org/guestfish.1.html#vfs-minimum-size[`vfs-minimum-size`] -* use methods described at: <> instead of putting builds on the root filesystem +* use methods described at: xref:gem5-restore-new-script[] instead of putting builds on the root filesystem Bibliography: https://stackoverflow.com/questions/49211241/is-there-a-way-to-automatically-detect-the-minimum-required-br2-target-rootfs-ex @@ -11674,7 +11675,7 @@ Bibliography: https://stackoverflow.com/questions/49211241/is-there-a-way-to-aut https://en.wikipedia.org/wiki/SquashFS[SquashFS] creation with `mksquashfs` does not take fixed sizes, and I have successfully booted from it, but it is readonly, which is unacceptable. -But then we could mount https://wiki.debian.org/ramfs[ramfs] on top of it with <> to make it writable, but my attempts failed exactly as mentioned at <>. +But then we could mount https://wiki.debian.org/ramfs[ramfs] on top of it with xref:overlayfs[] to make it writable, but my attempts failed exactly as mentioned at xref:overlayfs[]. This is the exact unanswered question: https://unix.stackexchange.com/questions/343484/mounting-squashfs-image-with-read-write-overlay-for-rootfs @@ -11739,9 +11740,9 @@ vim "$(./getvar --arch arm run_cmd_file)" ./"$(./getvar --arch arm run_cmd_file)" .... -If you are not already on the master of the given component, you can do that neatly with <>. +If you are not already on the master of the given component, you can do that neatly with xref:build-variants[]. -E.g., to check if a QEMU bug is still present on `master`, you can do as explained at <>: +E.g., to check if a QEMU bug is still present on `master`, you can do as explained at xref:qemu-build-variants[]: .... git -C "$(./getvar qemu_source_dir)" checkout master @@ -11751,9 +11752,9 @@ git -C "$(./getvar qemu_source_dir)" checkout - ./run --qemu-build-id master .... -Then, you will also want to do a <> to pinpoint the exact commit to blame, and CC that developer. +Then, you will also want to do a xref:bisection[] to pinpoint the exact commit to blame, and CC that developer. -Finally, give the images you used save upstream developpers time: <>. +Finally, give the images you used save upstream developpers time: xref:release-zip[]. For Buildroot problems, you should wither provide the config you have: @@ -11778,8 +11779,8 @@ BR2_TOOLCHAIN_BUILDROOT_GLIBC=y Ideally we would like to use uClibc, as it is more minimal and easier to understand, but unfortunately there are some very few packages that use some weird glibc extension that uClibc hasn't implemented yet, e.g.: -* <>. Trivial unmerged fix at: http://lists.busybox.net/pipermail/buildroot/2017-July/197793.html just missing the uClibc option to expose `fts.h`... -* <> +* xref:selinux[]. Trivial unmerged fix at: http://lists.busybox.net/pipermail/buildroot/2017-July/197793.html just missing the uClibc option to expose `fts.h`... +* xref:stress[] The full list of unsupported packages can be found by grepping the Buildroot source: @@ -11787,19 +11788,19 @@ The full list of unsupported packages can be found by grepping the Buildroot sou git -C "$(./getvar buildroot_source_dir)" grep 'depends on BR2_TOOLCHAIN_USES_GLIBC' .... -One "downside" of glibc is that it exercises much more kernel functionality on its more bloated pre-main init, which breaks user mode C hello worlds more often, see: <>. I quote "downside" because glibc is actually exposing emulator bugs which we should actually go and fix. +One "downside" of glibc is that it exercises much more kernel functionality on its more bloated pre-main init, which breaks user mode C hello worlds more often, see: xref:user-mode-simulation-with-glibc[]. I quote "downside" because glibc is actually exposing emulator bugs which we should actually go and fix. == Userland content -This section contains userland content, such as <>, <> and <> examples. +This section contains userland content, such as xref:c[C], xref:cpp[C++] and xref:posix[POSIX] examples. -Userland assembly content is located at: <>. It was split from this section basically becase we were hitting the HTML `h6` limit, stupid web :-) +Userland assembly content is located at: xref:userland-assembly[]. It was split from this section basically becase we were hitting the HTML `h6` limit, stupid web :-) This content makes up the bulk of the link:userland/[] directory. -Getting started at: <> +Getting started at: xref:userland-setup[] -The quickest way to run the arch agnostic examples, which comprise the majority of the examples, is natively with: <> +The quickest way to run the arch agnostic examples, which comprise the majority of the examples, is natively with: xref:userland-setup-getting-started-natively[] This section was originally moved in here from: https://github.com/cirosantilli/cpp-cheat @@ -11844,7 +11845,7 @@ Question: https://stackoverflow.com/questions/24685399/c-empty-struct-what-does- ===== OpenMP -GCC implements the <> threading implementation: https://stackoverflow.com/questions/3949901/pthreads-vs-openmp +GCC implements the xref:OpenMP[] threading implementation: https://stackoverflow.com/questions/3949901/pthreads-vs-openmp Example: link:userland/gcc/openmp.c[] @@ -11856,7 +11857,7 @@ pthreads are more versatile though and allow for a superset of OpenMP. The implementation lives under `libgomp` in the GCC tree, and is documented at: https://gcc.gnu.org/onlinedocs/libgomp/ -`strace` shows that OpenMP makes `clone()` syscalls in Linux. TODO: does it actually call `pthread_` functions, or does it make syscalls directly? Or in other words, can it work on <>? A quick grep shows many references to pthreads. +`strace` shows that OpenMP makes `clone()` syscalls in Linux. TODO: does it actually call `pthread_` functions, or does it make syscalls directly? Or in other words, can it work on xref:freestanding-programs[]? A quick grep shows many references to pthreads. [[cpp]] === C++ @@ -11865,7 +11866,7 @@ Programs under link:userland/cpp/[] are examples of https://en.wikipedia.org/wik * link:userland/cpp/empty.cpp[] * link:userland/cpp/hello.cpp[] -* ``: <> 32 "Atomic operations library" +* ``: xref:cpp17[] 32 "Atomic operations library" ** link:userland/cpp/atomic.cpp[] ==== C++ standards @@ -11896,19 +11897,19 @@ This section will document ISA agnostic concepts, and you should read it first. ISA specifics are covered at: -* <> under link:userland/arch/x86_64/[], originally migrated from: https://github.com/cirosantilli/x86-assembly-cheat -* <> originally migrated from https://github.com/cirosantilli/arm-assembly-cheat under: +* xref:x86-userland-assembly[] under link:userland/arch/x86_64/[], originally migrated from: https://github.com/cirosantilli/x86-assembly-cheat +* xref:arm-userland-assembly[] originally migrated from https://github.com/cirosantilli/arm-assembly-cheat under: ** link:userland/arch/arm/[] ** link:userland/arch/aarch64/[] -Like other userland programs, these programs can be run as explained at: <>. +Like other userland programs, these programs can be run as explained at: xref:userland-setup[]. As a quick reminder, the fastest setups to get started are: -* <> if your host can run the examples, e.g. x86 example on an x86 host: -* <> otherwise +* xref:userland-setup-getting-started-natively[] if your host can run the examples, e.g. x86 example on an x86 host: +* xref:userland-setup-getting-started-with-prebuilt-toolchain-and-qemu-user-mode[] otherwise -However, as usual, it is saner to build your toolchain as explained at: <>. +However, as usual, it is saner to build your toolchain as explained at: xref:qemu-user-mode-getting-started[]. The first examples you should look into are: @@ -11918,18 +11919,18 @@ The first examples you should look into are: ** link:userland/arch/aarch64/add.S[] * mov between register and memory ** link:userland/arch/x86_64/mov.S[] -** <> -** <> +** xref:arm-mov-instruction[] +** xref:arm-load-and-store-instructions[] * addressing modes -** <> -** <> -* registers: <> +** xref:x86-addressing-modes[] +** xref:arm-addressing-modes[] +* registers: xref:assembly-registers[] * jumping: -** <> -** <> +** xref:x86-control-transfer-instructions[] +** xref:arm-branch-instructions[] * SIMD -** <> -** <> +** xref:x86-simd[] +** xref:arm-simd[] The add examples in particular: @@ -11983,16 +11984,16 @@ Other infrastructure sanity checks that you might want to look into include: === Assembly registers -After seeing an <>, you need to learn the general registers: +After seeing an xref:userland-assembly[ADD hello world], you need to learn the general registers: -* x86: <> +* x86: xref:x86-registers[] * arm ** link:userland/arch/arm/registers.S[] * aarch64 ** link:userland/arch/aarch64/registers.S[] ** link:userland/arch/aarch64/pc.S[] -Bibliography: <> A2.3 "ARM core registers". +Bibliography: xref:armarm7[] A2.3 "ARM core registers". ==== ARMv8 aarch64 x31 register @@ -12012,7 +12013,7 @@ mov x0, sp mov x0, xzr .... -and the first one is an alias to ADD while the second an alias to <>. +and the first one is an alias to ADD while the second an alias to xref:arm-bitwise-instructions[ORR]. The difference is documented on a per instruction basis. Instructions that encode 31 as SP say: @@ -12031,7 +12032,7 @@ indicates that the argument takes the value zero, but does not indicate that the as a physical register. ____ -This is also described on <> C1.2.5 "Register names": +This is also described on xref:armarm8[] C1.2.5 "Register names": ____ There is no register named W31 or X31. @@ -12054,29 +12055,29 @@ For this reason, there are sometimes multiple ways to do floating point operatio Let's start as usual with floating point addition + register file: * arm -** <> -** <> +** xref:arm-vadd-instruction[] +** xref:arm-vfp-registers[] * aarch64 -** <> -** <> +** xref:armv8-aarch64-fadd-instruction[] +** xref:armv8-aarch64-floating-point-registers[] === SIMD assembly Much like ADD for non-SIMD, start learning SIMD instructions by looking at the integer and floating point SIMD ADD instructions of each ISA: * x86 -** <> -** <> +** xref:x86-sse-data-transfer-instructions[ADDPD] +** xref:x86-paddq-instruction[] * arm -** <> +** xref:arm-vadd-instruction[] * aarch64 -** <> -** <> +** xref:armv8-aarch64-add-vector-instruction[] +** xref:armv8-aarch64-fadd-instruction[] Then it is just a huge copy paste of infinite boring details: -* <> -* <> +* xref:x86-simd[] +* xref:arm-simd[] To debug these instructoins, you can see the register values in GDB with: @@ -12101,7 +12102,7 @@ Bibliography: https://stackoverflow.com/questions/1389712/getting-started-with-i Fused multiply add: -* x86: <> +* x86: xref:x86-fma[] Bibliography: @@ -12113,7 +12114,7 @@ Particularly important numerical analysis instruction, that is used in particula * Dot product * Matrix multiplication -FMA is so important that <> specifies it with single precision drop compared to a separate add and multiply! +FMA is so important that xref:ieee-754[] specifies it with single precision drop compared to a separate add and multiply! Micro-op fun: https://stackoverflow.com/questions/28630864/how-is-fma-implemented @@ -12123,19 +12124,19 @@ Historically, FMA instructions have been added relatively late to instruction se By "userland assembly", we mean "the parts of the ISA which can be freely used from userland". -Most ISAs are divided into a system and userland part, and to running the system part requires elevated privileges such as <> in x86. +Most ISAs are divided into a system and userland part, and to running the system part requires elevated privileges such as xref:ring0[] in x86. -One big difference between both is that we can run userland assembly on <>, which is easier to get running and debug. +One big difference between both is that we can run userland assembly on xref:userland-setup[], which is easier to get running and debug. -In particular, most userland assembly examples link to the C standard library: <>. +In particular, most userland assembly examples link to the C standard library: xref:userland-assembly-c-standard-library[]. -Userland assembly is generally simpler, and a pre-requisite for <>. +Userland assembly is generally simpler, and a pre-requisite for xref:baremetal-setup[]. -System-land assembly cheats will be put under: <>. +System-land assembly cheats will be put under: xref:baremetal-setup[]. === Userland assembly C standard library -All examples except the <> link to the C standard library. +All examples except the xref:freestanding-programs[] link to the C standard library. This allows using the C standard library for IO, which is very convenient and portable across host OSes. @@ -12153,7 +12154,7 @@ The C standard library infrastructure is implemented in the common userland / ba Unlike most our other assembly examples, which use the C standard library for portability, examples under `freestanding/` directories don't link to the C standard library. -As a result, those examples cannot do IO portably, and so they make raw system calls and only be run on one given OS, e.g. Linux: <>. +As a result, those examples cannot do IO portably, and so they make raw system calls and only be run on one given OS, e.g. Linux: xref:linux-system-calls[]. Such executables are called freestanding because they don't execute the glibc initialization code, but rather start directly on our custom hand written assembly. @@ -12173,7 +12174,7 @@ Examples under `arch//c/` directories show to how use inline assembly from * x86_64 ** link:userland/arch/x86_64/inline_asm/inc.c[] ** link:userland/arch/x86_64/inline_asm/add.c[] -** link:userland/arch/x86_64/inline_asm/sqrt_x87.c[] Shows how to use the <> from inline assembly. Bibliography: https://stackoverflow.com/questions/6514537/how-do-i-specify-immediate-floating-point-numbers-with-inline-assembly/52906126#52906126 +** link:userland/arch/x86_64/inline_asm/sqrt_x87.c[] Shows how to use the xref:x86-x87-fpu-instructions[] from inline assembly. Bibliography: https://stackoverflow.com/questions/6514537/how-do-i-specify-immediate-floating-point-numbers-with-inline-assembly/52906126#52906126 * arm ** link:userland/arch/arm/inline_asm/inc.c[] ** link:userland/arch/arm/inline_asm/inc_memory.c[] @@ -12186,7 +12187,7 @@ Examples under `arch//c/` directories show to how use inline assembly from ==== GCC inline assembly register variables -Used notably in some of the <> setups: +Used notably in some of the xref:linux-system-calls[] setups: * link:userland/arch/arm/inline_asm/reg_var.c[] * link:userland/arch/aarch64/inline_asm/reg_var.c[] @@ -12196,7 +12197,7 @@ In x86, makes it possible to access variables not exposed with the one letter re In arm, it is the only way to achieve this effect: https://stackoverflow.com/questions/10831792/how-to-use-specific-register-in-arm-inline-assembler -This feature notably useful for making system calls from C, see: <>. +This feature notably useful for making system calls from C, see: xref:linux-system-calls[]. Documentation: https://gcc.gnu.org/onlinedocs/gcc-4.4.2/gcc/Explicit-Reg-Vars.html @@ -12292,7 +12293,7 @@ Bibliography: === Linux system calls -The following <> programs illustrate how to make system calls: +The following xref:userland-setup[] programs illustrate how to make system calls: * x86_64 ** link:userland/arch/x86_64/freestanding/linux/hello.S[] @@ -12325,7 +12326,7 @@ Questions about the C inline assembly examples: === Linux calling conventions -Summary: <> +Summary: xref:table-linux-calling-conventions[] [[table-linux-calling-conventions]] .Summary of Linux calling conventions for several architectures @@ -12411,7 +12412,7 @@ Examples: Summary: * x86 always dollar `$` everywhere. -* ARM: can use either `#`, `$` or nothing depending on v7 vs v8 and <>. +* ARM: can use either `#`, `$` or nothing depending on v7 vs v8 and xref:gnu-gas-assembler-arm-unified-syntax[`.syntax unified`]. + Fuller explanation at: https://stackoverflow.com/questions/21652884/is-the-hash-required-for-immediate-values-in-arm-assembly/51987780#51987780 @@ -12428,7 +12429,7 @@ Let's see how many bytes go into each data type: * link:userland/arch/arm/gas_data_sizes.S[] * link:userland/arch/aarch64/gas_data_sizes.S[] -Conclusion: <> +Conclusion: xref:table-gas-data-sizes[] [[table-gas-data-sizes]] .Summary of GNU GAS assembler data sizes @@ -12487,17 +12488,17 @@ The concept of unified assembly is mentioned in ARM's official assembler documen Some of the differences include: -* `#` is optional in unified syntax int literals, see <> +* `#` is optional in unified syntax int literals, see xref:gnu-gas-assembler-immediates[] * many mnemonics changed: ** most of them are condition code position changes, e.g. ANDSEQ vs ANDEQS: https://stackoverflow.com/questions/51184921/wierd-gcc-behaviour-with-arm-assembler-andseq-instruction -** but there are some more drastic ones, e.g. SWI vs <>: https://stackoverflow.com/questions/8459279/are-arm-instructuons-swi-and-svc-exactly-same-thing/54078731#54078731 -* cannot have implicit destination with shift, see: <> +** but there are some more drastic ones, e.g. SWI vs xref:arm-svc-instruction[SVC]: https://stackoverflow.com/questions/8459279/are-arm-instructuons-swi-and-svc-exactly-same-thing/54078731#54078731 +* cannot have implicit destination with shift, see: xref:arm-shift-suffixes[] ===== GNU GAS assembler ARM .n and .w suffixes When reading disassembly, many instructions have either a `.n` or `.w` suffix. -`.n` means narrow, and stands for the <> of an instructions, while `.w` means wide and stands for the ARM encoding. +`.n` means narrow, and stands for the xref:arm-instruction-encodings[Thumb encoding] of an instructions, while `.w` means wide and stands for the ARM encoding. Bibliography: https://stackoverflow.com/questions/27147043/n-suffix-to-branch-instruction @@ -12518,7 +12519,7 @@ fails because cpp treats string and char literals magically. === NOP instructions * x86: link:userland/arch/x86_64/nop.S[NOP] -* ARM: <> +* ARM: xref:arm-nop-instruction[] No OPeration. @@ -12528,7 +12529,7 @@ Applications: https://stackoverflow.com/questions/234906/whats-the-purpose-of-th == x86 userland assembly -Arch agnostic infrastructure getting started at: <>. +Arch agnostic infrastructure getting started at: xref:userland-assembly[]. === x86 registers @@ -12566,8 +12567,8 @@ For the newer x86_64 registers, the naming convention is somewhat saner: Most of the 8 older x86 general purpose registers are not "really" general purpose in the sense that a few instructions magically use them without an explicit encoding. This is reflected in their names: -* RAX: Accumulator. The general place where you add, subtract and otherwise manipulate results in-place. Magic for example for <>. -* RCX, RSI, RDI: Counter, Source and Destination. Used in <> +* RAX: Accumulator. The general place where you add, subtract and otherwise manipulate results in-place. Magic for example for xref:x86-binary-arithmetic-instructions[MUL]. +* RCX, RSI, RDI: Counter, Source and Destination. Used in xref:x86-string-instructions[] ==== x86 FLAGS registers @@ -12607,7 +12608,7 @@ The common compiler usage is: Bibliography: -* <> 3.7.5 "Specifying an Offset" +* xref:intel-manual-1[] 3.7.5 "Specifying an Offset" * https://sourceware.org/binutils/docs-2.18/as/i386_002dMemory.html === x86 data transfer instructions @@ -12619,16 +12620,16 @@ Bibliography: ** link:userland/arch/x86_64/movzx.S[]: MOVZX ** link:userland/arch/x86_64/movsx.S[]: MOVSX * link:userland/arch/x86_64/bswap.S[]: BSWAP: convert between little endian and big endian -* link:userland/arch/x86_64/pushf.S[] PUSHF: <> the <> to / from the stack +* link:userland/arch/x86_64/pushf.S[] PUSHF: xref:x86-push-and-pop-instructions[push and pop] the xref:x86-flags-registers[] to / from the stack ==== x86 exchange instructions -<> 7.3.1.2 "Exchange Instructions": +xref:intel-manual-1[] 7.3.1.2 "Exchange Instructions": * link:userland/arch/x86_64/xadd.S[] XADD: exchange and add. This is how C++ ``'s' `++` is implemented in GCC 5.1. TODO: why is the exchange part needed? * link:userland/arch/x86_64/xchg.S[] XCHG: exchange two values -TODO: concrete multi-thread <> examples of how all those instructions are normally used as synchronization primitives. +TODO: concrete multi-thread xref:gcc-inline-assembly[] examples of how all those instructions are normally used as synchronization primitives. ===== x86 CMPXCHG instruction @@ -12687,7 +12688,7 @@ Has some Intel vs AT&T name overload hell: * https://stackoverflow.com/questions/17170388/trying-to-understand-the-assembly-instruction-cltd-on-x86/50315201#50315201 * https://sourceware.org/binutils/docs/as/i386_002dMnemonics.html -GNU GAS accepts both syntaxes: <> +GNU GAS accepts both syntaxes: xref:table-cqto-cltq[] [[table-cqto-cltq]] .CQTO and CLTQ family Intel vs AT&T @@ -12756,11 +12757,11 @@ Not necessarily faster because of branch prediction. This is partly why the ternary `?` C operator exists: https://stackoverflow.com/questions/3565368/ternary-operator-vs-if-else -It is interesting to compare this with ARMv7 conditional executaion: which is available for all instructions: <> +It is interesting to compare this with ARMv7 conditional executaion: which is available for all instructions: xref:arm-conditional-execution[] === x86 binary arithmetic instructions -<> 5.1.2 "Binary Arithmetic Instructions": +xref:intel-manual-1[] 5.1.2 "Binary Arithmetic Instructions": * link:userland/arch/x86_64/add.S[]: ADD ** link:userland/arch/x86_64/inc.S[]: INC @@ -12779,7 +12780,7 @@ It is interesting to compare this with ARMv7 conditional executaion: which is av === x86 logical instructions -<> 5.1.4 "Logical Instructions" +xref:intel-manual-1[] 5.1.4 "Logical Instructions" * link:userland/arch/x86_64/and.S[]: AND * link:userland/arch/x86_64/not.S[]: NOT @@ -12788,7 +12789,7 @@ It is interesting to compare this with ARMv7 conditional executaion: which is av === x86 shift and rotate instructions -<> 5.1.5 "Shift and Rotate Instructions" +xref:intel-manual-1[] 5.1.5 "Shift and Rotate Instructions" * link:userland/arch/x86_64/shl.S[SHL and SHR] + @@ -12817,7 +12818,7 @@ Like ROL and ROR, but insert the carry bit instead, which effectively generates === x86 bit and byte instructions -<> 5.1.6 "Bit and Byte Instructions" +xref:intel-manual-1[] 5.1.6 "Bit and Byte Instructions" * link:userland/arch/x86_64/bt.S[]: BT + @@ -12842,7 +12843,7 @@ Bibliography: https://stackoverflow.com/questions/1406783/how-to-read-and-write- Count the number of 1 bits. * link:userland/arch/x86_64/test.S[]: TEST + -Like <> but does AND instead of SUB: +Like xref:x86-binary-arithmetic-instructions[CMP] but does AND instead of SUB: + .... ZF = (!(X && Y)) ? 1 : 0 @@ -12850,7 +12851,7 @@ ZF = (!(X && Y)) ? 1 : 0 === x86 control transfer instructions -<> 5.1.7 "Control Transfer Instructions" +xref:intel-manual-1[] 5.1.7 "Control Transfer Instructions" * link:userland/arch/x86_64/jmp.S[]: JMP ** link:userland/arch/x86_64/jmp_indirect.S[]: JMP indirect @@ -12884,11 +12885,11 @@ JG vs JA and JL vs JB: link:userland/arch/x86_64/loop.S[] -Vs <>: https://stackoverflow.com/questions/6805692/x86-assembly-programming-loops-with-ecx-and-loop-instruction-versus-jmp-jcond Holy CISC! +Vs xref:x86-jcc-instructions[Jcc]: https://stackoverflow.com/questions/6805692/x86-assembly-programming-loops-with-ecx-and-loop-instruction-versus-jmp-jcond Holy CISC! ==== x86 string instructions -<> 5.1.8 "String Instructions" +xref:intel-manual-1[] 5.1.8 "String Instructions" These instructions do some operation on an array item, and automatically update the index to the next item: @@ -12906,7 +12907,7 @@ The direction of the index increment depends on the direction flag of the FLAGS These instructions were originally developed to speed up "string" operations such as those present in the `` header of the C standard library. -However, as computer architecture evolved, those instructions might not offer considerable speedups anymore, and modern glibc such as 2.29 just uses <> operations instead:, see also: https://stackoverflow.com/questions/33480999/how-can-the-rep-stosb-instruction-execute-faster-than-the-equivalent-loop +However, as computer architecture evolved, those instructions might not offer considerable speedups anymore, and modern glibc such as 2.29 just uses xref:x86-simd[] operations instead:, see also: https://stackoverflow.com/questions/33480999/how-can-the-rep-stosb-instruction-execute-faster-than-the-equivalent-loop ===== x86 REP prefix @@ -12960,13 +12961,13 @@ which restores RSP and RBP to the values they had before the prologue. === x86 miscellaneous instructions -<> 5.1.13 "Miscellaneous Instructions" +xref:intel-manual-1[] 5.1.13 "Miscellaneous Instructions" -NOP: <> +NOP: xref:nop-instructions[] === x86 random number generator instructions -<> 5.1.15 Random Number Generator Instructions +xref:intel-manual-1[] 5.1.15 Random Number Generator Instructions Example: link:userland/arch/x86_64/rdrand.S[]: RDRAND @@ -12988,7 +12989,7 @@ Fills EAX, EBX, ECX and EDX with CPU information. The exact data to show depends on the value of EAX, and for a few cases instructions ECX. When it depends on ECX, it is called a sub-leaf. Out test program prints `eax == 0`. -On <> for example the output EAX, EBX, ECX and EDX are: +On xref:p51[] for example the output EAX, EBX, ECX and EDX are: .... 0x00000016 @@ -13020,9 +13021,9 @@ There is also the `cpuinfo` command line tool that parses the CPUID instruction === x86 x87 FPU instructions -<> 5.2 "X87 FPU INSTRUCTIONS" +xref:intel-manual-1[] 5.2 "X87 FPU INSTRUCTIONS" -Old floating point unit that you should likely not use anymore, prefer instead the newer <> instructions. +Old floating point unit that you should likely not use anymore, prefer instead the newer xref:x86-simd[] instructions. * FPU basic examples, start here ** link:userland/arch/x86_64/fadd.S[] FADD. The x76 FPU works on a stack of floating point numbers. @@ -13048,7 +13049,7 @@ i r st0 st1 By counting the number of hex digits, we have 20 digits instead of 16! -Instructions such as FLDL convert standard <> 64-bit values from memory into this custom 80-bit format. +Instructions such as FLDL convert standard xref:ieee-754[] 64-bit values from memory into this custom 80-bit format. * https://stackoverflow.com/questions/3206101/extended-80-bit-double-floating-point-in-x87-not-sse2-we-dont-miss-it * https://en.wikipedia.org/wiki/Extended_precision#x86_extended_precision_format @@ -13059,8 +13060,8 @@ https://stackoverflow.com/questions/1844669/benefits-of-x87-over-sse Modern x86 has two main ways of doing floating point operations: -* <> -* <> +* xref:x86-x87-fpu-instructions[] +* xref:x86-simd[] Advantages of FPU: @@ -13072,7 +13073,7 @@ In GCC, you can choose between them with `-mfpmath=`. === x86 SIMD -Parent section: <> +Parent section: xref:simd-assembly[] History: @@ -13087,11 +13088,11 @@ History: ==== x86 SSE instructions -<> 5.5 "SSE INSTRUCTIONS" +xref:intel-manual-1[] 5.5 "SSE INSTRUCTIONS" ===== x86 SSE data transfer instructions -<> 5.5.1.1 "SSE Data Transfer Instructions" +xref:intel-manual-1[] 5.5.1.1 "SSE Data Transfer Instructions" * link:userland/arch/x86_64/movaps.S[]: MOVAPS: move 4 x 32-bits between two XMM registeres or XMM registers and 16-byte aligned memory * link:userland/arch/x86_64/movaps.S[]: MOVUPS: like MOVAPS but also works for unaligned memory @@ -13099,17 +13100,17 @@ History: ===== x86 SSE packed arithmetic instructions -<> 5.5.1.2 "SSE Packed Arithmetic Instructions" +xref:intel-manual-1[] 5.5.1.2 "SSE Packed Arithmetic Instructions" -* link:userland/arch/x86_64/addpd.S[]: ADDPS, ADDPD: good first instruction to learn SIMD: <> +* link:userland/arch/x86_64/addpd.S[]: ADDPS, ADDPD: good first instruction to learn SIMD: xref:simd-assembly[] ===== x86 SSE conversion instructions -<> 5.5.1.6 "SSE Conversion Instructions" +xref:intel-manual-1[] 5.5.1.6 "SSE Conversion Instructions" ==== x86 SSE2 instructions -<> 5.6 "SSE2 INSTRUCTIONS" +xref:intel-manual-1[] 5.6 "SSE2 INSTRUCTIONS" * link:userland/arch/x86_64/cvttss2si.S[]: CVTTSS2SI: convert 32-bit floating point to 32-bit integer, store the result in a general purpose register. Round towards 0. @@ -13117,14 +13118,14 @@ History: link:userland/arch/x86_64/paddq.S[]: PADDQ, PADDL, PADDW, PADDB -Good first instruction to learn SIMD: <> +Good first instruction to learn SIMD: xref:simd-assembly[] [[x86-fma]] ==== x86 fused multiply add (FMA) -<> 5.15 "FUSED-MULTIPLY-ADD (FMA)" +xref:intel-manual-1[] 5.15 "FUSED-MULTIPLY-ADD (FMA)" -* link:userland/arch/x86_64/vfmadd132pd.S[]: VFMADD132PD: "Multiply packed double-precision floating-point values from xmm1 and xmm3/mem, add to xmm2 and put result in xmm1." TODO: but I don't understand the manual, experimentally on <> Ubuntu 19.04 host the result is stored in XMM2! +* link:userland/arch/x86_64/vfmadd132pd.S[]: VFMADD132PD: "Multiply packed double-precision floating-point values from xmm1 and xmm3/mem, add to xmm2 and put result in xmm1." TODO: but I don't understand the manual, experimentally on xref:p51[] Ubuntu 19.04 host the result is stored in XMM2! These instructions were not part of any SSEn set: they actually have a dedicated CPUID flag for it! It appears under `/proc/cpuinfo` as `fma`. They were introduced into AVX512F however. @@ -13132,7 +13133,7 @@ They are also unusual for x86 instructions in that they take 3 operands, as you === x86 system instructions -<> 5.20 "SYSTEM INSTRUCTIONS" +xref:intel-manual-1[] 5.20 "SYSTEM INSTRUCTIONS" ==== x86 RDTSC instruction @@ -13149,9 +13150,9 @@ while true; do sleep 1 && ./userland/arch/x86_64/rdtsc.out; done RDTSC stores its output to EDX:EAX, even in 64-bit mode, top bits are zeroed out. -TODO: review this section, make a more controlled userland experiment with <> instrumentation. +TODO: review this section, make a more controlled userland experiment with xref:m5ops[] instrumentation. -Let's have some fun and try to correlate the gem5 <> `system.cpu.numCycles` cycle count with the https://en.wikipedia.org/wiki/Time_Stamp_Counter[x86 RDTSC instruction] that is supposed to do the same thing: +Let's have some fun and try to correlate the gem5 xref:gem5-stats-txt[] `system.cpu.numCycles` cycle count with the https://en.wikipedia.org/wiki/Time_Stamp_Counter[x86 RDTSC instruction] that is supposed to do the same thing: .... ./build-userland --static userland/arch/x86_64/inline_asm/rdtsc.S @@ -13198,13 +13199,13 @@ produces: .... -There is also the RDPID instruction that reads just the processor ID, but it appears to be very new for QEMU 4.0.0 or <>, as it fails with SIGILL on both. +There is also the RDPID instruction that reads just the processor ID, but it appears to be very new for QEMU 4.0.0 or xref:p51[], as it fails with SIGILL on both. Bibliography: https://stackoverflow.com/questions/22310028/is-there-an-x86-instruction-to-tell-which-core-the-instruction-is-being-run-on/56622112#56622112 ===== ARM PMCCNTR register -TODO We didn't manage to find a working ARM analogue to <>: link:kernel_modules/pmccntr.c[] is oopsing, and even it if weren't, it likely won't give the cycle count since boot since it needs to be activate before it starts counting anything: +TODO We didn't manage to find a working ARM analogue to xref:x86-rdtsc-instruction[]: link:kernel_modules/pmccntr.c[] is oopsing, and even it if weren't, it likely won't give the cycle count since boot since it needs to be activate before it starts counting anything: * https://stackoverflow.com/questions/40454157/is-there-an-equivalent-instruction-to-rdtsc-in-arm * https://stackoverflow.com/questions/31620375/arm-cortex-a7-returning-pmccntr-0-in-kernel-mode-and-illegal-instruction-in-u/31649809#31649809 @@ -13216,7 +13217,7 @@ Inline assembly example at: link:userland/cpp/atomic.cpp[] Ensures that memory modifications are visible across all CPUs, which is fundamental for thread synchronization. -Apparently already automatically implied by some of the <> +Apparently already automatically implied by some of the xref:x86-exchange-instructions[] Bibliography: @@ -13262,9 +13263,9 @@ Model specific extensions: http://web.archive.org/web/20190606075325/https://sof == ARM userland assembly -Arch general getting started at: <>. +Arch general getting started at: xref:userland-assembly[]. -Instructions here loosely grouped based on that of the <> Chapter A4 "The Instruction Sets". +Instructions here loosely grouped based on that of the xref:armarm7[] Chapter A4 "The Instruction Sets". We cover here mostly ARMv7, and then treat aarch64 differentially, since much of the ARMv7 userland is the same in aarch32. @@ -13272,7 +13273,7 @@ We cover here mostly ARMv7, and then treat aarch64 differentially, since much of The https://en.wikipedia.org/wiki/ARM_architecture[ARM architecture] is has been used on the vast majority of mobile phones in the 2010's, and on a large fraction of micro controllers. -It competes with <> because its implementations are designed for low power consumption, which is a major requirement of the cell phone market. +It competes with xref:x86-userland-assembly[] because its implementations are designed for low power consumption, which is a major requirement of the cell phone market. ARM is generally considered a RISC instruction set, although there are some more complex instructions which would not generally be classified as purely RISC. @@ -13282,11 +13283,11 @@ ARM Holdings was bought by the Japanese giant SoftBank in 2016. ==== ARMv8 vs ARMv7 vs AArch64 vs AArch32 -ARMv7 is the older architecture described at: <>. +ARMv7 is the older architecture described at: xref:armarm7[]. -ARMv8 is the newer architecture ISA https://developer.arm.com/docs/den0024/latest/preface[released in 2013] and described at: <>. It can be in either of two states: +ARMv8 is the newer architecture ISA https://developer.arm.com/docs/den0024/latest/preface[released in 2013] and described at: xref:armarm8[]. It can be in either of two states: -* <> +* xref:aarch32[] * aarch64 In the lose terminology of this repository: @@ -13302,7 +13303,7 @@ ARMv8 has https://en.wikipedia.org/wiki/ARM_architecture#ARMv8-A[had several upd * v8.4: TODO * v8.5: 2018 -They are described at: <> A1.7 "ARMv8 architecture extensions". +They are described at: xref:armarm8[] A1.7 "ARMv8 architecture extensions". ===== AArch32 @@ -13317,7 +13318,7 @@ For this reason, QEMU and GAS seems to enable both AArch32 and ARMv7 under `arm` There are however some extensions over ARMv7, many of them are functionality that ARMv8 has and that designers decided to backport on AArch32 as well, e.g.: -* <> +* xref:armv8-aarch32-vcvta-instruction[] ===== AArch32 vs AArch64 @@ -13325,8 +13326,8 @@ A great summary of differences can be found at: https://en.wikipedia.org/wiki/AR Some random ones: -* aarch32 has two encodings: Thumb and ARM: <> -* in ARMv8, the stack can be enforced to 16-byte alignment: <> +* aarch32 has two encodings: Thumb and ARM: xref:arm-instruction-encodings[] +* in ARMv8, the stack can be enforced to 16-byte alignment: xref:armv8-aarch64-stack-alignment[] ==== Free ARM implementations @@ -13356,7 +13357,7 @@ Bibliography: https://www.quora.com/Why-is-it-that-you-need-a-license-from-ARM-t ==== ARM instruction encodings -Understanding the basics of instruction encodings is fundamental to help you to remember what instructions do and why some things are possible or not, notably the <> and the <>. +Understanding the basics of instruction encodings is fundamental to help you to remember what instructions do and why some things are possible or not, notably the xref:arm-ldr-pseudo-instruction[] and the xref:arm-adr-instruction[ADRP instruction]. aarch32 has two "instruction sets", which to look just like encodings. @@ -13365,20 +13366,20 @@ The encodings are: * A32: every instruction is 4 bytes long. Can encode every instruction. * T32: most common instructions are 2 bytes long. Many others less common ones are 4 bytes long. + -T stands for "Thumb", which is the original name for the technology, <> A1.3.2 "The ARM instruction sets" says: +T stands for "Thumb", which is the original name for the technology, xref:armarm8[] A1.3.2 "The ARM instruction sets" says: + ____ In previous documentation, these instruction sets were called the ARM and Thumb instruction sets ____ + -See also: <> F2.1.3 "Instruction encodings". +See also: xref:armarm8[] F2.1.3 "Instruction encodings". Within each instruction set, there can be multiple encodings for a given function, and they are noted simply as: * A1, A2, ...: A32 encodings * T1, T2, ..m: T32 encodings -The state bit `PSTATE.T` determines if the processor is in thumb mode or not. <> says that this bit it can only be read from <> +The state bit `PSTATE.T` determines if the processor is in thumb mode or not. xref:armarm8[] says that this bit it can only be read from xref:arm-bx-instruction[] https://stackoverflow.com/questions/22660025/how-can-i-tell-if-i-am-in-arm-mode-or-thumb-mode-in-gdb @@ -13478,7 +13479,7 @@ The current ARM / Thumb mode is encoded in the least significant bit of lr. ===== ARM BX instruction -See: <> +See: xref:arm-thumb-encoding[] ===== ARMv8 aarch64 ret instruction @@ -13505,17 +13506,17 @@ Very handy! ==== ARM conditional execution -Weirdly, <> and family are not the only instructions that can execute conditionally on the flags: the same also applies to most instructions, e.g. ADD. +Weirdly, xref:arm-b-instruction[] and family are not the only instructions that can execute conditionally on the flags: the same also applies to most instructions, e.g. ADD. Example: link:userland/arch/arm/cond.S[] Just add the usual `eq`, `ne`, etc. suffixes just as for B. -The list of all extensions is documented at <> "A8.3 Conditional execution". +The list of all extensions is documented at xref:armarm7[] "A8.3 Conditional execution". === ARM load and store instructions -In ARM, there are only two instruction families that do memory access: <> to load and <> to store. +In ARM, there are only two instruction families that do memory access: xref:arm-ldr-instruction[] to load and xref:arm-str-instruction[] to store. Everything else works on register and immediates. @@ -13573,12 +13574,12 @@ The offset itself can come from the following sources: The indexed modes are convenient to loop over arrays. -Bibliography: <>: +Bibliography: xref:armarm7[]: * A4.6.5 "Addressing modes" * A8.5 "Memory accesses" -<>: C1.3.3 "Load/Store addressing modes" +xref:armarm8[]: C1.3.3 "Load/Store addressing modes" ====== ARM loop over array @@ -13599,19 +13600,19 @@ Store from memory into registers. Example: link:userland/arch/arm/str.S[] -Basically everything that applies to <> also applies here so we won't go into much detail. +Basically everything that applies to xref:arm-ldr-instruction[] also applies here so we won't go into much detail. ===== ARMv8 aarch64 STR instruction PC-relative STR is not possible in aarch64. -For LDR it works <>. +For LDR it works xref:arm-ldr-instruction[as in aarch32]. As a result, it is not possible to load from the literal pool for STR. Example: link:userland/arch/aarch64/str.S[] -This can be seen from <> C3.2.1 "Load/Store register": LDR simply has on extra PC encoding that STR does not. +This can be seen from xref:armarm8[] C3.2.1 "Load/Store register": LDR simply has on extra PC encoding that STR does not. ===== ARMv8 aarch64 LDP and STP instructions @@ -13623,21 +13624,21 @@ TODO minimal example. Currently used in `LKMC_PROLOGUE` at link:lkmc/aarch64.h[] In ARMv8, the stack can be enforced to 16-byte alignment. -This is why the main way to push things to stack is with 8-byte pair pushes with the <>. +This is why the main way to push things to stack is with 8-byte pair pushes with the xref:armv8-aarch64-ldp-and-stp-instructions[]. -<> C1.3.3 "Load/Store addressing modes" says: +xref:armarm8-db[] C1.3.3 "Load/Store addressing modes" says: ____ When stack alignment checking is enabled by system software and the base register is the SP, the current stack pointer must be initially quadword aligned, that is aligned to 16 bytes. Misalignment generates a Stack Alignment fault. The offset does not have to be a multiple of 16 bytes unless the specific Load/Store instruction requires this. SP cannot be used as a register offset. ____ -<> C3.2 "Loads and stores" says: +xref:armarm8-db[] C3.2 "Loads and stores" says: ____ The additional control bits SCTLR_ELx.SA and SCTLR_EL1.SA0 control whether the stack pointer must be quadword aligned when used as a base register. See SP alignment checking on page D1-2164. Using a misaligned stack pointer generates an SP alignment fault exception. ____ -<> D1.8.2 "SP alignment checking" is then the main section. +xref:armarm8-db[] D1.8.2 "SP alignment checking" is then the main section. TODO: what does the ABI say on this? Why don't I observe faults on QEMU as mentioned at: https://stackoverflow.com/questions/212466/what-is-a-bus-error/31877230#31877230 @@ -13673,7 +13674,7 @@ The registers are encoded as single bits inside the instruction: each bit repres As a consequence, the push order is fixed no matter how you write the assembly instruction: there is just not enough space to encode ordering. -AArch64 loses those instructions, likely because it was not possible anymore to encode all registers: https://stackoverflow.com/questions/27941220/push-lr-and-pop-lr-in-arm-arch64 and replaces them with the <> +AArch64 loses those instructions, likely because it was not possible anymore to encode all registers: https://stackoverflow.com/questions/27941220/push-lr-and-pop-lr-in-arm-arch64 and replaces them with the xref:armv8-aarch64-ldp-and-stp-instructions[] === ARM data processing instructions @@ -13741,14 +13742,14 @@ UBFX dest, src, lsb, width does: .... -dest = (src & ((1 << width) - 1)) >> lsb; +dest = (src & ((1 xref: width) - 1)) [] lsb; .... Bibliography: https://stackoverflow.com/questions/8366625/arm-bit-field-extract ===== ARM BFM instruction -TODO: explain. Similar to <> but leave untouched bits unmodified. +TODO: explain. Similar to xref:arm-ubfm-instruction[UBFM] but leave untouched bits unmodified. ====== ARM BFI instruction @@ -13759,18 +13760,18 @@ Examples: Move the lower bits of source register into any position in the destination: -* ARMv8: an alias for <> +* ARMv8: an alias for xref:arm-bfm-instruction[] * ARMv7: a real instruction ==== ARM MOV instruction Move an immediate to a register, or a register to another register. -Cannot load from or to memory, since only the LDR and STR instruction families can do that in ARM: <> +Cannot load from or to memory, since only the LDR and STR instruction families can do that in ARM: xref:arm-load-and-store-instructions[] Example: link:userland/arch/arm/mov.S[] -Since every instruction <>, there is not enough space to encode arbitrary 32-bit immediates in a single instruction, since some of the bits are needed to actually encode the instruction itself. +Since every instruction xref:arm-instruction-encodings[has a fixed 4 byte size], there is not enough space to encode arbitrary 32-bit immediates in a single instruction, since some of the bits are needed to actually encode the instruction itself. The solutions to this problem are mentioned at: @@ -13779,10 +13780,10 @@ The solutions to this problem are mentioned at: Summary of solutions: -* <> +* xref:arm-movw-and-movt-instructions[] * place it in memory. But then how to load the address, which is also a 32-bit value? ** use pc-relative addressing if the memory is close enough -** use <> encodable shifted immediates +** use xref:arm-bitwise-instructions[ORR] encodable shifted immediates The blog article summarizes nicely which immediates can be encoded and the design rationale: @@ -13800,13 +13801,13 @@ Set the higher or lower 16 bits of a register to an immediate in one go. Example: link:userland/arch/arm/movw.S[] -The armv8 version analogue is <>. +The armv8 version analogue is xref:armv8-aarch64-movk-instruction[]. ===== ARMv8 aarch64 movk instruction Fill a 64 bit register with 4 16-bit instructions one at a time. -Similar to <> in v7. +Similar to xref:arm-movw-and-movt-instructions[] in v7. Example: link:userland/arch/aarch64/movk.S[] @@ -13832,13 +13833,13 @@ The shift types are: * ROR: Rotate Right / Left. Wrap bits around. * ASR: Arithmetic Shift Right. Keep sign. -Documented at: <> "A4.4.1 Standard data-processing instructions" +Documented at: xref:armarm7[] "A4.4.1 Standard data-processing instructions" ===== ARM S suffix Example: link:userland/arch/arm/s_suffix.S[] -The `S` suffix, present on most <>, makes the instruction also set the Status register flags that control conditional jumps. +The `S` suffix, present on most xref:arm-data-processing-instructions[], makes the instruction also set the Status register flags that control conditional jumps. If the result of the operation is `0`, then it triggers BEQ, since comparison is a subtraction, with success on 0. @@ -13846,7 +13847,7 @@ CMP sets the flags by default of course. ==== ARM ADR instruction -Similar rationale to the <>, allowing to easily store a PC-relative reachable address into a register in one go, to overcome the 4-byte fixed instruction size. +Similar rationale to the xref:arm-ldr-pseudo-instruction[], allowing to easily store a PC-relative reachable address into a register in one go, to overcome the 4-byte fixed instruction size. Examples: @@ -13858,13 +13859,13 @@ More details: https://stackoverflow.com/questions/41906688/what-are-the-semantic ===== ARM ADRL instruction -See: <>. +See: xref:arm-adr-instruction[]. === ARM miscellaneous instructions ==== ARM NOP instruction -Parent section: <> +Parent section: xref:nop-instructions[] There are a few different ways to encode NOP, notably MOV a register into itself, and a dedicated miscellaneous instruction. @@ -13885,11 +13886,11 @@ Guaranteed undefined! Therefore raise illegal instruction signal. Used by GCC `_ * link:userland/arch/arm/udf.S[] * link:userland/arch/aarch64/udf.S[] -Why GNU GAS 2.29 does not have a mnemonic for it in A64 because it is very recent: shows in <> but not `ca`. +Why GNU GAS 2.29 does not have a mnemonic for it in A64 because it is very recent: shows in xref:armarm8-db[] but not `ca`. === ARM SIMD -Parent section: <> +Parent section: xref:simd-assembly[] ==== ARM VFP @@ -13910,7 +13911,7 @@ When a certain version of VFP is present on a CPU, the compiler prefix typically Bibliography: -* <> Appendix D6 "Common VFP Subarchitecture Specification". It is not part of the ISA, but just an extension. TODO: that spec does not seem to have the instructions documented, and instruction like VMOV just live with the main instructions. Is VMOV part of VFP? +* xref:armarm7[] Appendix D6 "Common VFP Subarchitecture Specification". It is not part of the ISA, but just an extension. TODO: that spec does not seem to have the instructions documented, and instruction like VMOV just live with the main instructions. Is VMOV part of VFP? * https://mindplusplus.wordpress.com/2013/06/25/arm-vfp-vector-programming-part-1-introduction/ * https://en.wikipedia.org/wiki/ARM_architecture#Floating-point_(VFP) @@ -13918,7 +13919,7 @@ Bibliography: TODO example -<> E1.3.1 "The SIMD and floating-point register file" Figure E1-1 "SIMD and floating-point register file, AArch32 operation": +xref:armarm8[] E1.3.1 "The SIMD and floating-point register file" Figure E1-1 "SIMD and floating-point register file, AArch32 operation": .... +-----+-----+-----+ @@ -13946,8 +13947,8 @@ And you can't access the higher bytes at D16 or greater with Sn. ===== ARM VADD instruction -* link:userland/arch/arm/vadd_scalar.S[]: see also: <> -* link:userland/arch/arm/vadd_vector.S[]: see also: <> +* link:userland/arch/arm/vadd_scalar.S[]: see also: xref:floating-point-assembly[] +* link:userland/arch/arm/vadd_vector.S[]: see also: xref:simd-assembly[] ===== ARM VCVT instruction @@ -13955,7 +13956,7 @@ Example: link:userland/arch/arm/vcvt.S[] Convert between integers and floating point. -<> on rounding: +xref:armarm7[] on rounding: ____ The floating-point to fixed-point operation uses the Round towards Zero rounding mode. The fixed-point to floating-point operation uses the Round to Nearest rounding mode. @@ -13973,9 +13974,9 @@ vld1.32.f32 Example: link:userland/arch/arm/vcvtr.S[] -Like <>, but the rounding mode is selected by the FPSCR.RMode field. +Like xref:arm-vcvt-instruction[], but the rounding mode is selected by the FPSCR.RMode field. -Selecting rounding mode explicitly per instruction was apparently not possible in ARMv7, but was made possible in <> e.g. with <>. +Selecting rounding mode explicitly per instruction was apparently not possible in ARMv7, but was made possible in xref:aarch32[] e.g. with xref:armv8-aarch32-vcvta-instruction[]. Rounding mode selection is exposed in the ANSI C standard through https://en.cppreference.com/w/c/numeric/fenv/feround[`fesetround`]. @@ -13985,9 +13986,9 @@ TODO: is the initial rounding mode specified by the ELF standard? Could not find Example: link:userland/arch/arm/vcvt.S[] -Added in ARMv8 <> only, not present in ARMv7. +Added in ARMv8 xref:aarch32[] only, not present in ARMv7. -In ARMv7, to use a non-round-to-zero rounding mode, you had to set the rounding mode with FPSCR and use the R version of the instruction e.g. <>. +In ARMv7, to use a non-round-to-zero rounding mode, you had to set the rounding mode with FPSCR and use the R version of the instruction e.g. xref:arm-vcvtr-instruction[]. Now in AArch32 it is possible to do it explicitly per-instruction. @@ -13995,17 +13996,17 @@ Also there was no ties to away mode in ARMv7. This mode does not exist in C99 ei ==== ARMv8 Advanced SIMD and floating-point support -The <> specifies floating point and SIMD support in the main architecture at A1.5 "Advanced SIMD and floating-point support". +The xref:armarm8[] specifies floating point and SIMD support in the main architecture at A1.5 "Advanced SIMD and floating-point support". The feature is often refered to simply as "SIMD&FP" throughout the manual. -The Linux kernel shows `/proc/cpuinfo` compatibility as `neon`, which is yet another intermediate name that came up at some point: <> +The Linux kernel shows `/proc/cpuinfo` compatibility as `neon`, which is yet another intermediate name that came up at some point: xref:arm-neon[] -Vs <>: https://stackoverflow.com/questions/4097034/arm-cortex-a8-whats-the-difference-between-vfp-and-neon +Vs xref:arm-vfp[]: https://stackoverflow.com/questions/4097034/arm-cortex-a8-whats-the-difference-between-vfp-and-neon ===== ARMv8 floating point availability -Support is semi-mandatory. <> A1.5 "Advanced SIMD and floating-point support": +Support is semi-mandatory. xref:armarm8[] A1.5 "Advanced SIMD and floating-point support": ____ ARMv8 can support the following levels of support for Advanced SIMD and floating-point instructions: @@ -14021,13 +14022,13 @@ ____ Therefore it is in theory optional, but highly available. -This is unlike ARMv7, where floating point is completely optional through <>. +This is unlike ARMv7, where floating point is completely optional through xref:arm-vfp[]. ===== ARM NEON Just an informal name for the "Advanced SIMD instructions"? Very confusing. -<> F2.9 "Additional information about Advanced SIMD and floating-point instructions" says: +xref:armarm8[] F2.9 "Additional information about Advanced SIMD and floating-point instructions" says: ____ The Advanced SIMD architecture, its associated implementations, and supporting software, are commonly referred to as NEON technology. @@ -14043,7 +14044,7 @@ ____ TODO example. -<> B1.2.1 "Registers in AArch64 state" describes the registers: +xref:armarm8[] B1.2.1 "Registers in AArch64 state" describes the registers: ____ 32 SIMD&FP registers, V0 to V31. Each register can be accessed as: @@ -14061,24 +14062,24 @@ Notice how Sn is very different between v7 and v8! In v7 it goes across Dn, and link:userland/arch/aarch64/add_vector.S[] -Good first instruction to learn SIMD: <> +Good first instruction to learn SIMD: xref:simd-assembly[] ===== ARMv8 aarch64 FADD instruction -* link:userland/arch/aarch64/fadd_vector.S[]: see also: <> -* link:userland/arch/aarch64/fadd_scalar.S[]: see also: <> +* link:userland/arch/aarch64/fadd_vector.S[]: see also: xref:simd-assembly[] +* link:userland/arch/aarch64/fadd_scalar.S[]: see also: xref:floating-point-assembly[] ====== ARM FADD vs VADD -It is very confusing, but FADDS and FADDD in Aarch32 are <> for `vadd.f32` and `vadd.f64` which we use in this tutorial: <> +It is very confusing, but FADDS and FADDD in Aarch32 are xref:gnu-gas-assembler-arm-unified-syntax[pre-UAL] for `vadd.f32` and `vadd.f64` which we use in this tutorial: xref:arm-vadd-instruction[] The same goes for most ARMv7 mnemonics: `f*` is old, and `v*` is the newer better syntax. -But then, in ARMv8, they decided to use <> as the main floating point add name, and get rid of VADD! +But then, in ARMv8, they decided to use xref:armv8-aarch64-fadd-instruction[] as the main floating point add name, and get rid of VADD! Also keep in mind that fused multiply add is FMADD. -Examples at: <> +Examples at: xref:simd-assembly[] ===== ARMv8 aarch64 ld2 instruction @@ -14086,7 +14087,7 @@ Example: link:userland/arch/aarch64/ld2.S[] We can load multiple vectors interleaved from memory in one single instruction! -This is why the `ldN` instructions take an argument list denoted by `{}` for the registers, much like armv7 <>. +This is why the `ldN` instructions take an argument list denoted by `{}` for the registers, much like armv7 xref:arm-ldmia-instruction[]. There are analogous LD3 and LD4 instruction. @@ -14103,7 +14104,7 @@ Example: link:userland/arch/aarch64/sve.S[] Scalable Vector Extension. -aarch64 only, newer than <>. +aarch64 only, newer than xref:arm-neon[]. It is called Scalable because it does not specify the vector width! Therefore we don't have to worry about new vector width instructions every few years! Hurray! @@ -14126,7 +14127,7 @@ Official spec: https://developer.arm.com/docs/100891/latest/sve-overview/introdu ====== SVE spec -<> A1.7 "ARMv8 architecture extensions" says: +xref:armarm8[] A1.7 "ARMv8 architecture extensions" says: ____ SVE is an optional extension to ARMv8.2. That is, SVE requires the implementation of ARMv8.2. @@ -14142,12 +14143,12 @@ That document then describes the SVE instructions and registers. ==== ARMv8.1 architecture extension -<> A1.7.3 "The ARMv8.1 architecture extension" +xref:armarm8-db[] A1.7.3 "The ARMv8.1 architecture extension" [[arm-lse]] ===== ARM Large System Extensions (LSE) -<> "ARMv8.1-LSE, ARMv8.1 Large System Extensions" +xref:armarm8-db[] "ARMv8.1-LSE, ARMv8.1 Large System Extensions" * LDADD: link:userland/cpp/atomic.cpp[] @@ -14232,7 +14233,7 @@ Latest version at: https://developer.arm.com/docs/den0024/latest/preface ARM also releases documentation specific to each given processor. -This adds extra details to the more portable <> ISA documentation. +This adds extra details to the more portable xref:armarm8[] ISA documentation. [[arm-cortex15-trm]] ===== ARM Cortex-A15 MPCore Processor Technical Reference Manual r4p0 @@ -14255,11 +14256,11 @@ Examples: == Baremetal -Getting started at: <> +Getting started at: xref:baremetal-setup[] === Baremetal GDB step debug -GDB step debug works on baremetal exactly as it does on the Linux kernel: <>. +GDB step debug works on baremetal exactly as it does on the Linux kernel: xref:gdb[]. Except that is is even cooler here since we can easily control and understand every single instruction that is being run! @@ -14275,13 +14276,13 @@ then on the second shell: ./run-gdb --arch arm --baremetal userland/c/hello.c -- main .... -Or if you are a <>, do everything in one go with: +Or if you are a xref:tmux[tmux pro], do everything in one go with: .... ./run --arch arm --baremetal userland/c/hello.c --gdb .... -Alternatively, to start from the very first executed instruction of our tiny <>: +Alternatively, to start from the very first executed instruction of our tiny xref:baremetal-bootloaders[]: .... ./run \ @@ -14309,7 +14310,7 @@ The cool thing about those examples is that you start at the very first instruct === Baremetal bootloaders -As can be seen from <>, all examples under link:baremetal/[], with the exception of `baremetal/arch//no_bootloader`, start from our tiny bootloaders: +As can be seen from xref:baremetal-gdb-step-debug[], all examples under link:baremetal/[], with the exception of `baremetal/arch//no_bootloader`, start from our tiny bootloaders: * link:baremetal/lib/arm.S[] * link:baremetal/lib/aarch64.S[] @@ -14322,7 +14323,7 @@ Out simplistic bootloaders basically setup up just enough system state to allow The most important things that we setup in the bootloaders are: * the stack pointer -* NEON: <> +* NEON: xref:aarch64-baremetal-neon-setup[] * TODO: we don't do this currently but maybe we should setup BSS The C functions that become available as a result are: @@ -14338,7 +14339,7 @@ For this reason, we tend to create examples with bootloaders, as it is easier to Semihosting is a publicly documented interface specified by ARM Holdings that allows us to do some magic operations very useful in development. -Semihosting is implemented both on some real devices and on simulators such as QEMU and <>. +Semihosting is implemented both on some real devices and on simulators such as QEMU and xref:gem5-semihosting[]. It is documented at: https://developer.arm.com/docs/100863/latest/introduction @@ -14370,7 +14371,7 @@ Other magic operations we can do with semihosting besides exiting the on the hos Alternatives exist for some semihosting operations, e.g.: * UART IO for host stdin and stdout in both emulators and real hardware -* <> for <>, e.g. `m5 exit` makes the emulator quit +* xref:m5ops[] for xref:gem5[], e.g. `m5 exit` makes the emulator quit The big advantage of semihosting is that it is standardized across all ARM boards, and therefore allows you to make a single image that does those magic operations instead of having to compile multiple images with different magic addresses. @@ -14537,7 +14538,7 @@ Since I had this compiled, I also decided to try it out on userland. I was also able to run a freestanding Linux userland example on it: https://github.com/cirosantilli/arm-assembly-cheat/blob/cd232dcaf32c0ba6399b407e0b143d19b6ec15f4/v7/linux/hello.S -It just ignores the <> however, and does not forward syscalls to the host like QEMU does. +It just ignores the xref:arm-svc-instruction[] however, and does not forward syscalls to the host like QEMU does. Then I tried a glibc example: https://github.com/cirosantilli/arm-assembly-cheat/blob/cd232dcaf32c0ba6399b407e0b143d19b6ec15f4/v7/mov.S @@ -14563,7 +14564,7 @@ Userland information can be found at: https://github.com/cirosantilli/arm-assemb ==== ARM exception levels -ARM exception levels are analogous to x86 <>. +ARM exception levels are analogous to x86 xref:ring0[rings]. The current EL can be determined by reading from certain registers, which we do with bit disassembly at: @@ -14652,7 +14653,7 @@ vmsr fpexc, r0 in link:baremetal/lib/arm.S[]. That patch however enables SIMD in baremetal, which I feel is more important. -According to <>, access to that register is controlled by other registers `NSACR.{CP11, CP10}` and `HCPTR` so those must be turned off, but I'm lazy to investigate now, even just trying to dump those registers in link:userland/arch/arm/dump_regs.c[] also leads to exceptions... +According to xref:armarm7[], access to that register is controlled by other registers `NSACR.{CP11, CP10}` and `HCPTR` so those must be turned off, but I'm lazy to investigate now, even just trying to dump those registers in link:userland/arch/arm/dump_regs.c[] also leads to exceptions... ==== ARM SVC instruction @@ -14764,11 +14765,11 @@ contains: So we see in both cases that the SVC is done, then an exception happens, and then we just continue running from the exception handler address. -The vector table format is described on <> Table D1-7 "Vector offsets from vector table base address". +The vector table format is described on xref:armarm8[] Table D1-7 "Vector offsets from vector table base address". -A good representation of the format of the vector table can also be found at <> Table 10-2 "Vector table offsets from vector table base address". +A good representation of the format of the vector table can also be found at xref:armv8-programmers-guide[] Table 10-2 "Vector table offsets from vector table base address". -The first part of the table contains: <> +The first part of the table contains: xref:table-armv8-vector-handlers[] [[table-armv8-vector-handlers]] .Summary of ARMv8 vector handlers @@ -14796,19 +14797,19 @@ The first part of the table contains: <> and the following other parts are analogous, but referring to SPx and lower ELs. -We are going to do everything in <> for now. +We are going to do everything in xref:arm-exception-levels[EL1] for now. On the terminal output, we observe the initial values of: * DAIF: 0x3c0, i.e. 4 bits (6 to 9) set to 1, which means that exceptions are masked for each exception type: Synchronous, System error, IRQ and FIQ. + -This reset value is defined by <> C5.2.2 "DAIF, Interrupt Mask Bits". +This reset value is defined by xref:armarm8[] C5.2.2 "DAIF, Interrupt Mask Bits". * SPSel: 0x1, which means: use SPx instead of SP0. + -This reset value is defined by <> C5.2.16 "SPSel, Stack Pointer Select". +This reset value is defined by xref:armarm8[] C5.2.16 "SPSel, Stack Pointer Select". * VBAR_EL1: 0x0 holds the base address of the vector table + -This reset value is defined UNKNOWN by <> D10.2.116 "VBAR_EL1, Vector Base Address Register (EL1)", so we must set it to something ourselves to have greater portability. +This reset value is defined UNKNOWN by xref:armarm8[] D10.2.116 "VBAR_EL1, Vector Base Address Register (EL1)", so we must set it to something ourselves to have greater portability. Bibliography: @@ -14865,7 +14866,7 @@ Exiting @ tick 36500 because m5_exit instruction encountered since gem5 is able to detect when nothing will ever happen, and exits. -When GDB step debugging, switch between cores with the usual `thread` commands, see also: <>. +When GDB step debugging, switch between cores with the usual `thread` commands, see also: xref:gdb-step-debug-multicore-userland[]. Bibliography: https://stackoverflow.com/questions/980999/what-does-multicore-assembly-language-look-like/33651438#33651438 @@ -14934,7 +14935,7 @@ To wake up CPU 1 on QEMU, we must use the Power State Coordination Interface (PS This interface uses HVC calls, and the calling convention is documented at "SMC CALLING CONVENTION" https://developer.arm.com/docs/den0028/latest[]. -If we boot the Linux kernel on QEMU and <>, we observe that it contains the address of the PSCI CPU_ON call: +If we boot the Linux kernel on QEMU and xref:get-device-tree-from-a-running-kernel[dump the auto-generated device tree], we observe that it contains the address of the PSCI CPU_ON call: .... psci { @@ -14961,7 +14962,7 @@ TODO: create and study a minimal examples in gem5 where the DMB instruction lead TODO get working. Attempt at: link:baremetal/arch/aarch64/timer.c[] -The timer is documented at: <> Chapter D10 "The Generic Timer in AArch64 state" +The timer is documented at: xref:armarm8-db[] Chapter D10 "The Generic Timer in AArch64 state" The key registers to keep in mind are: @@ -14972,7 +14973,7 @@ The key registers to keep in mind are: ==== ARM baremetal bibliography -First, also consider the userland bibliography: <>. +First, also consider the userland bibliography: xref:arm-assembly-bibliography[]. The most useful ARM baremetal example sets we've seen so far are: @@ -15103,7 +15104,7 @@ msr cpacr_el1, x1 isb .... -`cpacr_el1` is documented at <> D10.2.29 "CPACR_EL1, Architectural Feature Access Control Register". +`cpacr_el1` is documented at xref:armarm8[] D10.2.29 "CPACR_EL1, Architectural Feature Access Control Register". Here we touch the FPEN bits to 3, which enable floating point operations: @@ -15167,22 +15168,22 @@ ISB but it entered an exception loop at `MSR CPTR_EL3, XZR`. -We then found out that QEMU <>, and so we kept just the EL1 part, and it worked. Related: +We then found out that QEMU xref:arm-exception-levels[>, except that you have to add the `--mode baremetal` option, for example: +Baremetal tests work exactly like xref:user-mode-tests[], except that you have to add the `--mode baremetal` option, for example: .... ./test-executables --mode baremetal --arch aarch64 .... -In baremetal, we detect if tests failed by parsing logs for the <>. +In baremetal, we detect if tests failed by parsing logs for the xref:magic-failure-string[]. -See: <> for more useful testing tips. +See: xref:test-this-repo[] for more useful testing tips. == Android @@ -15249,13 +15250,13 @@ emulator: argv[41] = "-device" emulator: argv[42] = "virtio-blk-pci,drive=vendor,iothread=disk-iothread,modern-pio-notify" .... -The root directory is the <> given on the QEMU CLI, which `/proc/mounts` reports at: +The root directory is the xref:initrd[] given on the QEMU CLI, which `/proc/mounts` reports at: .... rootfs on / type rootfs (ro,seclabel,size=886392k,nr_inodes=221598) .... -This contains the <>, which through `.rc` must be mounting mounts the drives int o the right places TODO find exact point. +This contains the xref:android-init[], which through `.rc` must be mounting mounts the drives int o the right places TODO find exact point. The drive order is: @@ -15343,7 +15344,7 @@ When I install an app like F-Droid, it goes under `/data` according to: find / -iname '*fdroid*' .... -and it <>. +and it xref:disk-persistency[persists across boots]. `/data` is behind a RW LVM device: @@ -15407,7 +15408,7 @@ Tested on: `8.1.0_r60`. === Android init -For Linux in general, see: <>. +For Linux in general, see: xref:init[]. The `/init` executable interprets the `/init.rc` files, which is in a custom Android init system language: https://android.googlesource.com/platform/system/core/+/ee0e63f71d90537bb0570e77aa8a699cc222cfaf/init/README.md @@ -15436,7 +15437,7 @@ ranchu which is the codename for the QEMU virtual platform we are running on: https://www.oreilly.com/library/view/android-system-programming/9781787125360/9736a97c-cd09-40c3-b14d-955717648302.xhtml -TODO: is it possible to add a custom `.rc` file without modifying the initrd that <>? https://stackoverflow.com/questions/9768103/make-persistent-changes-to-init-rc +TODO: is it possible to add a custom `.rc` file without modifying the initrd that xref:android-image-structure[gets mounted on root]? https://stackoverflow.com/questions/9768103/make-persistent-changes-to-init-rc Tested on: `8.1.0_r60`. @@ -15446,11 +15447,11 @@ TODO: didn't fully port during refactor after 3b0a343647bed577586989fb702b760bd2 In this section document how benchmark builds and runs of this repo, and how to investigate what the bottleneck is. -Ideally, we should setup an automated build server that benchmarks those things continuously for us, but our <> attempt failed. +Ideally, we should setup an automated build server that benchmarks those things continuously for us, but our xref:travis[] attempt failed. So currently, we are running benchmarks manually when it seems reasonable and uploading them to: https://github.com/cirosantilli/linux-kernel-module-cheat-regression -All benchmarks were run on the <> machine, unless stated otherwise. +All benchmarks were run on the xref:p51[] machine, unless stated otherwise. Run all benchmarks and upload the results: @@ -15596,7 +15597,7 @@ Kernel panic - not syncing: Attempted to kill the idle task! ==== Benchmark builds -The build times are calculated after doing `./configure` and https://buildroot.org/downloads/manual/manual.html#_offline_builds[`make source`], which downloads the sources, and basically benchmarks the <>. +The build times are calculated after doing `./configure` and https://buildroot.org/downloads/manual/manual.html#_offline_builds[`make source`], which downloads the sources, and basically benchmarks the xref:benchmark-internets[Internet]. Sample build time at 2c12b21b304178a81c9912817b782ead0286d282: 28 minutes, 15 with full ccache hits. Breakdown: 19% GCC, 13% Linux kernel, 7% uclibc, 6% host-python, 5% host-qemu, 5% host-gdb, 2% host-binutils @@ -15696,7 +15697,7 @@ then rebuild with: and then copy the link command to a separate Bash file. Then you can time and modify it easily. -Some approximate reference values on <>: +Some approximate reference values on xref:p51[]: * `opt` ** unmodified: 10 seconds @@ -15750,7 +15751,7 @@ gem5: The host requirements depend a lot on which examples you want to run. -Some setups of this repository are very portable, notably setups under <>, e.g. <>, and will likely work on any host system with minimal modification. +Some setups of this repository are very portable, notably setups under xref:userland-setup[], e.g. xref:c[], and will likely work on any host system with minimal modification. The least portable setups are those that require Buildroot and crosstool-NG. @@ -15774,7 +15775,7 @@ After installing the missing packages for your distro, do the build with: which does everything as normal, except that it skips any `apt` commands. -If something does not work however, <> should just work on any Linux distro. +If something does not work however, xref:docker[] should just work on any Linux distro. Native Windows is unlikely feasible for Buildroot setups because Buildroot is a huge set of GNU Make scripts + host tools, just do everything from inside an Ubuntu in VirtualBox instance in that case. @@ -15807,9 +15808,9 @@ If you just want to run a command after boot ends without thinking much about it ./run --eval-after 'echo hello' .... -This option passes the command to our init scripts through <>, and uses a few clever tricks along the way to make it just work. +This option passes the command to our init scripts through xref:kernel-command-line-parameters[], and uses a few clever tricks along the way to make it just work. -See <> for the gory details. +See xref:init[] for the gory details. === Default command line arguments @@ -15874,8 +15875,8 @@ xdg-open out/README.html When running link:build-doc[], we do the following checks: * `<<>>` inner links are not broken -* `+link:somefile[]+` links point to paths that exist via <>. Upstream wontfix at: https://github.com/asciidoctor/asciidoctor/issues/3210 -* all links in non-README files to README IDs exist via `git grep` + <> +* `+link:somefile[]+` links point to paths that exist via xref:asciidoctor-extract-link-targets[]. Upstream wontfix at: https://github.com/asciidoctor/asciidoctor/issues/3210 +* all links in non-README files to README IDs exist via `git grep` + xref:asciidoctor-extract-header-ids[] The scripts prints what you have to fix and exits with an error status if there are any errors. @@ -15938,7 +15939,7 @@ The Asciidoctor extension scripts: hack the README `link:` targets to make them work from: * inside the `out/` directory with `../` -* <>, with explicit GitHub blob URLs +* xref:github-pages[], with explicit GitHub blob URLs ==== GitHub pages @@ -16014,7 +16015,7 @@ ls "$(./getvar buildroot_build_build_dir)" === ccache -https://en.wikipedia.org/wiki/Ccache[ccache] <> save you a lot of re-build when you decide to <> or create a new <>. +https://en.wikipedia.org/wiki/Ccache[ccache] xref:benchmark-builds[might] save you a lot of re-build when you decide to xref:clean-the-build[] or create a new xref:build-variants[build variant]. We have ccache enabled for everything we build by default. @@ -16067,7 +16068,7 @@ When doing long simulations sweeping across multiple system parameters, it becom This is specially true for gem5, which runs much slower than QEMU, and cannot use multiple host cores to speed up the simulation: https://github.com/cirosantilli-work/gem5-issues/issues/15[], so the only way to parallelize is to run multiple instances in parallel. -This also has a good synergy with <>. +This also has a good synergy with xref:build-variants[]. First shell: @@ -16097,7 +16098,7 @@ Each run gets a separate output directory. For example: ./run --arch aarch64 --emulator gem5 --run-id 1 &>/dev/null & .... -produces two separate <>: +produces two separate xref:m5out-directory[`m5out` directories]: .... echo "$(./getvar --arch aarch64 --emulator gem5 --run-id 0 m5out_dir)" @@ -16121,14 +16122,14 @@ To have more semantic output directories names for later inspection, you can use `--port-offset` defaults to the run ID when that is a number. -Like <>, you will need to pass the `-n` option to anything that needs to know runtime information, e.g. <>: +Like xref:cpu-architecture[], you will need to pass the `-n` option to anything that needs to know runtime information, e.g. xref:gdb[]: .... ./run --run-id 1 ./run-gdb --run-id 1 .... -To run multiple gem5 checkouts, see: <>. +To run multiple gem5 checkouts, see: xref:gem5-worktree[]. Implementation note: we create multiple namespaces for two things: @@ -16141,7 +16142,7 @@ If a port is not free, it just crashes. We assign a contiguous port range for each run ID. ** gem5 automatically increments ports until it finds a free one. + -gem5 60600f09c25255b3c8f72da7fb49100e2682093a does not seem to expose a way to set the terminal and VNC ports from `fs.py`, so we just let gem5 assign the ports itself, and use `-n` only to match what it assigned. Those ports both appear on <>. +gem5 60600f09c25255b3c8f72da7fb49100e2682093a does not seem to expose a way to set the terminal and VNC ports from `fs.py`, so we just let gem5 assign the ports itself, and use `-n` only to match what it assigned. Those ports both appear on xref:config-ini[]. + The GDB port can be assigned on `gem5.opt --remote-gdb-port`, but it does not appear on `config.ini`. @@ -16176,17 +16177,17 @@ git -C "$(./getvar linux_source_dir)" checkout - The `git fetch --unshallow` is needed the first time because `./build --download-dependencies` only does a shallow clone of the Linux kernel to save space and time, see also: https://stackoverflow.com/questions/6802145/how-to-convert-a-git-shallow-clone-to-a-full-clone -The `--linux-build-id` option should be passed to all scripts that support it, much like `--arch` for the <>, e.g. to step debug: +The `--linux-build-id` option should be passed to all scripts that support it, much like `--arch` for the xref:cpu-architecture[], e.g. to step debug: ..... ./run-gdb --linux-build-id v4.16 ..... -To run both kernels simultaneously, one on each QEMU instance, see: <>. +To run both kernels simultaneously, one on each QEMU instance, see: xref:simultaneous-runs[]. ==== QEMU build variants -Analogous to the <> but with the `--qemu-build-id` option instead: +Analogous to the xref:linux-kernel-build-variants[] but with the `--qemu-build-id` option instead: .... ./build-qemu @@ -16199,7 +16200,7 @@ git -C "$(./getvar qemu_source_dir)" checkout - ==== gem5 build variants -Analogous to the <> but with the `--gem5-build-id` option instead: +Analogous to the xref:linux-kernel-build-variants[] but with the `--gem5-build-id` option instead: .... # Build master. @@ -16222,11 +16223,11 @@ git -C "$(./getvar gem5_source_dir)" checkout some-branch Don't forget however that gem5 has Python scripts in its source code tree, and that those must match the source code of a given build. -Therefore, you can't forget to checkout to the sources to that of the corresponding build before running, unless you explicitly tell gem5 to use a non-default source tree with <>. This becomes inevitable when you want to launch multiple simultaneous runs at different checkouts. +Therefore, you can't forget to checkout to the sources to that of the corresponding build before running, unless you explicitly tell gem5 to use a non-default source tree with xref:gem5-worktree[]. This becomes inevitable when you want to launch multiple simultaneous runs at different checkouts. ===== gem5 worktree -<> goes a long way, but if you want to seamlessly switch between two gem5 tress without checking out multiple times, then `--gem5-worktree` is for you. +xref:gem5-build-variants[`--gem5-build-id`] goes a long way, but if you want to seamlessly switch between two gem5 tress without checking out multiple times, then `--gem5-worktree` is for you. .... # Build gem5 at the revision in the gem5 submodule. @@ -16297,7 +16298,7 @@ With this setup, both your private gem5 source and build are safely kept outside ===== gem5 debug build -The `gem5.debug` executable has optimizations turned off unlike the default `gem5.opt`, and provides a much better <>: +The `gem5.debug` executable has optimizations turned off unlike the default `gem5.opt`, and provides a much better xref:debug-the-emulator[debug experience]: .... ./build-gem5 --arch aarch64 --gem5-build-type debug @@ -16321,7 +16322,7 @@ so you will likely only use this when it is unavoidable. Allows you to have multiple versions of the GCC toolchain or root filesystem. -Analogous to the <> but with the `--build-id` option instead: +Analogous to the xref:linux-kernel-build-variants[] but with the `--build-id` option instead: .... ./build-buildroot @@ -16399,9 +16400,9 @@ You can force a rebuild with: ./build-buildroot --config 'BR2_PACKAGE_SAMPLE_PACKAGE=y' -- sample_package-reconfigure .... -Buildroot packages are convenient, but in general, if a package if very important to you, but not really mergeable back to Buildroot, you might want to just use a custom build script for it, and point it to the Buildroot toolchain, and then use `BR2_ROOTFS_OVERLAY`, much like we do for <>. +Buildroot packages are convenient, but in general, if a package if very important to you, but not really mergeable back to Buildroot, you might want to just use a custom build script for it, and point it to the Buildroot toolchain, and then use `BR2_ROOTFS_OVERLAY`, much like we do for xref:userland-setup[]. -A custom build script can give you more flexibility: e.g. the package can be made work with other root filesystems more easily, have better <<9p>> support, and rebuild faster as it evades some Buildroot boilerplate. +A custom build script can give you more flexibility: e.g. the package can be made work with other root filesystems more easily, have better xref:9p[] support, and rebuild faster as it evades some Buildroot boilerplate. ===== kernel_modules buildroot package @@ -16429,7 +16430,7 @@ Then test one of the modules with: Source: link:buildroot_packages/kernel_modules/buildroot_hello.c[] -As you have just seen, this sets up everything so that <> can correctly find the module. +As you have just seen, this sets up everything so that xref:modprobe[] can correctly find the module. `./build-buildroot --build-linux` and `./run --buildroot-linux` are needed because the Buildroot kernel modules must use the Buildroot Linux kernel at build and run time. @@ -16466,7 +16467,7 @@ We use this directory for: * customized configuration files * userland module test scripts that don't need to be compiled. + -Contrast this with <> that need compilation. +Contrast this with xref:userland-content[C examples] that need compilation. This directory is copied into the target filesystem by: @@ -16477,9 +16478,9 @@ This directory is copied into the target filesystem by: Source: link:copy-overlay[] -Build Buildroot is required for the same reason as described at: <>. +Build Buildroot is required for the same reason as described at: xref:your-first-kernel-module-hack[]. -However, since the link:rootfs_overlay[] directory does not require compilation, unlike say <>, we also make it <<9p>> available to the guest directly even without `./copy-overlay` at: +However, since the link:rootfs_overlay[] directory does not require compilation, unlike say xref:your-first-kernel-module-hack[kernel modules], we also make it xref:9p[] available to the guest directly even without `./copy-overlay` at: .... ls /mnt/9p/rootfs_overlay @@ -16494,7 +16495,7 @@ The files: * link:lkmc.c[] * link:lkmc.h[] -contain common C function helpers that can be used both in userland and baremetal. Oh, the infinite <>. +contain common C function helpers that can be used both in userland and baremetal. Oh, the infinite xref:about-the-baremetal-setup[joys of Newlib]. Those files also contain arch specific helpers under ifdefs like: @@ -16516,12 +16517,12 @@ Source: link:userland/linux/rand_check.c[] This can be used to check the determinism of: -* <> -* <> +* xref:norandmaps[] +* xref:qemu-record-and-replay[] ==== lkmc_home -`lkmc_home` refers to the target base directory in which we put all our custom built stuff, such as <> and <>. +`lkmc_home` refers to the target base directory in which we put all our custom built stuff, such as xref:userland-setup[userland executables] and xref:your-first-kernel-module-hack[kernel modules]. The current value can be found with: @@ -16533,8 +16534,8 @@ In the past, we used to dump everything into the root filesystem, but as the use To save you from typing that path every time, we have made our most common commands `cd` into that directory by default for you, e.g.: -* interactive shells `cd` there through <> -* `--eval` and `--eval-after` through <> and <> +* interactive shells `cd` there through xref:busybox-shell-initrc-files[] +* `--eval` and `--eval-after` through xref:replace-init[] and xref:init-busybox[] Whenever a relative path is used inside a guest sample command, e.g. `insmod hello.ko` or `./hello.out`, it means that the path lives in `lkmc_home` unless stated otherwise. @@ -16560,11 +16561,11 @@ Sources: The link:test[] script runs several different types of tests, which can also be run separately as explained at: * link:test-boot[] -* <> -* <> -* <> -* <> -* <> +* xref:test-userland-in-full-system[] +* xref:user-mode-tests[] +* xref:baremetal-tests[] +* xref:gdb-tests[] +* xref:gem5-unit-tests[] link:test[] does not all possible tests, because there are too many possible variations and that would take forever. The rationale is the same as for `./build all` and is explained in `./build --help`. @@ -16606,11 +16607,11 @@ You can make them exit immediately with the `--no-quit-on-fail` option, e.g.: ===== Test userland in full system -TODO: we really need a mechanism to automatically generate the test list automatically e.g. based on <>, currently there are many tests missing, and we have to add everything manually which is very annoying. +TODO: we really need a mechanism to automatically generate the test list automatically e.g. based on xref:path-properties[], currently there are many tests missing, and we have to add everything manually which is very annoying. We could just generate it on the fly on the host, and forward it to guest through CLI arguments. -Run all userland tests from inside full system simulation (i.e. not <>): +Run all userland tests from inside full system simulation (i.e. not xref:user-mode-simulation[]): .... ./test-userland-full-system @@ -16620,9 +16621,9 @@ This includes, in particular, userland programs that test the kernel modules, wh Basically just boots and runs: link:rootfs_overlay/lkmc/test_all.sh[] -Failure is detected by looking for the <> +Failure is detected by looking for the xref:magic-failure-string[] -Most userland programs that don't rely on kernel modules can also be tested in user mode simulation as explained at: <>. +Most userland programs that don't rely on kernel modules can also be tested in user mode simulation as explained at: xref:user-mode-tests[]. ===== GDB tests @@ -16661,7 +16662,7 @@ and possibly repeat the GDB steps manually with the usual: ./run-gdb --arch arm --baremetal baremetal/c/add.c --no-continue --verbose .... -To debug GDB problems on gem5, you might want to enable the following <> options: +To debug GDB problems on gem5, you might want to enable the following xref:gem5-tracing[tracing] options: .... ./run \ @@ -16680,8 +16681,8 @@ We do not know of any way to set the emulator exit status in QEMU arm full syste For other arch / emulator combinations, we know how to do it: * aarch64: aarch64 semihosting supports exit status -* gem5: <> works on all archs -* user mode: QEMU forwards exit status, gem5 we do some log parsing: <> +* gem5: xref:m5-fail[] works on all archs +* user mode: QEMU forwards exit status, gem5 we do some log parsing: xref:gem5-syscall-emulation-exit-status[] Since we can't do it for QEMU arm, the only reliable solution is to just parse the guest serial output for a magic failure string to check if tests failed. @@ -16706,14 +16707,14 @@ should output: This magic output string is notably generated by: -* link:rootfs_overlay/lkmc/test_fail.sh[], which is used by <> +* link:rootfs_overlay/lkmc/test_fail.sh[], which is used by xref:test-userland-in-full-system[] * the `exit()` baremetal function when `status != 1`. + Unfortunately the only way we found to set this up was with `on_exit`: https://github.com/cirosantilli/linux-kernel-module-cheat/issues/59[]. + Trying to patch `_exit` directly fails since at that point some de-initialization has already happened which prevents the print. + -So setup this `on_exit` automatically from all our <>, so it just works automatically for the examples that use the bootloaders: https://stackoverflow.com/questions/44097610/pass-parameter-to-atexit/49659697#49659697 +So setup this `on_exit` automatically from all our xref:baremetal-bootloaders[], so it just works automatically for the examples that use the bootloaders: https://stackoverflow.com/questions/44097610/pass-parameter-to-atexit/49659697#49659697 + The following examples end up testing that our setup is working: + @@ -16772,7 +16773,7 @@ You should also test that the Internet works: When updating the Linux kernel, QEMU and gem5, things sometimes break. -However, for many types of crashes, it is trivial to bisect down to the offending commit, in particular because we can make QEMU and gem5 exit with status 1 on kernel panic: <>. +However, for many types of crashes, it is trivial to bisect down to the offending commit, in particular because we can make QEMU and gem5 exit with status 1 on kernel panic: xref:exit-emulator-on-panic[]. For example, when updating from QEMU `v2.12.0` to `v3.0.0-rc3`, the Linux kernel boot started to panic for `arm`. @@ -16813,7 +16814,7 @@ Other bisection helpers include: [[path-properties]] === path_properties -In order to build and run each userland and <> example properly, we need per-file metadata such as compiler flags and required number of cores. +In order to build and run each userland and xref:baremetal-setup[baremetal] example properly, we need per-file metadata such as compiler flags and required number of cores. This data is stored is stored in link:path_properties.py[] at `path_properties_tuples`. @@ -16874,20 +16875,20 @@ git commit -m "linux: update to ${next_mainline_revision}" ==== Release procedure -Ensure that the <> are passing on a clean build: +Ensure that the xref:automated-tests[] are passing on a clean build: .... mv out out.bak ./build-test --size 3 && ./test --size 3 .... -The `./build-test` command builds a superset of what will be downloaded which also tests other things we would like to be working on the release. For the minimal build to generate the files to be uploaded, see: <> +The `./build-test` command builds a superset of what will be downloaded which also tests other things we would like to be working on the release. For the minimal build to generate the files to be uploaded, see: xref:release-zip[] -The clean build is necessary as it generates clean images since <> +The clean build is necessary as it generates clean images since xref:remove-buildroot-packages[it is not possible to remove Buildroot packages] -Run all tests in <> just QEMU x86_64 and QEMU aarch64. +Run all tests in xref:non-automated-tests[] just QEMU x86_64 and QEMU aarch64. -TODO: not working currently, so skipped: Ensure that the <> look fine: +TODO: not working currently, so skipped: Ensure that the xref:benchmark-this-repo[benchmarks] look fine: .... ./bench-all -A @@ -16911,7 +16912,7 @@ git clone https://github.com/cirosantilli/linux-kernel-module-cheat linux-kernel cd linux-kernel-module-cheat-release .... -Test <>. +Test xref:prebuilt[]. Clean up, and re-start from scratch: @@ -16922,7 +16923,7 @@ git clone https://github.com/cirosantilli/linux-kernel-module-cheat linux-kernel cd linux-kernel-module-cheat-release .... -Go through all the other <> sections in order. +Go through all the other xref:getting-started[] sections in order. Once everything looks fine, publish the release with: @@ -16937,7 +16938,7 @@ git push --follow-tags ==== release-zip -Create a zip containing all files required for <>: +Create a zip containing all files required for xref:prebuilt[]: .... ./build --all-archs release && ./release-zip --all-archs @@ -16957,7 +16958,7 @@ which you can then upload somewhere. After: -* running <> +* running xref:release-zip[] * creating and pushing a tag to GitHub you can upload the release to GitHub automatically with: @@ -16992,9 +16993,9 @@ System simulators are cool compared to real hardware because they are: The current components we focus the most on are: -* <> and Linux kernel modules -* full systems emulators, currently <> and <> -* <>. We use and therefore document, a large part of its feature set. +* xref:linux-kernel[] and Linux kernel modules +* full systems emulators, currently xref:qemu-buildroot-setup[qemu] and xref:gem5-buildroot-setup[gem5] +* xref:buildroot[]. We use and therefore document, a large part of its feature set. The following components are not covered, but they would also benefit from this setup, and it shouldn't be hard to add them: @@ -17006,7 +17007,7 @@ The design goals are to provide setups that are: * highly automated: "just works" * thoroughly documented: you know what "just works" means * can be fully built from source: to give visibility and allow modifications -* can also use <> as much as possible: in case you are lazy or unable to build from source +* can also use xref:prebuilt[ prebuilt binaries] as much as possible: in case you are lazy or unable to build from source We aim to make a documentation that contains a very high runnable example / theory bullshit ratio. @@ -17014,7 +17015,7 @@ Having at least one example per section is ideal, and it should be the very firs ==== Setup trade-offs -The trade-offs between the different <> are basically a balance between: +The trade-offs between the different xref:getting-started[setups] are basically a balance between: * speed ans size: how long and how much disk space do the build and run take? * visibility: can you GDB step debug everything and read source code? @@ -17022,13 +17023,13 @@ The trade-offs between the different <> are basically a * portability: does it work on a Windows host? Could it ever? * accuracy: how accurate does the simulation represent real hardware? * compatibility: how likely is is that all the components will work well together: emulator, compiler, kernel, standard library, ... -* guest software availability: how wide is your choice of easily installed guest software packages? See also: <> +* guest software availability: how wide is your choice of easily installed guest software packages? See also: xref:linux-distro-choice[] ==== Resource tradeoff guidelines Choosing which features go into our default builds means making tradeoffs, here are our guidelines: -* keep the root filesystem as tiny as possible to make <> small: only add BusyBox to have a small interactive system. +* keep the root filesystem as tiny as possible to make xref:prebuilt[] small: only add BusyBox to have a small interactive system. + It is easy to add new packages once you have the toolchain, and if you don't there are infinitely many packages to cover and we can't cover them all. * enable every feature possible on the toolchain (GCC, Binutils), because changes imply Buildroot rebuilds @@ -17036,18 +17037,18 @@ It is easy to add new packages once you have the toolchain, and if you don't the + -- ** easier to understand -** run faster, which is specially for <> which is slow +** run faster, which is specially for xref:gem5[] which is slow -- + Runtime basically just comes down to how we configure the Linux kernel, since in the root filesystem all that matters is `init=`, and that is easy to control. + One possibility we could play with is to build loadable modules instead of built-in modules to reduce runtime, but make it easier to get started with the modules. -In order to learn how to measure some of those aspects, see: <> +In order to learn how to measure some of those aspects, see: xref:benchmark-this-repo[] ==== Linux distro choice -We haven't found the ultimate distro yet, here is a summary table of trade-offs that we care about: <> +We haven't found the ultimate distro yet, here is a summary table of trade-offs that we care about: xref:table-lkmc-linux-distro-comparison[] [[table-lkmc-linux-distro-comparison]] .Comparison of Linux distros for usage in this repository @@ -17090,7 +17091,7 @@ We haven't found the ultimate distro yet, here is a summary table of trade-offs * (3): https://askubuntu.com/questions/120630/how-many-packages-are-in-the-main-repository * (4): `ls main community non-free | wc` * (5): yes, but on a separate Git tree... https://git.yoctoproject.org/cgit/cgit.cgi/yocto-docs/ -* (6): yes, but the initial Poky build / download still took 5 hours on <<38mbps-internet>>, and QEMU failed to boot at the end... https://bugzilla.yoctoproject.org/show_bug.cgi?id=12953 +* (6): yes, but the initial Poky build / download still took 5 hours on xref:38mbps-internet[], and QEMU failed to boot at the end... https://bugzilla.yoctoproject.org/show_bug.cgi?id=12953 * (7): `ls recipes-* | wc` * (8): Poky reference system: http://git.yoctoproject.org/cgit/cgit.cgi/poky