From 650a0bfd7216bc77599a9205bcd49c122be01327 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ciro=20Santilli=20=E5=85=AD=E5=9B=9B=E4=BA=8B=E4=BB=B6=20?= =?UTF-8?q?=E6=B3=95=E8=BD=AE=E5=8A=9F?= Date: Fri, 8 Nov 2019 00:00:00 +0000 Subject: [PATCH] a06672a20d160c1748223665501ca45295008423 --- index.html | 317 ++++++++++++++++++++++++++++------------------------- 1 file changed, 168 insertions(+), 149 deletions(-) diff --git a/index.html b/index.html index 8e79ff0..d7bb433 100644 --- a/index.html +++ b/index.html @@ -1771,122 +1771,123 @@ body.book #toc,body.book #preamble,body.book h1.sect0,body.book .sect1>h2{page-b
  • 30. Xephyr
    • -
  • 31. About this repo +
  • 31. Compilers
    • +
  • 32. About this repo
    • @@ -1903,14 +1904,14 @@ body.book #toc,body.book #preamble,body.book h1.sect0,body.book .sect1>h2{page-b

    If you don’t know which one to go for, start with QEMU Buildroot setup getting started.

    -

    Design goals of this project are documented at: Section 31.18.1, “Design goals”.

    +

    Design goals of this project are documented at: Section 32.18.1, “Design goals”.

    1.1. QEMU Buildroot setup

    1.1.1. QEMU Buildroot setup getting started

    -

    This setup has been mostly tested on Ubuntu. For other host operating systems see: Section 31.1, “Supported hosts”. For greater stability, consider using the latest release instead of master: https://github.com/cirosantilli/linux-kernel-module-cheat/releases

    +

    This setup has been mostly tested on Ubuntu. For other host operating systems see: Section 32.1, “Supported hosts”. For greater stability, consider using the latest release instead of master: https://github.com/cirosantilli/linux-kernel-module-cheat/releases

    Reserve 12Gb of disk and run:

    @@ -1927,7 +1928,7 @@ cd linux-kernel-module-cheat

    You don’t need to clone recursively even though we have .git submodules: download-dependencies fetches just the submodules that you need for this build to save time.

    -

    If something goes wrong, see: Section 31.2, “Common build issues” and use our issue tracker: https://github.com/cirosantilli/linux-kernel-module-cheat/issues

    +

    If something goes wrong, see: Section 32.2, “Common build issues” and use our issue tracker: https://github.com/cirosantilli/linux-kernel-module-cheat/issues

    The initial build will take a while (30 minutes to 2 hours) to clone and build, see Benchmark builds for more details.

    @@ -2010,7 +2011,7 @@ hello2 cleanup
    -

    To avoid typing --arch aarch64 many times, you can set the default arch as explained at: Section 31.4, “Default command line arguments”

    +

    To avoid typing --arch aarch64 many times, you can set the default arch as explained at: Section 32.4, “Default command line arguments”

    I now urge you to read the following sections which contain widely applicable information:

    @@ -2314,7 +2315,7 @@ hello /root/.profile

    If you really want to develop semiconductors, your only choice is to join an university or a semiconductor company that has the EDA licenses.

    -

    See also: Section 31.19.2, “Should you waste your life with systems programming?”.

    +

    See also: Section 32.19.2, “Should you waste your life with systems programming?”.

    While hacking QEMU, you will likely want to GDB step its source. That is trivial since QEMU is just another userland program like any other, but our setup has a shortcut to make it even more convenient, see: Section 18.7, “Debug the emulator”.

    @@ -2753,7 +2754,7 @@ j = 0

    This repository has been tested inside clean Docker containers.

    -

    This is a good option if you are on a Linux host, but the native setup failed due to your weird host distribution, and you have better things to do with your life than to debug it. See also: Section 31.1, “Supported hosts”.

    +

    This is a good option if you are on a Linux host, but the native setup failed due to your weird host distribution, and you have better things to do with your life than to debug it. See also: Section 32.1, “Supported hosts”.

    For example, to do a QEMU Buildroot setup inside Docker, run:

    @@ -3831,7 +3832,7 @@ xdg-open README.html
    -

    More information about our documentation internals can be found at: Section 31.5, “Documentation”

    +

    More information about our documentation internals can be found at: Section 32.5, “Documentation”

    @@ -7218,7 +7219,7 @@ qw er

    The gem5 tests require building statically with build id static, see also: Section 10.6, “gem5 syscall emulation mode”. TODO automate this better.

    -

    See: Section 31.13, “Test this repo” for more useful testing tips.

    +

    See: Section 32.13, “Test this repo” for more useful testing tips.

    @@ -7899,7 +7900,7 @@ hello
    -

    31. About this repo

    +

    31. Compilers

    +
    +
    +

    Argh, compilers are boring, let’s learn a bit about them.

    +
    +
    + +
    +
    +
    +
    +

    32. About this repo

    -

    31.1. Supported hosts

    +

    32.1. Supported hosts

    The host requirements depend a lot on which examples you want to run.

    @@ -30745,9 +30764,9 @@ west build -b qemu_aarch64 samples/hello_world
    -

    31.2. Common build issues

    +

    32.2. Common build issues

    -

    31.2.1. You must put some 'source' URIs in your sources.list

    +

    32.2.1. You must put some 'source' URIs in your sources.list

    If ./build --download-dependencies fails with:

    @@ -30761,7 +30780,7 @@ west build -b qemu_aarch64 samples/hello_world
    -

    31.2.2. Build from downloaded source zip files

    +

    32.2.2. Build from downloaded source zip files

    It does not work if you just download the .zip with the sources for this repository from GitHub because we use Git submodules, you must clone this repo.

    @@ -30771,7 +30790,7 @@ west build -b qemu_aarch64 samples/hello_world
    -

    31.3. Run command after boot

    +

    32.3. Run command after boot

    If you just want to run a command after boot ends without thinking much about it, just use the --eval-after option, e.g.:

    @@ -30788,7 +30807,7 @@ west build -b qemu_aarch64 samples/hello_world
    -

    31.4. Default command line arguments

    +

    32.4. Default command line arguments

    It gets annoying to retype --arch aarch64 for every single command, or to remember --config setups.

    @@ -30833,12 +30852,12 @@ west build -b qemu_aarch64 samples/hello_world
    -

    31.5. Documentation

    +

    32.5. Documentation

    To learn how to build the documentation see: Section 1.8, “Build the documentation”.

    -

    31.5.1. Documentation verification

    +

    32.5.1. Documentation verification

    When running build-doc, we do the following checks:

    @@ -30859,7 +30878,7 @@ west build -b qemu_aarch64 samples/hello_world

    The scripts prints what you have to fix and exits with an error status if there are any errors.

    - +

    Documentation for asciidoctor/extract-link-targets

    @@ -30882,7 +30901,7 @@ west build -b qemu_aarch64 samples/hello_world
    -
    31.5.1.2. asciidoctor/extract-header-ids
    +
    32.5.1.2. asciidoctor/extract-header-ids

    Documentation for asciidoctor/extract-header-ids

    @@ -30927,7 +30946,7 @@ explicitly-given
    - +

    The Asciidoctor extension scripts:

    @@ -30955,7 +30974,7 @@ explicitly-given
    -

    31.6.1. GitHub pages

    +

    32.6.1. GitHub pages

    As mentioned before the TOC, we have to push this README to GitHub pages due to: https://github.com/isaacs/github/issues/1610

    @@ -31005,7 +31024,7 @@ explicitly-given
    -

    31.7. Clean the build

    +

    32.7. Clean the build

    You did something crazy, and nothing seems to work anymore?

    @@ -31069,7 +31088,7 @@ ls "$(./getvar buildroot_build_dir)"
    -

    31.8. ccache

    +

    32.8. ccache

    ccache might save you a lot of re-build when you decide to Clean the build or create a new build variant.

    @@ -31138,7 +31157,7 @@ export CCACHE_MAXSIZE="20G"
    -

    31.9. Rebuild Buildroot while running

    +

    32.9. Rebuild Buildroot while running

    It is not possible to rebuild the root filesystem while running QEMU because QEMU holds the file qcow2 file:

    @@ -31149,7 +31168,7 @@ export CCACHE_MAXSIZE="20G"
    -

    31.10. Simultaneous runs

    +

    32.10. Simultaneous runs

    When doing long simulations sweeping across multiple system parameters, it becomes fundamental to do multiple simulations in parallel.

    @@ -31245,7 +31264,7 @@ less "$(./getvar --arch aarch64 --emulator gem5 --run-id 1 termout_file)"
    -

    To run multiple gem5 checkouts, see: Section 31.11.3.1, “gem5 worktree”.

    +

    To run multiple gem5 checkouts, see: Section 32.11.3.1, “gem5 worktree”.

    Implementation note: we create multiple namespaces for two things:

    @@ -31284,7 +31303,7 @@ less "$(./getvar --arch aarch64 --emulator gem5 --run-id 1 termout_file)"
    -

    31.11. Build variants

    +

    32.11. Build variants

    It often happens that you are comparing two versions of the build, a good and a bad one, and trying to figure out why the bad one is bad.

    @@ -31292,7 +31311,7 @@ less "$(./getvar --arch aarch64 --emulator gem5 --run-id 1 termout_file)"

    Our build variants system allows you to keep multiple built versions of all major components, so that you can easily switching between running one or the other.

    -

    31.11.1. Linux kernel build variants

    +

    32.11.1. Linux kernel build variants

    If you want to keep two builds around, one for the latest Linux version, and the other for Linux v4.16:

    @@ -31328,11 +31347,11 @@ git -C "$(./getvar linux_source_dir)" checkout -
    -

    To run both kernels simultaneously, one on each QEMU instance, see: Section 31.10, “Simultaneous runs”.

    +

    To run both kernels simultaneously, one on each QEMU instance, see: Section 32.10, “Simultaneous runs”.

    -

    31.11.2. QEMU build variants

    +

    32.11.2. QEMU build variants

    Analogous to the Linux kernel build variants but with the --qemu-build-id option instead:

    @@ -31348,7 +31367,7 @@ git -C "$(./getvar qemu_source_dir)" checkout -
    -

    31.11.3. gem5 build variants

    +

    32.11.3. gem5 build variants

    Analogous to the Linux kernel build variants but with the --gem5-build-id option instead:

    @@ -31379,7 +31398,7 @@ git -C "$(./getvar gem5_source_dir)" checkout some-branch

    Therefore, you can’t forget to checkout to the sources to that of the corresponding build before running, unless you explicitly tell gem5 to use a non-default source tree with gem5 worktree. This becomes inevitable when you want to launch multiple simultaneous runs at different checkouts.

    -
    31.11.3.1. gem5 worktree
    +
    32.11.3.1. gem5 worktree

    --gem5-build-id goes a long way, but if you want to seamlessly switch between two gem5 tress without checking out multiple times, then --gem5-worktree is for you.

    @@ -31432,7 +31451,7 @@ cd -
    -
    31.11.3.2. gem5 private source trees
    +
    32.11.3.2. gem5 private source trees

    Suppose that you are working on a private fork of gem5, but you want to use this repository to develop it as well.

    @@ -31476,7 +31495,7 @@ gem5_internal="$(pwd)/gem5-internal"
    -

    31.11.4. Buildroot build variants

    +

    32.11.4. Buildroot build variants

    Allows you to have multiple versions of the GCC toolchain or root filesystem.

    @@ -31496,9 +31515,9 @@ git -C "$(./getvar buildroot_source_dir)" checkout -
    -

    31.12. Directory structure

    +

    32.12. Directory structure

    -

    31.12.1. lkmc directory

    +

    32.12.1. lkmc directory

    lkmc/ contains sources and headers that are shared across kernel modules, userland and baremetal examples.

    @@ -31509,7 +31528,7 @@ git -C "$(./getvar buildroot_source_dir)" checkout -

    Another option would have been to name it as includes/lkmc, but that would make paths longer, and we might want to store source code in that directory as well in the future.

    -
    31.12.1.1. Userland objects vs header-only
    +
    32.12.1.1. Userland objects vs header-only

    When factoring out functionality across userland examples, there are two main options:

    @@ -31568,7 +31587,7 @@ git -C "$(./getvar buildroot_source_dir)" checkout -
    -

    31.12.2. buildroot_packages directory

    +

    32.12.2. buildroot_packages directory

    Source: buildroot_packages/.

    @@ -31617,7 +31636,7 @@ git -C "$(./getvar buildroot_source_dir)" checkout -

    A custom build script can give you more flexibility: e.g. the package can be made work with other root filesystems more easily, have better 9P support, and rebuild faster as it evades some Buildroot boilerplate.

    -
    31.12.2.1. kernel_modules buildroot package
    +
    32.12.2.1. kernel_modules buildroot package

    Source: buildroot_packages/kernel_modules/

    @@ -31664,9 +31683,9 @@ git -C "$(./getvar buildroot_source_dir)" checkout -
    -

    31.12.3. patches directory

    +

    32.12.3. patches directory

    -
    31.12.3.1. patches/global directory
    +
    32.12.3.1. patches/global directory

    Has the following structure:

    @@ -31683,7 +31702,7 @@ git -C "$(./getvar buildroot_source_dir)" checkout -
    -
    31.12.3.2. patches/manual directory
    +
    32.12.3.2. patches/manual directory

    Patches in this directory are never applied automatically: it is up to users to manually apply them before usage following the instructions in this documentation.

    @@ -31693,7 +31712,7 @@ git -C "$(./getvar buildroot_source_dir)" checkout -
    -

    31.12.4. rootfs_overlay

    +

    32.12.4. rootfs_overlay

    Source: rootfs_overlay.

    @@ -31740,7 +31759,7 @@ git -C "$(./getvar buildroot_source_dir)" checkout -

    This way you can just hack away the scripts and try them out immediately without any further operations.

    -
    31.12.4.1. out_rootfs_overlay_dir
    +
    32.12.4.1. out_rootfs_overlay_dir

    This path can be found with:

    @@ -31774,7 +31793,7 @@ git -C "$(./getvar buildroot_source_dir)" checkout -
    -

    31.12.5. lkmc.c

    +

    32.12.5. lkmc.c

    The files:

    @@ -31804,7 +31823,7 @@ git -C "$(./getvar buildroot_source_dir)" checkout -
    -

    31.12.6. rand_check.out

    +

    32.12.6. rand_check.out

    Print out several parameters that normally change randomly from boot to boot:

    @@ -31831,7 +31850,7 @@ git -C "$(./getvar buildroot_source_dir)" checkout -
    -

    31.12.7. lkmc_home

    +

    32.12.7. lkmc_home

    lkmc_home refers to the target base directory in which we put all our custom built stuff, such as userland executables and kernel modules.

    @@ -31865,9 +31884,9 @@ git -C "$(./getvar buildroot_source_dir)" checkout -
    -

    31.13. Test this repo

    +

    32.13. Test this repo

    -

    31.13.1. Automated tests

    +

    32.13.1. Automated tests

    Run almost all tests:

    @@ -31923,7 +31942,7 @@ echo $?

    test does not all possible tests, because there are too many possible variations and that would take forever. The rationale is the same as for ./build all and is explained in ./build --help.

    -
    31.13.1.1. Test arch and emulator selection
    +
    32.13.1.1. Test arch and emulator selection

    You can select multiple archs and emulators of interest, as for an other command, with:

    @@ -31956,7 +31975,7 @@ echo $?
    -
    31.13.1.2. Quit on fail
    +
    32.13.1.2. Quit on fail

    By default, continue running even after the first failure happens, and they show a summary at the end.

    @@ -31970,7 +31989,7 @@ echo $?
    -
    31.13.1.3. Test userland in full system
    +
    32.13.1.3. Test userland in full system

    TODO: we really need a mechanism to automatically generate the test list automatically e.g. based on path_properties, currently there are many tests missing, and we have to add everything manually which is very annoying.

    @@ -31999,7 +32018,7 @@ echo $?
    -
    31.13.1.4. GDB tests
    +
    32.13.1.4. GDB tests

    We have some pexpect automated tests for GDB for both userland and baremetal programs!

    @@ -32072,7 +32091,7 @@ echo $?
    -
    31.13.1.5. Magic failure string
    +
    32.13.1.5. Magic failure string

    We do not know of any way to set the emulator exit status in QEMU arm full system.

    @@ -32175,9 +32194,9 @@ echo $?
    -

    31.13.2. Non-automated tests

    +

    32.13.2. Non-automated tests

    -
    31.13.2.1. Test GDB Linux kernel
    +
    32.13.2.1. Test GDB Linux kernel

    For the Linux kernel, do the following manual tests for now.

    @@ -32215,7 +32234,7 @@ echo $?
    -
    31.13.2.2. Test the Internet
    +
    32.13.2.2. Test the Internet

    You should also test that the Internet works:

    @@ -32226,7 +32245,7 @@ echo $?
    -
    31.13.2.3. CLI script tests
    +
    32.13.2.3. CLI script tests

    build-userland and test-executables have a wide variety of target selection modes, and it was hard to keep them all working without some tests:

    @@ -32244,7 +32263,7 @@ echo $?
    -

    31.14. Bisection

    +

    32.14. Bisection

    When updating the Linux kernel, QEMU and gem5, things sometimes break.

    @@ -32300,7 +32319,7 @@ git submodule update
    -

    31.15. path_properties

    +

    32.15. path_properties

    In order to build and run each userland and baremetal example properly, we need per-file metadata such as compiler flags and required number of cores.

    @@ -32343,7 +32362,7 @@ git submodule update
    -

    31.16. Update a forked submodule

    +

    32.16. Update a forked submodule

    This is a template update procedure for submodules for which we have some patches on on top of mainline.

    @@ -32372,9 +32391,9 @@ git commit -m "linux: update to ${next_mainline_revision}"
    -

    31.17. Release

    +

    32.17. Release

    -

    31.17.1. Release procedure

    +

    32.17.1. Release procedure

    Ensure that the Automated tests are passing on a clean build:

    @@ -32385,7 +32404,7 @@ git commit -m "linux: update to ${next_mainline_revision}"
    -

    The ./build-test command builds a superset of what will be downloaded which also tests other things we would like to be working on the release. For the minimal build to generate the files to be uploaded, see: Section 31.17.2, “release-zip”

    +

    The ./build-test command builds a superset of what will be downloaded which also tests other things we would like to be working on the release. For the minimal build to generate the files to be uploaded, see: Section 32.17.2, “release-zip”

    The clean build is necessary as it generates clean images since it is not possible to remove Buildroot packages

    @@ -32455,7 +32474,7 @@ git push --follow-tags
    -

    31.17.2. release-zip

    +

    32.17.2. release-zip

    Create a zip containing all files required for Prebuilt setup:

    @@ -32480,7 +32499,7 @@ git push --follow-tags
    -

    31.17.3. release-upload

    +

    32.17.3. release-upload

    After:

    @@ -32528,9 +32547,9 @@ git push --follow-tags
    -

    31.18. Design rationale

    +

    32.18. Design rationale

    -

    31.18.1. Design goals

    +

    32.18.1. Design goals

    This project was created to help me understand, modify and test low level system components by using system simulators.

    @@ -32606,7 +32625,7 @@ git push --follow-tags
    -

    31.18.2. Setup trade-offs

    +

    32.18.2. Setup trade-offs

    The trade-offs between the different setups are basically a balance between:

    @@ -32631,13 +32650,13 @@ git push --follow-tags

    compatibility: how likely is is that all the components will work well together: emulator, compiler, kernel, standard library, …​

  • -

    guest software availability: how wide is your choice of easily installed guest software packages? See also: Section 31.18.4, “Linux distro choice”

    +

    guest software availability: how wide is your choice of easily installed guest software packages? See also: Section 32.18.4, “Linux distro choice”

    • -

    31.18.3. Resource tradeoff guidelines

    +

    32.18.3. Resource tradeoff guidelines

    Choosing which features go into our default builds means making tradeoffs, here are our guidelines:

    @@ -32682,7 +32701,7 @@ git push --follow-tags
    -

    31.18.4. Linux distro choice

    +

    32.18.4. Linux distro choice

    We haven’t found the ultimate distro yet, here is a summary table of trade-offs that we care about: Table 7, “Comparison of Linux distros for usage in this repository”.

    @@ -32785,9 +32804,9 @@ git push --follow-tags
    -

    31.19. Soft topics

    +

    32.19. Soft topics

    -

    31.19.1. Fairy tale

    +

    32.19.1. Fairy tale

    @@ -32824,7 +32843,7 @@ git push --follow-tags
    -

    31.19.2. Should you waste your life with systems programming?

    +

    32.19.2. Should you waste your life with systems programming?

    Being the hardcore person who fully understands an important complex system such as a computer, it does have a nice ring to it doesn’t it?

    @@ -32909,7 +32928,7 @@ git push --follow-tags
    -

    31.20. Bibliography

    +

    32.20. Bibliography

    Runnable stuff: