readme: setup github pages to overcome 512k limit

This commit is contained in:
Ciro Santilli 六四事件 法轮功
2019-07-05 00:00:00 +00:00
parent c98d93f7f7
commit de1457c932
9 changed files with 384 additions and 51 deletions

View File

@@ -13,6 +13,8 @@ The perfect emulation setup to study and develop the <<linux-kernel>> v5.1, kern
TL;DR: <<qemu-buildroot-setup-getting-started>>
The source code for this page is located at: https://github.com/cirosantilli/linux-kernel-module-cheat[]. Due to link:https://github.com/isaacs/github/issues/1610[a GitHub limitation], this README is too long and not fully rendered on github.com. Either use: https://www.cirosantilli.com/linux-kernel-module-cheat or <<build-the-documentation,build the docs yourself>>.
toc::[]
== Getting started
@@ -1371,8 +1373,8 @@ continue
See also:
* http://stackoverflow.com/questions/11408041/how-to-debug-the-linux-kernel-with-gdb-and-qemu/33203642#33203642
* http://stackoverflow.com/questions/4943857/linux-kernel-live-debugging-how-its-done-and-what-tools-are-used/42316607#42316607
* https://stackoverflow.com/questions/11408041/how-to-debug-the-linux-kernel-with-gdb-and-qemu/33203642#33203642
* https://stackoverflow.com/questions/4943857/linux-kernel-live-debugging-how-its-done-and-what-tools-are-used/42316607#42316607
==== GDB step debug kernel boot other archs
@@ -1545,7 +1547,7 @@ From inside tmux, you can create new terminals on a new window with `Ctrl-B C` s
=== GDB step debug kernel module
http://stackoverflow.com/questions/28607538/how-to-debug-linux-kernel-modules-with-qemu/44095831#44095831
https://stackoverflow.com/questions/28607538/how-to-debug-linux-kernel-modules-with-qemu/44095831#44095831
Loadable kernel modules are a bit trickier since the kernel can place them at different memory locations depending on load order.
@@ -4417,7 +4419,7 @@ TODO <<kmscube>> failed on `aarch64` with:
kmscube[706]: unhandled level 2 translation fault (11) at 0x00000000, esr 0x92000006, in libgbm.so.1.0.0[7fbf6a6000+e000]
....
Tested on: link:http://github.com/cirosantilli/linux-kernel-module-cheat/commit/38fd6153d965ba20145f53dc1bb3ba34b336bde9[38fd6153d965ba20145f53dc1bb3ba34b336bde9]
Tested on: link:https://github.com/cirosantilli/linux-kernel-module-cheat/commit/38fd6153d965ba20145f53dc1bb3ba34b336bde9[38fd6153d965ba20145f53dc1bb3ba34b336bde9]
==== Graphic mode gem5 aarch64
@@ -6570,7 +6572,7 @@ sh -x ./fops.sh
We have put printks on each fop, so this allows you to see which system calls are being made for each command.
No, there no official documentation: http://stackoverflow.com/questions/15213932/what-are-the-struct-file-operations-arguments
No, there no official documentation: https://stackoverflow.com/questions/15213932/what-are-the-struct-file-operations-arguments
==== seq_file
@@ -6843,8 +6845,8 @@ The sleep is done with `usleep_range`, see: <<sleep>>.
Bibliography:
* http://stackoverflow.com/questions/10177641/proper-way-of-handling-threads-in-kernel
* http://stackoverflow.com/questions/4084708/how-to-wait-for-a-linux-kernel-thread-kthreadto-exit
* https://stackoverflow.com/questions/10177641/proper-way-of-handling-threads-in-kernel
* https://stackoverflow.com/questions/4084708/how-to-wait-for-a-linux-kernel-thread-kthreadto-exit
==== kthreads
@@ -7041,7 +7043,7 @@ Therefore they produce more accurate timing than thread scheduling, which is mor
Bibliography:
* http://stackoverflow.com/questions/10812858/timers-in-linux-device-drivers
* https://stackoverflow.com/questions/10812858/timers-in-linux-device-drivers
* https://gist.github.com/yagihiro/310149
=== IRQ
@@ -7831,7 +7833,7 @@ It only appears once on every log I've seen so far, checked with `grep 0x1000000
+
Then when we count the instructions that run before the kernel entry point, there is only about 100k instructions, which is insignificant compared to the kernel boot itself.
+
TODO `--arch arm` and `--arch aarch64` does not count firmware instructions properly because the entry point address of the ELF file (`ffffff8008080000` for `aarch64`) does not show up on the trace at all. Tested on link:http://github.com/cirosantilli/linux-kernel-module-cheat/commit/f8c0502bb2680f2dbe7c1f3d7958f60265347005[f8c0502bb2680f2dbe7c1f3d7958f60265347005].
TODO `--arch arm` and `--arch aarch64` does not count firmware instructions properly because the entry point address of the ELF file (`ffffff8008080000` for `aarch64`) does not show up on the trace at all. Tested on link:https://github.com/cirosantilli/linux-kernel-module-cheat/commit/f8c0502bb2680f2dbe7c1f3d7958f60265347005[f8c0502bb2680f2dbe7c1f3d7958f60265347005].
* We can also discount the instructions after `init` runs by using `readelf` to get the initial address of `init`. One easy way to do that now is to just run:
+
....
@@ -8492,7 +8494,7 @@ Bibliography:
* https://en.wikipedia.org/wiki/Direct_Rendering_Manager
* https://en.wikipedia.org/wiki/Mode_setting KMS
Tested on: link:http://github.com/cirosantilli/linux-kernel-module-cheat/commit/93e383902ebcc03d8a7ac0d65961c0e62af9612b[93e383902ebcc03d8a7ac0d65961c0e62af9612b]
Tested on: link:https://github.com/cirosantilli/linux-kernel-module-cheat/commit/93e383902ebcc03d8a7ac0d65961c0e62af9612b[93e383902ebcc03d8a7ac0d65961c0e62af9612b]
==== kmscube
@@ -8541,7 +8543,7 @@ failed to initialize legacy DRM
See also: https://github.com/robclark/kmscube/issues/12 and https://stackoverflow.com/questions/26920835/can-egl-application-run-in-console-mode/26921287#26921287
Tested on: link:http://github.com/cirosantilli/linux-kernel-module-cheat/commit/2903771275372ccfecc2b025edbb0d04c4016930[2903771275372ccfecc2b025edbb0d04c4016930]
Tested on: link:https://github.com/cirosantilli/linux-kernel-module-cheat/commit/2903771275372ccfecc2b025edbb0d04c4016930[2903771275372ccfecc2b025edbb0d04c4016930]
==== kmscon
@@ -9386,7 +9388,7 @@ and go back to the terminal with:
Ctrl-A C
....
* http://stackoverflow.com/questions/14165158/how-to-switch-to-qemu-monitor-console-when-running-with-curses
* https://stackoverflow.com/questions/14165158/how-to-switch-to-qemu-monitor-console-when-running-with-curses
* https://superuser.com/questions/488263/how-to-switch-to-the-qemu-control-panel-with-nographics
When in graphic mode, we can do it from the GUI:
@@ -10372,7 +10374,7 @@ Source: link:userland/libs/eigen/hello.cpp[]
This example just creates a matrix and prints it out.
Tested on: link:http://github.com/cirosantilli/linux-kernel-module-cheat/commit/a4bdcf102c068762bb1ef26c591fcf71e5907525[a4bdcf102c068762bb1ef26c591fcf71e5907525]
Tested on: link:https://github.com/cirosantilli/linux-kernel-module-cheat/commit/a4bdcf102c068762bb1ef26c591fcf71e5907525[a4bdcf102c068762bb1ef26c591fcf71e5907525]
===== PARSEC benchmark
@@ -11328,7 +11330,7 @@ We setup 2 big and 2 small CPUs, but `cat /proc/cpuinfo` shows 4 identical CPUs
TODO: why is the `--dtb` required despite `fs_bigLITTLE.py` having a DTB generation capability? Without it, nothing shows on terminal, and the simulation terminates with `simulate() limit reached @ 18446744073709551615`. The magic `vmlinux.vexpress_gem5_v1.20170616` works however without a DTB.
Tested on: link:http://github.com/cirosantilli/linux-kernel-module-cheat/commit/18c1c823feda65f8b54cd38e261c282eee01ed9f[18c1c823feda65f8b54cd38e261c282eee01ed9f]
Tested on: link:https://github.com/cirosantilli/linux-kernel-module-cheat/commit/18c1c823feda65f8b54cd38e261c282eee01ed9f[18c1c823feda65f8b54cd38e261c282eee01ed9f]
=== gem5 unit tests
@@ -12065,7 +12067,7 @@ Particularly important numerical analysis instruction, that is used in particula
FMA is so important that <<ieee-754>> specifies it with single precision drop compared to a separate add and multiply!
Micro-op fun: http://stackoverflow.com/questions/28630864/how-is-fma-implemented
Micro-op fun: https://stackoverflow.com/questions/28630864/how-is-fma-implemented
Historically, FMA instructions have been added relatively late to instruction sets.
@@ -12331,8 +12333,8 @@ Official specification: http://infocenter.arm.com/help/topic/com.arm.doc.ihi0042
Bibliography:
* https://en.wikipedia.org/wiki/Calling_convention#ARM_(A32) Wiki contains the master list as usual.
* http://stackoverflow.com/questions/8422287/calling-c-functions-from-arm-assembly
* http://stackoverflow.com/questions/261419/arm-to-c-calling-convention-registers-to-save
* https://stackoverflow.com/questions/8422287/calling-c-functions-from-arm-assembly
* https://stackoverflow.com/questions/261419/arm-to-c-calling-convention-registers-to-save
* https://stackoverflow.com/questions/10494848/arm-whats-the-difference-between-apcs-and-aapcs-abi
=== GNU GAS assembler
@@ -12455,7 +12457,7 @@ Bibliography: https://stackoverflow.com/questions/27147043/n-suffix-to-branch-in
link:userland/arch/x86_64/char_literals.S[]
http://stackoverflow.com/questions/33246811/how-to-use-character-literals-in-gnu-gas-to-replace-numbers
https://stackoverflow.com/questions/33246811/how-to-use-character-literals-in-gnu-gas-to-replace-numbers
This syntax plays horribly with the C preprocessor:
@@ -12474,7 +12476,7 @@ No OPeration.
Does nothing except take up one processor cycle and occupy some instruction memory.
Applications: http://stackoverflow.com/questions/234906/whats-the-purpose-of-the-nop-opcode
Applications: https://stackoverflow.com/questions/234906/whats-the-purpose-of-the-nop-opcode
== x86 userland assembly
@@ -12616,7 +12618,7 @@ mov (%rsp), %rax
add $8, %rsp
....
Why do those instructions exist at all vs MOV / ADD / SUB: http://stackoverflow.com/questions/4584089/what-is-the-function-of-push-pop-registers-in-x86-assembly/33583134#33583134
Why do those instructions exist at all vs MOV / ADD / SUB: https://stackoverflow.com/questions/4584089/what-is-the-function-of-push-pop-registers-in-x86-assembly/33583134#33583134
==== x86 CQTO and CLTQ instructions
@@ -12697,14 +12699,14 @@ where `cc` are the same flags as Jcc.
Vs jmp:
* http://stackoverflow.com/questions/14131096/why-is-a-conditional-move-not-vulnerable-for-branch-prediction-failure
* http://stackoverflow.com/questions/27136961/what-is-it-about-cmov-which-improves-cpu-pipeline-performance
* http://stackoverflow.com/questions/26154488/difference-between-conditional-instructions-cmov-and-jump-instructions
* http://stackoverflow.com/questions/6754454/speed-difference-between-if-else-and-ternary-operator-in-c?lq=1#comment8007791_6754495
* https://stackoverflow.com/questions/14131096/why-is-a-conditional-move-not-vulnerable-for-branch-prediction-failure
* https://stackoverflow.com/questions/27136961/what-is-it-about-cmov-which-improves-cpu-pipeline-performance
* https://stackoverflow.com/questions/26154488/difference-between-conditional-instructions-cmov-and-jump-instructions
* https://stackoverflow.com/questions/6754454/speed-difference-between-if-else-and-ternary-operator-in-c?lq=1#comment8007791_6754495
Not necessarily faster because of branch prediction.
This is partly why the ternary `?` C operator exists: http://stackoverflow.com/questions/3565368/ternary-operator-vs-if-else
This is partly why the ternary `?` C operator exists: https://stackoverflow.com/questions/3565368/ternary-operator-vs-if-else
It is interesting to compare this with ARMv7 conditional executaion: which is available for all instructions: <<arm-conditional-execution>>
@@ -12880,7 +12882,7 @@ REP and REPZ also additionally stop if the comparison operation they repeat fail
link:userland/arch/x86_64/enter.S[]
These instructions were designed to allocate and deallocate function stack frames in the prologue and epilogue: http://stackoverflow.com/questions/5959890/enter-vs-push-ebp-mov-ebp-esp-sub-esp-imm-and-leave-vs-mov-esp-ebp
These instructions were designed to allocate and deallocate function stack frames in the prologue and epilogue: https://stackoverflow.com/questions/5959890/enter-vs-push-ebp-mov-ebp-esp-sub-esp-imm-and-leave-vs-mov-esp-ebp
ENTER appears obsolete and is kept mostly for backwards compatibility. LEAVE is still emitted by some compilers.
@@ -12897,7 +12899,7 @@ which implies an allocation of:
* one dword to remember EBP
* A bytes for local function variables
I didn't have the patience to study the B parameter, and it does not seem to be used often: http://stackoverflow.com/questions/26323215/do-any-languages-compilers-utilize-the-x86-enter-instruction-with-a-nonzero-ne
I didn't have the patience to study the B parameter, and it does not seem to be used often: https://stackoverflow.com/questions/26323215/do-any-languages-compilers-utilize-the-x86-enter-instruction-with-a-nonzero-ne
LEAVE is equivalent to:
@@ -13005,7 +13007,7 @@ Instructions such as FLDL convert standard <<ieee-754>> 64-bit values from memor
==== x86 x87 FPU vs SIMD
http://stackoverflow.com/questions/1844669/benefits-of-x87-over-sse
https://stackoverflow.com/questions/1844669/benefits-of-x87-over-sse
Modern x86 has two main ways of doing floating point operations:
@@ -13622,7 +13624,7 @@ The registers are encoded as single bits inside the instruction: each bit repres
As a consequence, the push order is fixed no matter how you write the assembly instruction: there is just not enough space to encode ordering.
AArch64 loses those instructions, likely because it was not possible anymore to encode all registers: http://stackoverflow.com/questions/27941220/push-lr-and-pop-lr-in-arm-arch64 and replaces them with the <<armv8-aarch64-ldp-and-stp-instructions>>
AArch64 loses those instructions, likely because it was not possible anymore to encode all registers: https://stackoverflow.com/questions/27941220/push-lr-and-pop-lr-in-arm-arch64 and replaces them with the <<armv8-aarch64-ldp-and-stp-instructions>>
=== ARM data processing instructions
@@ -15772,7 +15774,9 @@ but note that this does not include script specific options.
=== Build the documentation
You don't need to depend on GitHub. To build the first time use:
You don't need to depend on GitHub.
To build the first time use:
....
./build --download-dependencies docs
@@ -15794,7 +15798,6 @@ Then just open the HTML output at:
xdg-open out/README.html
....
[[documentation-verification]]
==== Documentation verification
When running link:build-doc[], we do the following checks:
@@ -15853,6 +15856,43 @@ explicitly-given
One application we have in mind for this is that as of 2.0.10 Asciidoctor does not warn on header ID collisions between auto-generated IDs: https://github.com/asciidoctor/asciidoctor/issues/3147 But this script doesn't solve that yet as it would require generating the section IDs without the `-N` suffix. Section generation happens at `Section.generate_id` in Asciidoctor code.
[[asciidoctor-link-target-up-rb]]
=== asciidoctor/link-target-up.rb
The Asciidoctor extension scripts:
* link:asciidoctor-link-up.rb
* link:asciidoctor-link-github.rb
hack the README `link:` targets to make them work from:
* inside the `out/` directory with `../`
* <<github-pages>>, with explicit GitHub blob URLs
==== GitHub pages
As mentioned before the TOC, we have to push this README to GitHub pages due to: https://github.com/isaacs/github/issues/1610
For now, instead of pushing with `git push`, I just remember to always push with:
....
./publish-gh-pages
....
Source: link:publish-gh-pages[]
I'm going this way for now because:
* the Jekyll Asciidoctor plugin is not enabled by default on GitHub: https://webapps.stackexchange.com/questions/114606/can-github-pages-render-asciidoc
* link:https://stackoverflow.com/questions/1797074/local-executing-hook-after-a-git-push[post-push hooks don't exist]
* I'm lazy to setup a proper Travis CI push
* I'm the only contributor essentially, so no problems with pull requests
The only files used by the GitHub pages are:
* link:Gemfile[]
* link:_config.yml[]
=== Clean the build
You did something crazy, and nothing seems to work anymore?