mirror of
https://github.com/cirosantilli/linux-kernel-module-cheat.git
synced 2026-01-25 11:11:35 +01:00
378 lines
14 KiB
Plaintext
378 lines
14 KiB
Plaintext
[[gem5]]
|
|
= GEM5
|
|
|
|
[[introduction]]
|
|
== Introduction
|
|
|
|
GEM5 is a system simulator, much like QEMU: http://gem5.org/
|
|
|
|
Vs QEMU:
|
|
|
|
* advantage: simulates a generic more realistic pipelined and optionally
|
|
out of order CPU cycle by cycle, including a realistic DRAM memory
|
|
access model with latencies, caches and page table manipulations. This
|
|
allows us to:
|
|
** do much more realistic performance benchmarking with it, which makes
|
|
absolutely no sense in QEMU, which is purely functional
|
|
** make functional cache observations, e.g. to use Linux kernel APIs
|
|
that flush memory like DMA, which are crucial for driver development. In
|
|
QEMU, the driver would still work even if we forget to flush caches.
|
|
+
|
|
It is not of course truly cycle accurate, as that would require exposing
|
|
proprietary information of the CPU designs:
|
|
https://stackoverflow.com/questions/17454955/can-you-check-performance-of-a-program-running-with-qemu-simulator/33580850#33580850,
|
|
but the approximation is reasonable.
|
|
+
|
|
It is used mostly for research purposes: when you are making a new chip
|
|
technology, you don't really need to specialize enormously to an
|
|
existing microarchitecture, but rather develop something that will work
|
|
with a wide range of future architectures.
|
|
* disadvantage: slower than QEMU by TODO 10x?
|
|
+
|
|
This also implies that the user base is much smaller, since no Android
|
|
devs.
|
|
+
|
|
Instead, we have only chip makers, who keep everything that really works
|
|
closed, and researchers, who can't version track or document code
|
|
properly >:-) And this implies that:
|
|
** the documentation is more scarce
|
|
** it takes longer to support new hardware features
|
|
|
|
[[arm]]
|
|
== ARM
|
|
|
|
....
|
|
./configure && ./build -a arm-gem5
|
|
./rungem5 -a arm-gem5
|
|
....
|
|
|
|
On another shell:
|
|
|
|
....
|
|
./rungem5-shell
|
|
....
|
|
|
|
[[kernel-command-line-arguments]]
|
|
=== Kernel command line arguments
|
|
|
|
E.g., to add `printk.time=y`, run:
|
|
|
|
....
|
|
./rungem5 -a arm-gem5 -- --command-line='earlyprintk=pl011,0x1c090000 console=ttyAMA0 lpj=19988480 norandmaps rw loglevel=8 mem=512MB root=/dev/sda printk.time=y'
|
|
....
|
|
|
|
When you use `--command-line=`, it overrides default command lines,
|
|
which are required to boot properly.
|
|
|
|
So if you pass just `--command-line='printk.time=y'`, it removes the
|
|
required options, and boot fails.
|
|
|
|
An easy way to find the other options is to to an initial boot:
|
|
|
|
....
|
|
./rungem5 -a arm-gem5
|
|
....
|
|
|
|
and then look at the line of the linux kernel that starts with
|
|
|
|
....
|
|
Kernel command line:
|
|
....
|
|
|
|
We might copy the default `--command-line` into our startup scripts to
|
|
make things easier at some point, but it would be fun to debug when the
|
|
defaults change upstream and we don't notice :-(
|
|
|
|
[[qemu-with-gem5-kernel-configuration]]
|
|
=== QEMU with GEM5 kernel configuration
|
|
|
|
TODO: QEMU did not work with the GEM5 kernel configurations.
|
|
|
|
To test this, hack up `run` to use the `buildroot/output.arm-gem5~`
|
|
directory, and then run:
|
|
|
|
....
|
|
./run -a arm
|
|
....
|
|
|
|
Now QEMU hangs at:
|
|
|
|
....
|
|
audio: Could not init `oss' audio driver
|
|
....
|
|
|
|
and the display shows:
|
|
|
|
....
|
|
Guest has not initialized the display (yet).
|
|
....
|
|
|
|
[[gem5-with-qemu-kernel-configuration]]
|
|
=== GEM5 with QEMU kernel configuration
|
|
|
|
Test it out with:
|
|
|
|
....
|
|
./rungem5 -a arm
|
|
....
|
|
|
|
TODO hangs at:
|
|
|
|
....
|
|
**** REAL SIMULATION ****
|
|
warn: Existing EnergyCtrl, but no enabled DVFSHandler found.
|
|
info: Entering event queue @ 0. Starting simulation...
|
|
1614868500: system.terminal: attach terminal 0
|
|
....
|
|
|
|
and the `telnet` at:
|
|
|
|
....
|
|
2017-12-28-11-59-51@ciro@ciro-p51$ ./rungem5-shell
|
|
Trying 127.0.0.1...
|
|
Connected to localhost.
|
|
Escape character is '^]'.
|
|
==== m5 slave terminal: Terminal 0 ====
|
|
....
|
|
|
|
I have also tried to copy the exact same kernel command line options
|
|
used by QEMU, but nothing changed.
|
|
|
|
[[x86]]
|
|
== x86
|
|
|
|
TODO didn't get it working yet.
|
|
|
|
Related threads:
|
|
|
|
* https://www.mail-archive.com/gem5-users@gem5.org/msg11384.html
|
|
* https://stackoverflow.com/questions/37906425/booting-gem5-x86-ubuntu-full-system-simulation
|
|
* http://www.lowepower.com/jason/creating-disk-images-for-gem5.html
|
|
claims to have a working config for x86_64 kernel 4.8.13
|
|
|
|
[[our-best-attempt]]
|
|
=== Our best attempt
|
|
|
|
....
|
|
./configure && ./build -a x86_64-gem5
|
|
./rungem5 -a x86_64-gem5
|
|
....
|
|
|
|
telnet:
|
|
|
|
....
|
|
i8042: PNP: No PS/2 controller found.
|
|
i8042: Probing ports directly.
|
|
Connection closed by foreign host.
|
|
....
|
|
|
|
stdout:
|
|
|
|
....
|
|
panic: Data written for unrecognized command 0xd1
|
|
Memory Usage: 1235908 KBytes
|
|
Program aborted at tick 427627410500
|
|
....
|
|
|
|
The same failure happens if we use the working QEMU Linux kernel, and /
|
|
or if we use the kernel 4.8.13 as proposed in lowepower's post..
|
|
|
|
If we look a bit into the source, the panic message comes from
|
|
`i8042.cc`, and on the header we see that the missing command is:
|
|
|
|
....
|
|
WriteOutputPort = 0xD1,
|
|
....
|
|
|
|
The kernel was compiled with `CONFIG_SERIO_I8042=y`, I didn't dare
|
|
disable it yet. The Linux kernel driver has no `grep` hits for either of
|
|
`0xd1` nor `output.?port`, it must be using some random bitmask to build
|
|
it then.
|
|
|
|
This byte is documented at
|
|
http://wiki.osdev.org/%228042%22_PS/2_Controller, as usual :-)
|
|
|
|
There are also a bunch of `i8042` kernel CLI options, I tweaked all of
|
|
them but nothing.
|
|
|
|
[[working-baseline-with-magic-image]]
|
|
=== Working baseline with magic image
|
|
|
|
Working x86 with the pre-built magic image with an ancient 2.6.22.9
|
|
kernel starting point:
|
|
|
|
....
|
|
sudo mkdir -p /dist/m5/system
|
|
sudo chmod 777 /dist/m5/system
|
|
cd /dist/m5/system
|
|
# Backed up at:
|
|
# https://github.com/cirosantilli/media/releases/tag/gem5
|
|
wget http://www.gem5.org/dist/current/x86/x86-system.tar.bz2
|
|
tar xvf x86-system.tar.bz2
|
|
cd x86-system
|
|
dd if=/dev/zero of=disks/linux-bigswap2.img bs=1024 count=65536
|
|
mkswap disks/linux-bigswap2.img
|
|
cd ..
|
|
|
|
git clone https://gem5.googlesource.com/public/gem5
|
|
cd gem5
|
|
git checkout da79d6c6cde0fbe5473ce868c9be4771160a003b
|
|
scons -j$(nproc) build/X86/gem5.opt
|
|
# That old blob has wrong filenames.
|
|
./build/X86/gem5.opt \
|
|
-d /tmp/output \
|
|
--disk-image=/dist/m5/system/disks/linux-x86.img \
|
|
--kernel=/dist/m5/system/binaries/x86_64-vmlinux-2.6.22.9 \
|
|
configs/example/fs.py
|
|
....
|
|
|
|
On another shell:
|
|
|
|
....
|
|
telnet localhost 3456
|
|
....
|
|
|
|
[[unmodified-buildroot-images-2]]
|
|
=== Unmodified Buildroot images 2
|
|
|
|
bzImage fails, so we always try with vmlinux obtained from inside
|
|
build/.
|
|
|
|
rootfs.ext2 and vmlinux from
|
|
670366caaded57d318b6dbef34e863e3b30f7f29ails as:
|
|
|
|
Fails as:
|
|
|
|
....
|
|
Global frequency set at 1000000000000 ticks per second
|
|
warn: DRAM device capacity (8192 Mbytes) does not match the address range assigned (512 Mbytes)
|
|
info: kernel located at: /data/git/linux-kernel-module-cheat/buildroot/output.x86_64~/build/linux-custom/vmlinux
|
|
Listening for com_1 connection on port 3456
|
|
0: rtc: Real-time clock set to Sun Jan 1 00:00:00 2012
|
|
0: system.remote_gdb.listener: listening for remote gdb #0 on port 7000
|
|
warn: Reading current count from inactive timer.
|
|
**** REAL SIMULATION ****
|
|
info: Entering event queue @ 0. Starting simulation...
|
|
warn: instruction 'fninit' unimplemented
|
|
warn: Don't know what interrupt to clear for console.
|
|
12516923000: system.pc.com_1.terminal: attach terminal 0
|
|
warn: i8042 "Write output port" command not implemented.
|
|
warn: i8042 "Write keyboard output buffer" command not implemented.
|
|
warn: Write to unknown i8042 (keyboard controller) command port.
|
|
hack: Assuming logical destinations are 1 << id.
|
|
panic: Resetting mouse wrap mode unimplemented.
|
|
Memory Usage: 1003456 KBytes
|
|
Program aborted at tick 632745027500
|
|
--- BEGIN LIBC BACKTRACE ---
|
|
./build/X86/gem5.opt(_Z15print_backtracev+0x15)[0x12b8165]
|
|
./build/X86/gem5.opt(_Z12abortHandleri+0x39)[0x12c32f9]
|
|
/lib/x86_64-linux-gnu/libpthread.so.0(+0x11390)[0x7fe047a71390]
|
|
/lib/x86_64-linux-gnu/libc.so.6(gsignal+0x38)[0x7fe046601428]
|
|
/lib/x86_64-linux-gnu/libc.so.6(abort+0x16a)[0x7fe04660302a]
|
|
./build/X86/gem5.opt(_ZN6X86ISA8PS2Mouse11processDataEh+0xf5)[0x1391095]
|
|
./build/X86/gem5.opt(_ZN6X86ISA5I80425writeEP6Packet+0x51c)[0x13927ec]
|
|
./build/X86/gem5.opt(_ZN7PioPort10recvAtomicEP6Packet+0x66)[0x139f7b6]
|
|
./build/X86/gem5.opt(_ZN15NoncoherentXBar10recvAtomicEP6Packets+0x200)[0x1434af0]
|
|
./build/X86/gem5.opt(_ZN6Bridge15BridgeSlavePort10recvAtomicEP6Packet+0x5d)[0x140ee9d]
|
|
./build/X86/gem5.opt(_ZN12CoherentXBar10recvAtomicEP6Packets+0x3e7)[0x1415b77]
|
|
./build/X86/gem5.opt(_ZN15AtomicSimpleCPU8writeMemEPhjm5FlagsIjEPm+0x327)[0xa790a7]
|
|
./build/X86/gem5.opt(_ZN17SimpleExecContext8writeMemEPhjm5FlagsIjEPm+0x19)[0xa856b9]
|
|
./build/X86/gem5.opt(_ZNK10X86ISAInst2St7executeEP11ExecContextPN5Trace10InstRecordE+0x235)[0xfb9e65]
|
|
./build/X86/gem5.opt(_ZN15AtomicSimpleCPU4tickEv+0x23c)[0xa784fc]
|
|
./build/X86/gem5.opt(_ZN10EventQueue10serviceOneEv+0xc5)[0x12be0d5]
|
|
./build/X86/gem5.opt(_Z9doSimLoopP10EventQueue+0x38)[0x12cd558]
|
|
./build/X86/gem5.opt(_Z8simulatem+0x2eb)[0x12cdbdb]
|
|
./build/X86/gem5.opt(_ZZN8pybind1112cpp_function10initializeIRPFP22GlobalSimLoopExitEventmES3_ImEINS_4nameENS_5scopeENS_7siblingENS_5arg_vEEEEvOT_PFT0_DpT1_EDpRKT2_ENUlRNS_6detail13function_callEE1_4_FUNESO_+0x41)[0x13fca11]
|
|
./build/X86/gem5.opt(_ZN8pybind1112cpp_function10dispatcherEP7_objectS2_S2_+0x8d8)[0xfc7398]
|
|
/usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x7852)[0x7fe047d3b552]
|
|
/usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x85c)[0x7fe047e6501c]
|
|
/usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x6ffd)[0x7fe047d3acfd]
|
|
/usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x7124)[0x7fe047d3ae24]
|
|
/usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x7124)[0x7fe047d3ae24]
|
|
/usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x85c)[0x7fe047e6501c]
|
|
/usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0(PyEval_EvalCode+0x19)[0x7fe047d33b89]
|
|
/usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x613b)[0x7fe047d39e3b]
|
|
/usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x85c)[0x7fe047e6501c]
|
|
/usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x6ffd)[0x7fe047d3acfd]
|
|
/usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x85c)[0x7fe047e6501c]
|
|
/usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0(PyEval_EvalCode+0x19)[0x7fe047d33b89]
|
|
--- END LIBC BACKTRACE ---
|
|
Aborted (core dumped)
|
|
....
|
|
|
|
Boot goes quite far, on telnet:
|
|
|
|
....
|
|
ALSA device list:
|
|
No soundcards found.
|
|
....
|
|
|
|
So just looks like we have to disable some Linux configs which GEM5 does
|
|
not support... so fragile.
|
|
|
|
[[copy-upstream-2.6-configs-on-4.9-kernel]]
|
|
=== Copy upstream 2.6 configs on 4.9 kernel
|
|
|
|
The magic image provides its kernel configurations, so let's try that.
|
|
|
|
The configs are present at:
|
|
|
|
....
|
|
wget http://www.gem5.org/dist/current/x86/config-x86.tar.bz2
|
|
....
|
|
|
|
backed up at: https://github.com/cirosantilli/media/releases/tag/gem5
|
|
|
|
Copy `linux-2.6.22.9` into the kernel tree as `.config`,
|
|
`git checkout v4.9.6`, `make olddefconfig`, `make`, then use the
|
|
Buildroot filesystem as above, failure:
|
|
|
|
....
|
|
panic: Invalid IDE control register offset: 0
|
|
Memory Usage: 931272 KBytes
|
|
Program aborted at tick 382834812000
|
|
--- BEGIN LIBC BACKTRACE ---
|
|
./build/X86/gem5.opt(_Z15print_backtracev+0x15)[0x12b8165]
|
|
./build/X86/gem5.opt(_Z12abortHandleri+0x39)[0x12c32f9]
|
|
/lib/x86_64-linux-gnu/libpthread.so.0(+0x11390)[0x7fc2081c6390]
|
|
/lib/x86_64-linux-gnu/libc.so.6(gsignal+0x38)[0x7fc206d56428]
|
|
/lib/x86_64-linux-gnu/libc.so.6(abort+0x16a)[0x7fc206d5802a]
|
|
./build/X86/gem5.opt(_ZN7IdeDisk11readControlEmiPh+0xd9)[0xa96989]
|
|
./build/X86/gem5.opt(_ZN13IdeController14dispatchAccessEP6Packetb+0x53e)[0xa947ae]
|
|
./build/X86/gem5.opt(_ZN13IdeController4readEP6Packet+0xe)[0xa94a5e]
|
|
./build/X86/gem5.opt(_ZN7PioPort10recvAtomicEP6Packet+0x3f)[0x139f78f]
|
|
./build/X86/gem5.opt(_ZN15NoncoherentXBar10recvAtomicEP6Packets+0x200)[0x1434af0]
|
|
./build/X86/gem5.opt(_ZN6Bridge15BridgeSlavePort10recvAtomicEP6Packet+0x5d)[0x140ee9d]
|
|
./build/X86/gem5.opt(_ZN12CoherentXBar10recvAtomicEP6Packets+0x3e7)[0x1415b77]
|
|
./build/X86/gem5.opt(_ZN15AtomicSimpleCPU7readMemEmPhj5FlagsIjE+0x3ef)[0xa780ef]
|
|
./build/X86/gem5.opt(_ZN17SimpleExecContext7readMemEmPhj5FlagsIjE+0x11)[0xa85671]
|
|
./build/X86/gem5.opt(_ZNK10X86ISAInst2Ld7executeEP11ExecContextPN5Trace10InstRecordE+0x130)[0xfb6c00]
|
|
./build/X86/gem5.opt(_ZN15AtomicSimpleCPU4tickEv+0x23c)[0xa784fc]
|
|
./build/X86/gem5.opt(_ZN10EventQueue10serviceOneEv+0xc5)[0x12be0d5]
|
|
./build/X86/gem5.opt(_Z9doSimLoopP10EventQueue+0x38)[0x12cd558]
|
|
./build/X86/gem5.opt(_Z8simulatem+0x2eb)[0x12cdbdb]
|
|
./build/X86/gem5.opt(_ZZN8pybind1112cpp_function10initializeIRPFP22GlobalSimLoopExitEventmES3_ImEINS_4nameENS_5scopeENS_7siblingENS_5arg_vEEEEvOT_PFT0_DpT1_EDpRKT2_ENUlRNS_6detail13function_callEE1_4_FUNESO_+0x41)[0x13fca11]
|
|
./build/X86/gem5.opt(_ZN8pybind1112cpp_function10dispatcherEP7_objectS2_S2_+0x8d8)[0xfc7398]
|
|
/usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x7852)[0x7fc208490552]
|
|
/usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x85c)[0x7fc2085ba01c]
|
|
/usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x6ffd)[0x7fc20848fcfd]
|
|
/usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x7124)[0x7fc20848fe24]
|
|
/usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x7124)[0x7fc20848fe24]
|
|
/usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x85c)[0x7fc2085ba01c]
|
|
/usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0(PyEval_EvalCode+0x19)[0x7fc208488b89]
|
|
/usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x613b)[0x7fc20848ee3b]
|
|
/usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x85c)[0x7fc2085ba01c]
|
|
/usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0(PyEval_EvalFrameEx+0x6ffd)[0x7fc20848fcfd]
|
|
/usr/lib/x86_64-linux-gnu/libpython2.7.so.1.0(PyEval_EvalCodeEx+0x85c)[0x7fc2085ba01c]
|
|
--- END LIBC BACKTRACE ---
|
|
Aborted (core dumped)
|
|
....
|
|
|
|
[[use-upstream-2.6-configs-and-2.6-kernel]]
|
|
=== Use upstream 2.6 configs and 2.6 kernel
|
|
|
|
If we checkout to the ancient kernel `v2.6.22.9`, it fails to compile
|
|
with modern GNU make 4.1:
|
|
https://stackoverflow.com/questions/35002691/makefile-make-clean-why-getting-mixed-implicit-and-normal-rules-deprecated-s
|
|
lol
|