mkdir -p /mnt/9p/gem5 +mount -t 9p -o trans=virtio,version=9p2000.L gem5 /mnt/9p/data+
If you don’t know which one to go for, start with QEMU Buildroot setup getting started.
Design goals of this project are documented at: Section 30.18.1, “Design goals”.
+Design goals of this project are documented at: Section 31.18.1, “Design goals”.
This setup has been mostly tested on Ubuntu. For other host operating systems see: Section 30.1, “Supported hosts”. For greater stability, consider using the latest release instead of master: https://github.com/cirosantilli/linux-kernel-module-cheat/releases
+This setup has been mostly tested on Ubuntu. For other host operating systems see: Section 31.1, “Supported hosts”. For greater stability, consider using the latest release instead of master: https://github.com/cirosantilli/linux-kernel-module-cheat/releases
Reserve 12Gb of disk and run:
@@ -1888,7 +1906,7 @@ cd linux-kernel-module-cheatYou don’t need to clone recursively even though we have .git submodules: download-dependencies fetches just the submodules that you need for this build to save time.
If something goes wrong, see: Section 30.2, “Common build issues” and use our issue tracker: https://github.com/cirosantilli/linux-kernel-module-cheat/issues
+If something goes wrong, see: Section 31.2, “Common build issues” and use our issue tracker: https://github.com/cirosantilli/linux-kernel-module-cheat/issues
The initial build will take a while (30 minutes to 2 hours) to clone and build, see Benchmark builds for more details.
@@ -1971,7 +1989,7 @@ hello2 cleanupTo avoid typing --arch aarch64 many times, you can set the default arch as explained at: Section 30.4, “Default command line arguments”
To avoid typing --arch aarch64 many times, you can set the default arch as explained at: Section 31.4, “Default command line arguments”
I now urge you to read the following sections which contain widely applicable information:
@@ -2275,7 +2293,7 @@ hello /root/.profileIf you really want to develop semiconductors, your only choice is to join an university or a semiconductor company that has the EDA licenses.
See also: Section 30.19.2, “Should you waste your life with systems programming?”.
+See also: Section 31.19.2, “Should you waste your life with systems programming?”.
While hacking QEMU, you will likely want to GDB step its source. That is trivial since QEMU is just another userland program like any other, but our setup has a shortcut to make it even more convenient, see: Section 18.7, “Debug the emulator”.
@@ -2714,7 +2732,7 @@ j = 0This repository has been tested inside clean Docker containers.
This is a good option if you are on a Linux host, but the native setup failed due to your weird host distribution, and you have better things to do with your life than to debug it. See also: Section 30.1, “Supported hosts”.
+This is a good option if you are on a Linux host, but the native setup failed due to your weird host distribution, and you have better things to do with your life than to debug it. See also: Section 31.1, “Supported hosts”.
For example, to do a QEMU Buildroot setup inside Docker, run:
@@ -3792,7 +3810,7 @@ xdg-open README.htmlMore information about our documentation internals can be found at: Section 30.5, “Documentation”
+More information about our documentation internals can be found at: Section 31.5, “Documentation”
The gem5 tests require building statically with build id static, see also: Section 10.6, “gem5 syscall emulation mode”. TODO automate this better.
See: Section 30.13, “Test this repo” for more useful testing tips.
+See: Section 31.13, “Test this repo” for more useful testing tips.
modules built with Buildroot, see: Section 30.12.2.1, “kernel_modules buildroot package”
+modules built with Buildroot, see: Section 31.12.2.1, “kernel_modules buildroot package”
modules built from the kernel tree itself, see: Section 15.12.2, “dummy-irq”
@@ -8504,7 +8522,7 @@ git -C "$(./getvar linux_source_dir)" checkout -We cannot use mainline Linux because the gem5 arm Linux kernel patches are required at least to provide the CONFIG_DRM_VIRT_ENCODER option.
We disable networking by default because it starts an userland process, and we want to keep the number of userland processes to a minimum to make the system more understandable as explained at: Section 30.18.3, “Resource tradeoff guidelines”
+We disable networking by default because it starts an userland process, and we want to keep the number of userland processes to a minimum to make the system more understandable as explained at: Section 31.18.3, “Resource tradeoff guidelines”
To enable networking on Buildroot, simply run:
@@ -9041,6 +9059,9 @@ uname -a > guest mount -t 9p -o trans=virtio,version=9p2000.L host0 /mnt/my9pwhere mount tag host0 is set by the emulator (mount_tag flag on QEMU CLI), and can be found in the guest with: cat /sys/bus/virtio/drivers/9pnet_virtio/virtio0/mount_tag as documented at: https://www.kernel.org/doc/Documentation/filesystems/9p.txt.
Launch QEMU with -virtfs as in your run script
From the source, there is just one exported tag named gem5, so we could try on the guest:
mkdir -p /mnt/9p/gem5 +mount -t 9p -o trans=virtio,version=9p2000.L gem5 /mnt/9p/data+
This also makes this repo the perfect setup to develop the Linux kernel.
In case something breaks while updating the Linux kernel, you can try to bisect it to understand the root cause, see: Section 30.14, “Bisection”.
+In case something breaks while updating the Linux kernel, you can try to bisect it to understand the root cause, see: Section 31.14, “Bisection”.
First, use use the branching procedure described at: Section 30.16, “Update a forked submodule”
+First, use use the branching procedure described at: Section 31.16, “Update a forked submodule”
Because the kernel is so central to this repository, almost all tests must be re-run, so basically just follow the full testing procedure described at: Section 30.13, “Test this repo”. The only tests that can be skipped are essentially the Baremetal tests.
+Because the kernel is so central to this repository, almost all tests must be re-run, so basically just follow the full testing procedure described at: Section 31.13, “Test this repo”. The only tests that can be skipped are essentially the Baremetal tests.
Before comitting, don’t forget to update:
@@ -16746,11 +16776,30 @@ IN:PANDA can list memory addresses, so I bet it can also decode the instructions: https://github.com/panda-re/panda/blob/883c85fa35f35e84a323ed3d464ff40030f06bd6/panda/docs/LINE_Censorship.md I wonder why they don’t just upstream those things to QEMU’s tracing: https://github.com/panda-re/panda/issues/290
gem5 can do it as shown at: Section 18.8.6, “gem5 tracing”.
+gem5 can do it as shown at: Section 18.8.7, “gem5 tracing”.
Not possible apparently, not even with the memory_region_ops_read and memory_region_ops_write trace events, Peter comments https://lists.gnu.org/archive/html/qemu-devel/2015-06/msg07482.html
++++No. You will miss all the fast-path memory accesses, which are +done with custom generated assembly in the TCG backend. In +general QEMU is not designed to support this kind of monitoring +of guest operations.
+
We can further use Binutils' addr2line to get the line that corresponds to each address:
QEMU runs, unlike gem5, are not deterministic by default, however it does support a record and replay mechanism that allows you to replay a previous run deterministically.
Solved on unmerged c42634d8e3428cfa60672c3ba89cabefc720cde9 from https://github.com/ispras/qemu/tree/rr-180725
TODO get working.
TODO: is there any way to distinguish which instruction runs on each core? Doing:
gem5 provides also provides a tracing mechanism documented at: http://www.gem5.org/Trace_Based_Debugging:
Our wrapper just forwards the options to the --debug-flags gem5 option.
Keep in mind however that the disassembly is very broken in several places as of 2019q2, so you can’t always trust it.
Be warned, the trace is humongous, at 16Gb.
+This would produce a lot of output however, so you will likely not want that when tracing a Linux kernel boot instructions. But it can be very convenient for smaller traces such as Baremetal.
The traces are generated from DPRINTF(<trace-id> calls scattered throughout the code.
Trace internals are discussed at: gem5 trace internals.
As can be seen on the Sconstruct, Exec is just an alias that enables a set of flags.
Be warned, the trace is humongous, at 16Gb.
-We can make the trace smaller by naming the trace file as trace.txt.gz, which enables GZIP compression, but that is not currently exposed on our scripts, since you usually just need something human readable to work on.
TODO: 7452d399290c9c1fc6366cdad129ef442f323564 ./trace2line this is too slow and takes hours. QEMU’s processing of 170k events takes 7 seconds. gem5’s processing is analogous, but there are 140M events, so it should take 7000 seconds ~ 2 hours which seems consistent with what I observe, so maybe there is no way to speed this up… The workaround is to just use gem5’s ExecSymbol to get function granularity, and then GDB individually if line detail is needed?
gem5 traces are generated from DPRINTF(<trace-id> calls scattered throughout the code, except for ExecAll instruction traces, which uses Debug::ExecEnable directly..
The trace IDs are themselves encoded in SConscript files, e.g.:
DebugFlag('Event'
+in src/cpu/SConscript.
The build system then automatically adds the options to the --debug-flags.
For this entry, the build system then generates a file build/ARM/debug/ExecEnable.hh, which contains:
namespace Debug {
+class SimpleFlag;
+extern SimpleFlag ExecEnable;
+}
+and must be included in from callers of DPRINTF( as <debug/ExecEnable.hh>.
Tested in b4879ae5b0b6644e6836b0881e4da05c64a6550d.
+This debug flag traces all instructions.
This flag shows a more detailed register usage than gem5 ExecAll trace format.
As of gem5 16eeee5356585441a49d05c78abc328ef09f7ace the default tracer is ExeTracer. It is set at:
gem5 ARM at least appears to implement more low level CPU functionality than QEMU, e.g. QEMU only added EL2 in 2018: https://stackoverflow.com/questions/42824706/qemu-system-aarch64-entering-el1-when-emulating-a53-power-up See also: Section 27.8.1, “ARM exception levels”
gem5 offers more advanced logging, even for non micro architectural things which QEMU models in some way, e.g. QEMU trace memory accesses, because QEMU’s binary translation optimizations reduce visibility
+Internals under other sections:
+In order to develop complex C++ software such as gem5, a good IDE setup is fundamental.
+The best setup I’ve reached is with Eclipse. It is not perfect, and there is a learning curve, but is worth it.
+I recommend the following settings, tested in Eclipse 2019.09, Ubuntu 18.04:
+fix all missing stdlib headers: https://stackoverflow.com/questions/10373788/how-to-solve-unresolved-inclusion-iostream-in-a-c-file-in-eclipse-cdt/51099533#51099533
+use spaces instead of tabs: Window, Preferences, Code Style, C/C++, Formatter, New, Edit, Tab Policy, Spaces Only
+add to the include search path:
+./src/ in the source tree
+the ISA specific build directory which contains some self-generated stuff, e.g.: out/gem5/default/build/ARM
+To run and GDB step debug the executable, just copy the full command line from the output ./run, and configure it into Eclipse.
The interaction uses the Python C extension interface https://docs.python.org/2/extending/extending.html interface through the pybind11 helper library: https://github.com/pybind/pybind11
The main is at: src/sim/main.cc. It calls:
ret = initM5Python();+
src/sim/init.cc:
+230 int
+231 initM5Python()
+232 {
+233 EmbeddedPyBind::initAll();
+234 return EmbeddedPython::initAll();
+235 }
+initAll basically just initializes the _m5 Python object, which is used across multiple .py.
Back on main:
ret = m5Main(argc, argv);+
which goes to:
+result = PyRun_String(*command, Py_file_input, dict, dict);+
with commands looping over:
+import m5 +m5.main()+
which leads into:
+src/python/m5/main.py#main+
which finally calls your config file like fs.py with:
filename = sys.argv[0] +filedata = file(filename, 'r').read() +filecode = compile(filedata, filename, 'exec') +[...] +exec filecode in scope+
TODO: the file path name appears to be passed as a command line argument to the Python script, but I didn’t have the patience to fully understand the details.
+Tested at gem5 b4879ae5b0b6644e6836b0881e4da05c64a6550d.
+gem5 is an event based simulator, and as such the event queue is of of the crucial elements in the system.
+The gem5 event queue stores one callback event for each future point in time.
+The event queue is implemented in the class EventQueue in the file src/sim/eventq.hh.
Not all times need to have an associated event: if a given time has no events, gem5 just skips it and jumps to the next event: the queue is basically a linked list of events.
+Important examples of events include:
+CPU ticks
+TODO peripherals and memory
+At the beginning of simulation, gem5 sets up exactly two events:
+the first CPU cycle
+one exit event at the end of time which triggers gem5 simulate() limit reached
+Tick events then get triggered one by one as simulation progresses, in addition to any other system events.
+The EventQueue class has one awesome dump() function that prints a human friendly representation of the queue, and can be easily called from GDB. TODO example.
We can also observe what is going on in the event queue with the Event debug flag.
Event execution is done at EventQueue::serviceOne():
Event *exit_event = eventq->serviceOne();+
This calls the Event::process method of the event.
Let’s now analyze every single event on a minimal gem5 syscall emulation mode in the simplest CPU that we have:
+./run \ + --arch aarch64 \ + --emulator gem5 \ + --userland userland/arch/aarch64/freestanding/linux/hello.S \ + --trace Event \ + --trace-stdout \ +;+
At LKMC a0ea29835b9bacc6aa1cceb24c79d895315991d4 + 1 this outputs:
+0: AtomicSimpleCPU tick.wrapped_function_event: EventFunctionWrapped event scheduled @ 0 +**** REAL SIMULATION **** + 0: Event_70: generic event scheduled @ 0 +info: Entering event queue @ 0. Starting simulation... + 0: Event_70: generic event rescheduled @ 18446744073709551615 + 0: AtomicSimpleCPU tick.wrapped_function_event: EventFunctionWrapped event rescheduled @ 500 + 500: AtomicSimpleCPU tick.wrapped_function_event: EventFunctionWrapped event rescheduled @ 1000 + 1000: AtomicSimpleCPU tick.wrapped_function_event: EventFunctionWrapped event rescheduled @ 1500 + 1500: AtomicSimpleCPU tick.wrapped_function_event: EventFunctionWrapped event rescheduled @ 2000 +hello + 2000: AtomicSimpleCPU tick.wrapped_function_event: EventFunctionWrapped event rescheduled @ 2500 + 2500: AtomicSimpleCPU tick.wrapped_function_event: EventFunctionWrapped event rescheduled @ 3000 + 3000: AtomicSimpleCPU tick.wrapped_function_event: EventFunctionWrapped event rescheduled @ 3500 + 3500: Event_71: generic event scheduled @ 3500+
which we immediately guess means that there is one event per tick when comparing to the ExecAll trace:
0: system.cpu A0 T0 : @asm_main_after_prologue : movz x0, #1, #0 : IntAlu : D=0x0000000000000001 flags=(IsInteger) + 500: system.cpu A0 T0 : @asm_main_after_prologue+4 : adr x1, #28 : IntAlu : D=0x0000000000400098 flags=(IsInteger) + 1000: system.cpu A0 T0 : @asm_main_after_prologue+8 : ldr w2, #4194464 : MemRead : D=0x0000000000000006 A=0x4000a0 flags=(IsInteger|IsMemRef|IsLoad) + 1500: system.cpu A0 T0 : @asm_main_after_prologue+12 : movz x8, #64, #0 : IntAlu : D=0x0000000000000040 flags=(IsInteger) + 2000: system.cpu A0 T0 : @asm_main_after_prologue+16 : svc #0x0 : IntAlu : flags=(IsSerializeAfter|IsNonSpeculative|IsSyscall) +hello + 2500: system.cpu A0 T0 : @asm_main_after_prologue+20 : movz x0, #0, #0 : IntAlu : D=0x0000000000000000 flags=(IsInteger) + 3000: system.cpu A0 T0 : @asm_main_after_prologue+24 : movz x8, #93, #0 : IntAlu : D=0x000000000000005d flags=(IsInteger) + 3500: system.cpu A0 T0 : @asm_main_after_prologue+28 : svc #0x0 : IntAlu : flags=(IsSerializeAfter|IsNonSpeculative|IsSyscall) +Exiting @ tick 3500 because exiting with last active thread context+
On the event trace, we can see:
+AtomicSimpleCPU tick.wrapped_function_event: EventFunctionWrapped event scheduled @ 0 schedules a tick event for time 0, and this leads to the first clock tick
0: Event_70: generic event scheduled @ 0: schedules the end of time event for time 0, which is later rescheduled to the actual end of time: 0: Event_70: generic event rescheduled @ 18446744073709551615
at 0: AtomicSimpleCPU tick.wrapped_function_event: EventFunctionWrapped event rescheduled @ 500 the first clock tick must have finished running, and so to represent the next one, it was simply rescheduled for the next time 500! This is done at the end of AtomicSimpleCPU::tick():
if (_status != Idle) + reschedule(tickEvent, curTick() + latency, true);+
at 3500: Event_71: generic event scheduled @ 3500 the exit system call is called and the simulation ends.
A new event is scheduled for the current time itself. TODO what is this event?
+Let’s study the first event. From GDB, it’s stack trace is:
+Trace::OstreamLogger::logMessage() at trace.cc:149 0x5555593b3b1e +void Trace::Logger::dprintf_flag<char const*, char const*, unsigned long>() at 0x55555949e603 +void Trace::Logger::dprintf<char const*, char const*, unsigned long>() at 0x55555949de58 +Event::trace() at eventq.cc:395 0x55555946d109 +EventQueue::schedule() at eventq_impl.hh:65 0x555557195441 +EventManager::schedule() at eventq.hh:746 0x555557194aa2 +AtomicSimpleCPU::activateContext() at atomic.cc:239 0x555559075531 +SimpleThread::activate() at simple_thread.cc:177 0x555559545a63 +Process::initState() at process.cc:283 0x555559484011 +ArmProcess64::initState() at process.cc:126 0x55555730827a +ArmLinuxProcess64::initState() at process.cc:1,777 0x5555572d5e5e+
The interesting call is at AtomicSimpleCPU::activateContext:
schedule(tickEvent, clockEdge(Cycles(0)));+
which calls EventManager::schedule.
AtomicSimpleCPU is an EventManager because SimObject inherits from it.
tickEvent is an EventFunctionWrapper which contains a std::function<void(void)> callback;, and is initialized in the constructor as:
tickEvent([this]{ tick(); }, "AtomicSimpleCPU tick",
+ false, Event::CPU_Tick_Pri),
+So that’s how the main atomic tick loop works, fully understood!
+The second event has backtrace:
+Trace::OstreamLogger::logMessage() at trace.cc:149 0x5555593b3b1e +void Trace::Logger::dprintf_flag<char const*, char const*, unsigned long>() at 0x55555949e603 +void Trace::Logger::dprintf<char const*, char const*, unsigned long>() at 0x55555949de58 +Event::trace() at eventq.cc:395 0x55555946d109 +EventQueue::schedule() at eventq_impl.hh:65 0x555557195441 +BaseGlobalEvent::schedule() at global_event.cc:78 0x55555946d6f1 +GlobalEvent::GlobalEvent() at 0x55555949d177 +GlobalSimLoopExitEvent::GlobalSimLoopExitEvent() at sim_events.cc:61 0x555559474470 +simulate() at simulate.cc:104 0x555559476d6f+
so gets scheduled automatically at object creation simulate() through the GlobalEvent() constructor:
simulate_limit_event = + new GlobalSimLoopExitEvent(mainEventQueue[0]->getCurTick(), + "simulate() limit reached", 0);+
This event indicates that the simulation should finish by overriding bool isExitEvent() which gets checked in the main simulation at EventQueue::serviceOne:
if (event->isExitEvent()) {
+ assert(!event->flags.isSet(Event::Managed) ||
+ !event->flags.isSet(Event::IsMainQueue)); // would be silly
+ return event;
+And at long, we can guess without reading the code that Event_71 is comes from the SE implementation of the exit syscall, so let’s just confirm, the trace contains:
exitSimLoop() at sim_events.cc:97 0x5555594746e0 +exitImpl() at syscall_emul.cc:215 0x55555948c046 +exitFunc() at syscall_emul.cc:225 0x55555948c147 +SyscallDesc::doSyscall() at syscall_desc.cc:72 0x5555594949b6 +Process::syscall() at process.cc:401 0x555559484717 +SimpleThread::syscall() at 0x555559558059 +ArmISA::SupervisorCall::invoke() at faults.cc:856 0x5555572950d7 +BaseSimpleCPU::advancePC() at base.cc:681 0x555559083133 +AtomicSimpleCPU::tick() at atomic.cc:757 0x55555907834c+
and exitSimLoop() does:
new GlobalSimLoopExitEvent(when + simQuantum, message, exit_code, repeat);+
Tested at b4879ae5b0b6644e6836b0881e4da05c64a6550d.
+The events for the Atomic CPU were pretty simple: basically just ticks.
+But as we venture into more complex CPU models such as MinorCPU, the events get much more complex and interesting.
TODO: analyze the trace for:
+./run \ + --arch aarch64 \ + --emulator gem5 \ + --userland userland/arch/aarch64/freestanding/linux/hello.S \ + --trace Event \ + --trace-stdout \ + -- \ + --cpu-type MinorCPU \ + --caches \ +;+
This describes the internals of the gem5 m5out/stats.txt file.
+GDB call stack to dumpstats:
Stats::pythonDump () at build/ARM/python/pybind11/stats.cc:58 +Stats::StatEvent::process() () +GlobalEvent::BarrierEvent::process (this=0x555559fa6a80) at build/ARM/sim/global_event.cc:131 +EventQueue::serviceOne (this=this@entry=0x555558c36080) at build/ARM/sim/eventq.cc:228 +doSimLoop (eventq=0x555558c36080) at build/ARM/sim/simulate.cc:219 +simulate (num_cycles=<optimized out>) at build/ARM/sim/simulate.cc:132+
Stats::pythonDump does:
void
+pythonDump()
+{
+ py::module m = py::module::import("m5.stats");
+ m.attr("dump")();
+}
+This calls src/python/m5/stats/init.py in def dump does the main dumping
That function does notably:
+for output in outputList: + if output.valid(): + output.begin() + for stat in stats_list: + stat.visit(output) + output.end()+
begin and end are defined in C++ and output the header and tail respectively
void
+Text::begin()
+{
+ ccprintf(*stream, "\n---------- Begin Simulation Statistics ----------\n");
+}
+
+void
+Text::end()
+{
+ ccprintf(*stream, "\n---------- End Simulation Statistics ----------\n");
+ stream->flush();
+}
+stats_list contains the stats, and stat.visit prints them, outputList contains by default just the text output. I don’t see any other types of output in gem5, but likely JSON / binary formats could be envisioned.
Tested in gem5 b4879ae5b0b6644e6836b0881e4da05c64a6550d.
+E.g. src/cpu/decode_cache.hh includes:
#include "arch/isa_traits.hh"+
which in turn is meant to refer to files of form:
+src/arch/<isa>/isa_traits.hh+
What happens is that the build system creates a file:
+build/ARM/arch/isa_traits.hh+
which contains just:
+#include "arch/arm/isa_traits.hh"+
and puts that in the -I include path during build.
It appears to be possible to deal with it using preprocessor macros, but it is ugly: https://stackoverflow.com/questions/3178946/using-define-to-include-another-file-in-c-c/3179218#3179218
+In addition to the header polymorphism, gem5 also namespaces classes with TheISA::, e.g. in src/cpu/decode_cache.hh:
Value items[TheISA::PageBytes];+
which is defined at:
+… +build/ARM/config/the_isa.hh +…
+as:
+#define TheISA ArmISA+
and forces already arm/ specific headers to define their symbols under:
namespace ArmISA+
so I don’t see the point of this pattern, why not just us PageBytes directly? Looks like a documentation mechanism to indicate that a certain symbol is ISA specific.
Tested in gem5 2a242c5f59a54bc6b8953f82486f7e6fe0aa9b3d.
+Some scons madness.
+https://scons.org/doc/2.4.1/HTML/scons-user.html#idp1378838508 generates hard links by default.
+Then the a5bc2291391b0497fdc60fdc960e07bcecebfb8f SConstruct use symlinks in a futile attempt to make things better for editors or build systems from the past century.
+It was not possible to disable the symlinks automatically for the entire project when I last asked: https://stackoverflow.com/questions/53656787/how-to-set-disable-duplicate-0-for-all-scons-build-variants-without-repeating-th
+The clean is necessary because the source files didn’t change, so make would just check the timestamps and not build anything.
You will then likely want to make those more permanent as explained at: Section 30.4, “Default command line arguments”.
+You will then likely want to make those more permanent as explained at: Section 31.4, “Default command line arguments”.
If none of those methods are flexible enough for you, you can just fork or hack up buildroot_packages/sample_package the sample package to do what you want.
For how to use that package, see: Section 30.12.2, “buildroot_packages directory”.
+For how to use that package, see: Section 31.12.2, “buildroot_packages directory”.
Then iterate trying to do what you want and reading the manual until it works: https://buildroot.org/downloads/manual/manual.html
@@ -20934,7 +21606,7 @@ git -C "$(./getvar qemu_source_dir)" checkout -Then, you will also want to do a Bisection to pinpoint the exact commit to blame, and CC that developer.
Finally, give the images you used save upstream developers' time as shown at: Section 30.17.2, “release-zip”.
+Finally, give the images you used save upstream developers' time as shown at: Section 31.17.2, “release-zip”.
For Buildroot problems, you should wither provide the config you have:
@@ -21147,6 +21819,9 @@ git -C "$(./getvar qemu_source_dir)" checkout -userland/linux/open_o_tmpfile.c: https://stackoverflow.com/questions/4508998/what-is-an-anonymous-inode-in-linux/44388030#44388030
+fstream
+ +random
+In baremetal, we detect if tests failed by parsing logs for the Magic failure string.
See: Section 30.13, “Test this repo” for more useful testing tips.
+See: Section 31.13, “Test this repo” for more useful testing tips.
Xephyr is an RTOS that has POSIX support. I think it works much like our Baremetal setup which uses Newlib and generates individual ELF files that contain both our C program’s code, and the Xephyr libraries.
+TODO get a hello world working, and then consider further integration in this repo, e.g. being able to run all C userland content on it.
+TODO: Cortex-A CPUs are not currently supported, there are some qemu_cortex_m0 boards, but can’t find a QEMU Cortex-A. There is an x86_64 qemu board, but we don’t currently have an x86 baremetal toolchain. For this reason, we won’t touch this further for now.
However, unlike Newlib, Xephyr must be setting up a simple pre-main runtime to be able to handle threads.
+Failed attempt:
+# https://askubuntu.com/questions/952429/is-there-a-good-ppa-for-cmake-backports +wget -O - https://apt.kitware.com/keys/kitware-archive-latest.asc 2>/dev/null | sudo apt-key add - +sudo apt-add-repository 'deb https://apt.kitware.com/ubuntu/ bionic-rc main' +sudo apt-get update +sudo apt-get install cmake +git clone https://github.com/zephyrproject-rtos/zephyr +pip3 install --user -U west packaging +cd zephyr +git checkout v1.14.1 +west init zephyrproject +west update +export ZEPHYR_TOOLCHAIN_VARIANT=xtools +export XTOOLS_TOOLCHAIN_PATH="$(pwd)/out/crosstool-ng/build/default/install/aarch64/bin/" +source zephyr-env.sh +west build -b qemu_aarch64 samples/hello_world+
The build system of that project is a bit excessive / wonky. You need an edge CMake not present in Ubuntu 18.04, which I don’t want to install right now, and it uses the weird custom west build tool frontend.
The host requirements depend a lot on which examples you want to run.
If ./build --download-dependencies fails with:
It does not work if you just download the .zip with the sources for this repository from GitHub because we use Git submodules, you must clone this repo.
If you just want to run a command after boot ends without thinking much about it, just use the --eval-after option, e.g.:
It gets annoying to retype --arch aarch64 for every single command, or to remember --config setups.
To learn how to build the documentation see: Section 1.8, “Build the documentation”.
When running build-doc, we do the following checks:
The scripts prints what you have to fix and exits with an error status if there are any errors.
Documentation for asciidoctor/extract-link-targets
Documentation for asciidoctor/extract-header-ids
The Asciidoctor extension scripts:
As mentioned before the TOC, we have to push this README to GitHub pages due to: https://github.com/isaacs/github/issues/1610
You did something crazy, and nothing seems to work anymore?
ccache might save you a lot of re-build when you decide to Clean the build or create a new build variant.
It is not possible to rebuild the root filesystem while running QEMU because QEMU holds the file qcow2 file:
When doing long simulations sweeping across multiple system parameters, it becomes fundamental to do multiple simulations in parallel.
To run multiple gem5 checkouts, see: Section 30.11.3.1, “gem5 worktree”.
+To run multiple gem5 checkouts, see: Section 31.11.3.1, “gem5 worktree”.
Implementation note: we create multiple namespaces for two things:
@@ -30022,7 +30762,7 @@ less "$(./getvar --arch aarch64 --emulator gem5 --run-id 1 termout_file)"It often happens that you are comparing two versions of the build, a good and a bad one, and trying to figure out why the bad one is bad.
Our build variants system allows you to keep multiple built versions of all major components, so that you can easily switching between running one or the other.
If you want to keep two builds around, one for the latest Linux version, and the other for Linux v4.16:
To run both kernels simultaneously, one on each QEMU instance, see: Section 30.10, “Simultaneous runs”.
+To run both kernels simultaneously, one on each QEMU instance, see: Section 31.10, “Simultaneous runs”.
Analogous to the Linux kernel build variants but with the --qemu-build-id option instead:
Analogous to the Linux kernel build variants but with the --gem5-build-id option instead:
Therefore, you can’t forget to checkout to the sources to that of the corresponding build before running, unless you explicitly tell gem5 to use a non-default source tree with gem5 worktree. This becomes inevitable when you want to launch multiple simultaneous runs at different checkouts.
--gem5-build-id goes a long way, but if you want to seamlessly switch between two gem5 tress without checking out multiple times, then --gem5-worktree is for you.
Suppose that you are working on a private fork of gem5, but you want to use this repository to develop it as well.
Allows you to have multiple versions of the GCC toolchain or root filesystem.
lkmc/ contains sources and headers that are shared across kernel modules, userland and baremetal examples.
Another option would have been to name it as includes/lkmc, but that would make paths longer, and we might want to store source code in that directory as well in the future.
When factoring out functionality across userland examples, there are two main options:
Source: buildroot_packages/
A custom build script can give you more flexibility: e.g. the package can be made work with other root filesystems more easily, have better 9P support, and rebuild faster as it evades some Buildroot boilerplate.
Has the following structure:
Patches in this directory are never applied automatically: it is up to users to manually apply them before usage following the instructions in this documentation.
We use this directory for:
The files:
Print out several parameters that normally change randomly from boot to boot:
lkmc_home refers to the target base directory in which we put all our custom built stuff, such as userland executables and kernel modules.
Run almost all tests:
test does not all possible tests, because there are too many possible variations and that would take forever. The rationale is the same as for ./build all and is explained in ./build --help.
You can select multiple archs and emulators of interest, as for an other command, with:
By default, continue running even after the first failure happens, and they show a summary at the end.
TODO: we really need a mechanism to automatically generate the test list automatically e.g. based on path_properties, currently there are many tests missing, and we have to add everything manually which is very annoying.
We have some pexpect automated tests for GDB for both userland and baremetal programs!
We do not know of any way to set the emulator exit status in QEMU arm full system.
For the Linux kernel, do the following manual tests for now.
You should also test that the Internet works:
build-userland and test-executables have a wide variety of target selection modes, and it was hard to keep them all working without some tests:
When updating the Linux kernel, QEMU and gem5, things sometimes break.
In order to build and run each userland and baremetal example properly, we need per-file metadata such as compiler flags and required number of cores.
This is a template update procedure for submodules for which we have some patches on on top of mainline.
Ensure that the Automated tests are passing on a clean build:
The ./build-test command builds a superset of what will be downloaded which also tests other things we would like to be working on the release. For the minimal build to generate the files to be uploaded, see: Section 30.17.2, “release-zip”
The ./build-test command builds a superset of what will be downloaded which also tests other things we would like to be working on the release. For the minimal build to generate the files to be uploaded, see: Section 31.17.2, “release-zip”
The clean build is necessary as it generates clean images since it is not possible to remove Buildroot packages
@@ -31158,7 +31898,7 @@ git push --follow-tagsCreate a zip containing all files required for Prebuilt setup:
After:
This project was created to help me understand, modify and test low level system components by using system simulators.
The trade-offs between the different setups are basically a balance between:
compatibility: how likely is is that all the components will work well together: emulator, compiler, kernel, standard library, …
guest software availability: how wide is your choice of easily installed guest software packages? See also: Section 30.18.4, “Linux distro choice”
+guest software availability: how wide is your choice of easily installed guest software packages? See also: Section 31.18.4, “Linux distro choice”
Choosing which features go into our default builds means making tradeoffs, here are our guidelines:
We haven’t found the ultimate distro yet, here is a summary table of trade-offs that we care about: Table 7, “Comparison of Linux distros for usage in this repository”.
@@ -31527,7 +32267,7 @@ git push --follow-tags
Being the hardcore person who fully understands an important complex system such as a computer, it does have a nice ring to it doesn’t it?