baremetal: allow arbitrary exit status with the magic string

test-baremetal: fix missing setting x0 return value

Examples were just returning on ret without setting x0, which led to
failures... those were not noticed because of how broken the testing system
was ;-)
This commit is contained in:
Ciro Santilli 六四事件 法轮功
2019-05-06 00:00:01 +00:00
parent ff8cbe9d7a
commit 26cab92bfc
20 changed files with 133 additions and 77 deletions

View File

@@ -3524,7 +3524,7 @@ To stop at the very first instruction of a freestanding program, just use `--no-
=== User mode tests
Automatically run userland tests that can be run in user mode simulation, and check that they exit with status 0:
Automatically run all userland tests that can be run in user mode simulation, and check that they exit with status 0:
....
./build --all-archs test-user-mode
@@ -12608,14 +12608,25 @@ We then found out that QEMU starts in EL1, and so we kept just the EL1 part, and
=== Baremetal tests
Automatically run non-interactive baremetal tests:
Automatically run all non-interactive baremetal tests:
....
./test-baremetal
./test-baremetal --arch aarch64
....
Source: link:test-baremetal[]
Analogously to <<user-mode-tests>>, we can select individual tests or directories with:
....
./test-baremetal --arch aarch64 baremetal/hello.c baremetal/arch/aarch64/no_bootloader/
....
which would run all of:
* link:baremetal/hello.c[]
* all tests under the directory: link:baremetal/arch/aarch64/no_bootloader/[]
We detect if tests failed by parsing logs for the <<magic-failure-string>>.
We also skip tests that cannot work on certain conditions based on their basenames, e.g.:
@@ -14003,19 +14014,27 @@ To debug GDB problems on gem5, you might want to enable the following <<gem5-tra
===== Magic failure string
Since there is no standardized exit status concept that works across all emulators for full system, we just parse the terminal output for a magic failure string to check if tests failed.
We do not know of any way to set the emulator exit status in QEMU arm full system.
If a full system simulation outputs a line containing only exactly the magic string:
For other arch / emulator combinations, we know how to do it:
* aarch64: aarch64 semihosting supports exit status
* gem5: <<m5-fail>> works on all archs
* user mode: QEMU forwards exit status, gem5 we do some log parsing: <<gem5-syscall-emulation-exit-status>>
For this reason, we just parse the terminal output for a magic failure string to check if tests failed.
In order to cover all archs, our run scripts parse the serial output looking for a line line containing only exactly the magic regular expression:
....
lkmc_test_fail
lkmc_exit_status_(\d+)
....
to the terminal, then our run scripts detect that and exit with status `1`.
and then exit with the given regular expression.
This magic output string is notably used by:
This magic output string is notably generated by:
* the `lkmc_assert_fail()` function, which is used by <<baremetal-tests>>
* the `exit()` baremetal function when `status != 1`. This is in turn called by failed assertions for example from `lkmc_assert_fail`, which is used by <<baremetal-tests>>
* link:rootfs_overlay/lkmc/test_fail.sh[], which is used by <<test-userland-in-full-system>>
==== Non-automated tests