merge test-user-mode and test-baremetal into test-executables

Differentiate with --mode userland vs --mode baremetal.

The code was basically copied, now it's DRY.

README.adoc

@@ -1063,7 +1063,7 @@ The `build` scripts inside link:userland/[] are just symlinks to link:build-user
 
 So you can use any option supported by `build-userland` script freely with `build-userland-in-tree` and `build`.
 
-The situation is analogous for link:userland/test[], link:test-user-mode-in-tree[] and link:test-user-mode[], which are further documented at: <<user-mode-tests>>.
+The situation is analogous for link:userland/test[], link:test-executables-in-tree[] and link:test-executables[], which are further documented at: <<user-mode-tests>>.
 
 Do a more clean out-of-tree build instead and run the program:
 
@@ -1117,7 +1117,7 @@ This present the usual trade-offs of using prebuilts as mentioned at: <<prebuilt
 Other functionality are analogous, e.g. testing:
 
 ....
-./test-user-mode \
+./test-executables \
   --arch aarch64 \
   --gcc-which host \
   --qemu-which host \
@@ -1209,7 +1209,7 @@ The examples that work include most <<c,C examples>> that don't rely on complica
 The exact list of userland programs that work in baremetal is specified in <<path-properties>> with the `baremetal` property, but you can also easily find it out with a <<baremetal-tests,baremetal test dry run>>:
 
 ....
-./test-baremetal --arch aarch64 --dry-run
+./test-executables --arch aarch64 --dry-run --mode baremetal
 ....
 
 For example, we can run the C hello world link:userland/c/hello.c[] simply as:
@@ -1273,7 +1273,7 @@ echo $?
 You can run all the baremetal examples in one go and check that all assertions passed with:
 
 ....
-./test-baremetal --arch aarch64
+./test-executables --arch aarch64 --mode baremetal
 ....
 
 To use gem5 instead of QEMU do:
@@ -3603,18 +3603,18 @@ To stop at the very first instruction of a freestanding program, just use `--no-
 Automatically run all userland tests that can be run in user mode simulation, and check that they exit with status 0:
 
 ....
-./build --all-archs test-user-mode
+./build --all-archs test-executables-userland
-./test-user-mode --all-archs --all-emulators
+./test-executables --all-archs --all-emulators
 ....
 
 Or just for QEMU:
 
 ....
-./build --all-archs test-user-mode-qemu
+./build --all-archs test-executables-userland-qemu
-./test-user-mode --all-archs --emulator qemu
+./test-executables --all-archs --emulator qemu
 ....
 
-Source: link:test-user-mode[]
+Source: link:test-executables[]
 
 This script skips a manually configured list of tests, notably:
 
@@ -13898,15 +13898,20 @@ We then found out that QEMU <<arm-exception-levels,<starts in EL1>>, and so we k
 Automatically run all non-interactive baremetal tests:
 
 ....
-./test-baremetal --arch aarch64
+./test-executables --mode baremetal --arch aarch64
 ....
 
-Source: link:test-baremetal[]
+Source: link:test-executables[]
 
 Analogously to <<user-mode-tests>>, we can select individual tests or directories with:
 
 ....
-./test-baremetal --arch aarch64 userland/c/hello.c baremetal/arch/aarch64/no_bootloader/
+./test-executables \
+    --arch aarch64 \
+    --mode baremetal \
+    userland/c/hello.c \
+    baremetal/arch/aarch64/no_bootloader/ \
+;
 ....
 
 which would run all of:
@@ -15238,7 +15243,7 @@ link:test[] does not all possible tests, because there are too many possible var
 You can select multiple archs and emulators of interest, as for an other command, with:
 
 ....
-./test-user-mode \
+./test-executables \
   --arch x86_64 \
   --arch aarch64 \
   --emulator gem5 \
@@ -15249,7 +15254,7 @@ You can select multiple archs and emulators of interest, as for an other command
 You can also test all supported archs and emulators with:
 
 ....
-./test-user-mode \
+./test-executables \
   --all-archs \
   --all-emulators \
 ;
@@ -15261,16 +15266,14 @@ Without those flags, it defaults to just running the default arch and emulator o
 
 ===== Quit on fail
 
-By default, tests stop running as soon as the first failure happens.
+By default, continue running even after the first failure happens, and they show a summary at the end.
 
-You can prevent this with the `--no-quit-on-fail option, e.g.:
+You can make them exit immediately with the `--no-quit-on-fail` option, e.g.:
 
 ....
-./test-user-mode --no-quit-on-fail
+./test-executables --quit-on-fail
 ....
 
-You can then see which tests failed on the test summary report at the end.
-
 ===== Test userland in full system
 
 TODO: we really need a mechanism to automatically generate the test list automatically e.g. based on <<path-properties>>, currently there are many tests missing, and we have to add everything manually which is very annoying.
@@ -15430,10 +15433,10 @@ You should also test that the Internet works:
 
 ===== CLI script tests
 
-`build-userland` and `test-user-mode` have a wide variety of target selection modes, and it was hard to keep them all working without some tests:
+`build-userland` and `test-executables` have a wide variety of target selection modes, and it was hard to keep them all working without some tests:
 
 * link:test-build-userland[]
-* link:test-test-user-mode[]
+* link:test-test-executables[]
 
 === Bisection
 

build

@@ -321,15 +321,15 @@ so looping over all of them would waste time.
                     'userland',
                 ],
             ),
-            'test-user-mode': _Component(dependencies=[
+            'test-executables-userland': _Component(dependencies=[
-                'test-user-mode-qemu',
+                'test-executables-userland-qemu',
-                'test-user-mode-gem5',
+                'test-executables-userland-gem5',
             ]),
-            'test-user-mode-qemu': _Component(dependencies=[
+            'test-executables-userland-qemu': _Component(dependencies=[
                 'user-mode-qemu',
                 'userland',
             ]),
-            'test-user-mode-gem5': _Component(dependencies=[
+            'test-executables-userland-gem5': _Component(dependencies=[
                 'gem5',
                 'userland-gem5',
             ]),

build-test

@@ -12,4 +12,4 @@ while [ $# -gt 0 ]; do
   esac
 done
 ./build-test-boot --size "$test_size"
-./build --all-archs test-gdb test-user-mode
+./build --all-archs test-gdb test-executables-userland

common.py

@@ -260,10 +260,7 @@ If given, this differentiates between them.
             '--nproc',
             default=len(os.sched_getaffinity(0)),
             type=int,
-            help='''Number of processors to use for the action.
+            help='''Number of processors (Jobs) to use for the action.''',
-This is currently only implemented for the following scripts:
-all ./build-* scripts, test-user-mode.
-''',
         )
         self.add_argument(
             '-q',

path_properties.py

@@ -115,25 +115,39 @@ class PathProperties:
         env,
         link=False,
     ):
-        if len(self.path_components) > 1 and \
+        ext = os.path.splitext(self.path_components[-1])[1]
+        return (
+            not (
+                len(self.path_components) > 1 and \
                 self.path_components[1] == 'libs' and \
                 not env['package_all'] and \
-                not self.path_components[2] in env['package']:
+                not self.path_components[2] in env['package']
-            return False
+            ) and
-        return \
+            not self['no_build'] and
-            not self['no_build'] and \
             (
                 self['allowed_archs'] is None or
                 env['arch'] in self['allowed_archs']
-            ) and \
+            ) and
             (
-                (env['mode'] == 'userland'  and self['userland'] ) or
+                (
-                (env['mode'] == 'baremetal' and self['baremetal'])
+                    env['mode'] == 'userland' and
-            ) and \
+                    (
+                        self['userland'] and
+                        ext in env['build_in_exts']
+                    )
+                ) or
+                (
+                    env['mode'] == 'baremetal' and (
+                        self['baremetal'] and
+                        ext in env['baremetal_build_in_exts']
+                    )
+                )
+            ) and
             not (
                 link and
                 self['no_executable']
             )
+        )
 
     def should_be_tested(self, env):
         return (

test

@@ -30,8 +30,8 @@ Size of the tests to run. Scale:
         test_boot_args['size'] = self.env['size']
         self.run_test(lkmc.import_path.import_path_main('test-boot'), test_boot_args, 'test-boot')
         self.run_test(lkmc.import_path.import_path_main('test-userland-full-system'), run_args, 'test-userland')
-        self.run_test(lkmc.import_path.import_path_main('test-baremetal'), run_args, 'test-baremetal')
+        self.run_test(lkmc.import_path.import_path_main('test-executables'), {**run_args, **{'mode': 'baremetal'}}, 'test-executables-baremetal')
-        self.run_test(lkmc.import_path.import_path_main('test-user-mode'), run_args, 'test-user-mode')
+        self.run_test(lkmc.import_path.import_path_main('test-executables'), run_args, 'test-executables-userland')
         self.run_test(lkmc.import_path.import_path_main('test-gdb'), run_args, 'test-gdb')
         if self.env['emulator'] == 'gem5':
             gem5_unit_test_args = run_args.copy()

test-baremetal

@@ -1,72 +0,0 @@
-#!/usr/bin/env python3
-
-import os
-import sys
-
-import common
-import lkmc.import_path
-import path_properties
-import thread_pool
-
-class Main(common.TestCliFunction):
-    def __init__(self):
-        super().__init__(
-            defaults={
-                'mode': 'baremetal'
-            },
-        )
-        self.add_argument(
-            'tests',
-            nargs='*',
-            help='''\
-If given, run only the given tests. Otherwise, run all tests.
-'''
-        )
-
-    def setup_one(self):
-        self.env['tests'] = self.resolve_targets(
-            [
-                self.env['baremetal_source_dir'],
-                self.env['userland_source_dir']
-            ],
-            self.env['tests']
-        )
-
-    def timed_main(self):
-        run_args = self.get_common_args()
-        rootdir_abs_len = len(self.env['root_dir'])
-        with thread_pool.ThreadPool(
-            self.run_test,
-            handle_output=self.handle_output_function,
-            nthreads=self.env['nproc'],
-            thread_id_arg='thread_id',
-            submit_raise_exit=self.env['quit_on_fail'],
-        ) as my_thread_pool:
-            for test in self.env['tests']:
-                for path, in_dirnames, in_filenames in self.sh.walk(test):
-                    path_abs = os.path.abspath(path)
-                    dirpath_relative_root = path_abs[rootdir_abs_len + 1:]
-                    for in_filename in in_filenames:
-                        if os.path.splitext(in_filename)[1] in self.env['baremetal_build_in_exts']:
-                            path_relative_root = os.path.join(dirpath_relative_root, in_filename)
-                            my_path_properties = path_properties.get(path_relative_root)
-                            if my_path_properties.should_be_tested(self.env):
-                                cur_run_args = run_args.copy()
-                                cur_run_args.update({
-                                    'baremetal': os.path.relpath(os.path.join(path_abs, in_filename), os.getcwd()),
-                                })
-                                cur_run_args.update(my_path_properties['test_run_args'])
-                                run_test_args = {
-                                    'expected_exit_status': my_path_properties['exit_status'],
-                                    'run_args': cur_run_args,
-                                    'run_obj': lkmc.import_path.import_path_main('run'),
-                                    'test_id': path_relative_root,
-                                }
-                                signal = my_path_properties['signal_received']
-                                if signal is not None:
-                                    run_test_args['expected_exit_status'] = 128 + signal.value
-                                my_thread_pool.submit(run_test_args)
-        return self._handle_thread_pool_errors(my_thread_pool)
-
-if __name__ == '__main__':
-    Main().cli()

test-executables

@@ -12,8 +12,12 @@ class Main(common.TestCliFunction):
     def __init__(self, *args, **kwargs):
         if not 'description' in kwargs:
             kwargs['description'] = '''\
-https://github.com/cirosantilli/linux-kernel-module-cheat#user-mode-tests
+Test userland executables in user mode, or baremetal executables in full system
-TODO: expose all userland relevant ./run args here as well somehow.
+depending on the value of the --mode option. See also:
+
+* https://github.com/cirosantilli/linux-kernel-module-cheat#user-mode-tests
+* https://github.com/cirosantilli/linux-kernel-module-cheat#baremetal-tests
+* https://github.com/cirosantilli/linux-kernel-module-cheat#userland-setup-getting-started-natively
 '''
         if not 'defaults' in kwargs:
             kwargs['defaults'] = {}
@@ -30,15 +34,17 @@ If given, run only the given tests. Otherwise, run all tests.
 
     def setup_one(self):
         self.env['tests'] = self.resolve_targets(
-            [self.env['userland_source_dir']],
+            [
+                self.env['baremetal_source_dir'],
+                self.env['userland_source_dir']
+            ],
             self.env['tests']
         )
 
     def timed_main(self):
         run_args = self.get_common_args()
-        if self.env['emulator'] == 'gem5':
+        if self.env['mode'] == 'userland' and self.env['emulator'] == 'gem5':
             run_args['userland_build_id'] = 'static'
-        had_failure = False
         rootdir_abs_len = len(self.env['root_dir'])
         with thread_pool.ThreadPool(
             self.run_test,
@@ -58,17 +64,23 @@ If given, run only the given tests. Otherwise, run all tests.
                             if my_path_properties.should_be_tested(self.env):
                                 cur_run_args = run_args.copy()
                                 cur_run_args.update({
-                                    'userland': os.path.relpath(os.path.join(path_abs, in_filename), os.getcwd()),
+                                    self.env['mode']: os.path.relpath(
+                                        os.path.join(path_abs, in_filename),
+                                        os.getcwd()
+                                    ),
                                 })
                                 cur_run_args.update(my_path_properties['test_run_args'])
                                 run_test_args = {
                                     'expected_exit_status': my_path_properties['exit_status'],
                                     'run_args': cur_run_args,
                                     'run_obj': lkmc.import_path.import_path_main('run'),
-                                    'test_id': path_relative_root,
+                                    'test_id': '{} {}'.format(self.env['mode'], path_relative_root),
                                 }
                                 signal = my_path_properties['signal_received']
                                 if signal is not None:
+                                    if self.env['mode'] == 'baremetal':
+                                        run_test_args['expected_exit_status'] = 128 + signal.value
+                                    elif self.env['mode'] == 'userland':
                                         # Python subprocess reports signals differently from Bash's 128 + signal rule.
                                         run_test_args['expected_exit_status'] = -signal.value
                                 my_thread_pool.submit(run_test_args)

test-executables-in-tree

@@ -2,7 +2,7 @@
 
 import lkmc.import_path
 
-test_user_mode = lkmc.import_path.import_path_relative_root('test-user-mode')
+test_user_mode = lkmc.import_path.import_path_relative_root('test-executables')
 
 class Main(test_user_mode.Main):
     def __init__(self):

test-test-executables

@@ -9,24 +9,24 @@ set -eux
 
 f="$(tempfile)"
 
-./test-user-mode | tee "$f"
+./test-executables | tee "$f"
 grep -E '^PASS .* userland/c/hello' "$f"
 grep -E '^PASS .* userland/posix/uname' "$f"
 
-./test-user-mode userland | tee "$f"
+./test-executables userland | tee "$f"
 grep -E '^PASS .* userland/c/hello' "$f"
 grep -E '^PASS .* userland/posix/uname' "$f"
 
-./test-user-mode userland/c | tee "$f"
+./test-executables userland/c | tee "$f"
 grep -E '^PASS .* userland/c/hello' "$f"
 ! grep -E '^PASS .* userland/posix/uname' "$f"
 
-./test-user-mode userland/c/hello.c | tee "$f"
+./test-executables userland/c/hello.c | tee "$f"
 grep -E '^PASS .* userland/c/hello' "$f"
 ! grep -E '^PASS .* userland/c/false' "$f"
 ! grep -E '^PASS .* userland/posix/uname' "$f"
 
-./test-user-mode-in-tree | tee "$f"
+./test-executables-in-tree | tee "$f"
 grep -E '^PASS .* userland/c/hello' "$f"
 grep -E '^PASS .* userland/posix/uname' "$f"
 

userland/test

@@ -1 +1 @@
-../test-user-mode-in-tree
+../test-executables-in-tree