userland: make --host awesome

README.adoc

@@ -1995,7 +1995,7 @@ And you are back in KDB. Now you can:
 
 And you will break whenever `__x64_sys_write` is hit.
 
-TODO: `bp __x64_sys_write` is failing with `illegal numeric value` as of 10dd9178c6dccf1964002cc9368a5aa83b345487. I think it worked before.
+TODO: `bp __x64_sys_write` is failing with `illegal numeric value` as of 10dd9178c6dccf1964002cc9368a5aa83b345487. I think it worked before, so needs bisection.
 
 The other KDB commands allow you to instruction steps, view memory, registers and some higher level kernel runtime data.
 
@@ -7703,7 +7703,9 @@ QEMU user mode completely bypasses the kernel that we've built: all it takes is
 
 This runs link:userland/print_argv.c[]. `--userland` path resolution is analogous to <<baremetal-setup-getting-started,that of `--baremetal`>>.
 
-QEMU user mode also supports dynamically linked executables.
+`./build-userland` is further documented at: <<userland-directory>>.
+
+As we've just seen, QEMU user mode supports dynamically linked executables.
 
 This requires point it to the root filesystem with the `-L` option so that it can find the dynamic linker and shared libraries.
 
@@ -11034,13 +11036,6 @@ They contain data structs and magic constant for kernel to userland communicatio
 
 Userland test programs.
 
-It is possible to build and run those examples directly on your host:
-
-....
-cd userland
-make
-....
-
 For usage in the guest, build with:
 
 ....
@@ -11057,12 +11052,48 @@ In order to place them in the root filesystem image itself, you must also run:
 ./build-buildroot
 ....
 
-To force rebuild all examples, pass the `-B` option to `make`:
+It is possible to build and run those examples directly on your host:
 
 ....
-./build-buildroot --make-args=-B
+cd userland
+make
+./hello.out
+make clean
 ....
 
+or more cleanly out of tree:
+
+....
+./build-userland --host --userland-build-id host
+$(./getvar --userland-build-id host userland_build_dir)/hello.out
+....
+
+Extra make flags may be passed as:
+
+....
+./build-userland --host --userland-build-id host-static --make-args="-B CFLAGS_EXTRA=-static"
+$(./getvar --userland-build-id host-static userland_build_dir)/hello.out
+....
+
+This for example would both force a rebuild due to `-B` and link statically due to `CFLAGS_EXTRA=-static`.
+
+TODO: OpenMP does not like `-static`:
+
+....
+/usr/lib/gcc/x86_64-linux-gnu/5/libgomp.a(target.o): In function `gomp_target_init':
+(.text+0xba): warning: Using 'dlopen' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
+....
+
+See: https://stackoverflow.com/questions/23869981/linking-openmp-statically-with-gcc
+
+It is also possible to build other architectures with the host toolchain for other archs than your host arch:
+
+....
+./build-userland --arch arm --host --userland-build-id host
+....
+
+You won't be able to run those executables directly, but this is interesting if you are playing around with <<qemu-user-mode>>.
+
 ==== packages directory
 
 Every directory inside it is a Buildroot package.
@@ -11109,18 +11140,26 @@ These are typically patches that don't contain fundamental functionality, so we
 
 ==== rootfs_overlay
 
-Copied into the target filesystem.
-
-We use it for:
+We use this directory for:
 
 * customized configuration files
 * userland module test scripts that don't need to be compiled.
 +
 C files for example need compilation, and must go through the regular package system, e.g. through link:kernel_modules/user[].
 
-This directory gets <<9p>> mounted on the guest at: `/mnt/9p/rootfs_overlay`
+This directory is copied into the target filesystem by link:copy-overlay[], which then it visible via <<9p>> on the guest at:
 
-Furthermore, link:copy-overlay[] copies it to our generated overlay, which appears on the guest at `/mnt/9p/out_rootfs_overlay`.
+....
+ls /mnt/9p/out_rootfs_overlay
+....
+
+Furthermore, since this directory does not require compilation, we also make it <<9p>> available to the guest directly even without `copy-overlay` at:
+
+....
+ls /mnt/9p/rootfs_overlay
+....
+
+This way you can just hack away the scripts and try them out immediately without any further operations.
 
 === Test this repo
 

baremetal/README.adoc

@@ -0,0 +1 @@
+https://github.com/cirosantilli/linux-kernel-module-cheat#baremetal-setup

build-modules

@@ -19,7 +19,10 @@ class ModulesComponent(common.Component):
             '--host',
             action='store_true',
             default=False,
-            help='Build the Linux kernel modules for the host instead of guest',
+            help='''\
+Build the Linux kernel modules for the host instead of guest.
+Use the host packaged cross toolchain.
+''',
         )
         parser.add_argument(
             'kernel_modules',

build-userland

@@ -17,6 +17,15 @@ class UserlandComponent(common.Component):
             help='''\
 Indicate that a given package is present in the root filesystem, which
 allows us to build examples that rely on it.
+''',
+        )
+        parser.add_argument(
+            '--host',
+            action='store_true',
+            default=False,
+            help='''\
+Build the userland programs for the host instead of guest.
+Use the host packaged cross toolchain.
 ''',
         )
         parser.add_argument(
@@ -38,7 +47,10 @@ has the OpenBLAS libraries and headers installed.
     def do_build(self, args):
         build_dir = self.get_build_dir(args)
         os.makedirs(build_dir, exist_ok=True)
-        allowed_toolchains = ['buildroot']
+        if args.host:
+            allowed_toolchains = ['host']
+        else:
+            allowed_toolchains = ['buildroot']
         cc = common.get_toolchain_tool('gcc', allowed_toolchains=allowed_toolchains)
         cxx = common.get_toolchain_tool('g++', allowed_toolchains=allowed_toolchains)
         common.run_cmd(

common.py

@@ -264,7 +264,9 @@ See the documentation for other values known to work.
     )
     parser.add_argument(
         '-M', '--gem5-build-id',
-        help='gem5 build ID. Allows you to keep multiple separate gem5 builds. Default: %(default)s'.format(default_build_id)
+        help='''\
+gem5 build ID. Allows you to keep multiple separate gem5 builds. Default: {}
+'''.format(default_build_id)
     )
     parser.add_argument(
         '-N', '--gem5-worktree',
@@ -298,7 +300,7 @@ Default: %(default)s
         help='''\
 Use prebuilt packaged host utilities as much as possible instead
 of the ones we built ourselves. Saves build time, but decreases
-the likelihood of compatibility.
+the likelihood of incompatibilities.
 '''
     )
     parser.add_argument(
@@ -318,7 +320,7 @@ Default: the run ID (-n) if that is an integer, otherwise 0.
         help='gem5 build type, most often used for "debug" builds. Default: %(default)s'
     )
     parser.add_argument(
-        '--userland-build-id', default=default_build_id
+        '--userland-build-id', default=None
     )
     parser.add_argument(
         '-v', '--verbose', default=False, action='store_true',
@@ -666,6 +668,11 @@ def setup(parser):
         gem5_build_id_given = False
     else:
         gem5_build_id_given = True
+    if args.userland_build_id is None:
+        args.userland_build_id = default_build_id
+        this_module.userland_build_id_given = False
+    else:
+        this_module.userland_build_id_given = True
     if args.gem5_worktree is not None and not gem5_build_id_given:
         args.gem5_build_id = args.gem5_worktree
     this_module.machine = args.machine

download-dependencies

@@ -93,7 +93,9 @@ cpio \
 expect \
 flex \
 gcc-aarch64-linux-gnu \
-gcc-arm-linux-gnueabi \
+gcc-arm-linux-gnueabihf \
+g++-aarch64-linux-gnueabi \
+g++-arm-linux-gnueabihf \
 git \
 libguestfs-tools \
 moreutils \
@@ -150,7 +152,7 @@ EOF
   # Building SDL for QEMU in Buildroot was rejected upstream because it adds many dependencies:
   # https://patchwork.ozlabs.org/patch/770684/
   # We are just using the host SDL for now, if it causes too much problems we might remove it.
-  # libsdl2-dev needs to be installed separatedly from sudo apt-get build-dep qemu
+  # libsdl2-dev needs to be installed separately from sudo apt-get build-dep qemu
   # because Ubuntu 16.04's QEMU uses SDL 1.
   $mysudo apt-get install $y \
     $pkgs \