diff --git a/README.adoc b/README.adoc index 6db3828..b928c43 100644 --- a/README.adoc +++ b/README.adoc @@ -1670,6 +1670,46 @@ Seems possible! Lets do it: * http://gem5.org/wiki/images/b/b8/Summit2017_wa_devlib.pdf * http://gem5.org/WA-gem5 +==== OverlayFS + +It would be uber awesome if we could overlay a 9p filesystem on top of the root. + +That would allow us to have a second Buildroot `target/` directory, and without any extra configs, keep the root filesystem image small, which implies: + +* less host disk usage, no need to copy the entire `target/` to the image again +* faster rebuild turnaround: +** no need to regenerate the root filesystem at all and reboot +** overcomes the `check_bin_arch` problem: <> +* no need to worry about <> + +But TODO we didn't get it working yet: + +* https://stackoverflow.com/questions/41119656/how-can-i-overlayfs-the-root-filesystem-on-linux +* https://unix.stackexchange.com/questions/316018/how-to-use-overlayfs-to-protect-the-root-filesystem + +Test with the script: + +.... +/overlayfs.sh +.... + +It shows that files from the `upper/` does not show on the root. + +Furthermore, if you try to mount the root elsewhere to prepare for a chroot: + +.... +/overlayfs.sh / /overlay +# chroot /overlay +.... + +it does not work well either because sub filesystems like `/proc` do not show on the mount: + +.... +ls /overlay/proc +.... + +A less good alternative is to set `LD_LIBRARY_PATH` on the 9p mount and run executables directly from the mount. + === Guest host networking ==== Host to guest @@ -2232,23 +2272,33 @@ The rebuild is required because some of the input sizes Separating input sizes also allows to create smaller images when only running the smaller benchmarks. -We don't have a perfect way to find the right value for `BR2_TARGET_ROOTFS_EXT2_SIZE`, one good heuristic is: +====== BR2_TARGET_ROOTFS_EXT2_SIZE -.... -du -hsx buildroot/output.arm-gem5~/target/parsec -.... - -Also dots cannot be used as in `1.5G`, so just use Megs as in `1500M` instead. - -If you don't set it high enough, you will get the message: +When adding new large package to the Buildroot root filesystem, it may fail with the message: .... Maybe you need to increase the filesystem size (BR2_TARGET_ROOTFS_EXT2_SIZE) .... +The solution is to simply add: + +.... +./build -c BR2_TARGET_ROOTFS_EXT2_SIZE="500M" +.... + +where 500M is "large enough". + +Note that dots cannot be used as in `1.5G`, so just use Megs as in `1500M` instead. + +Unfortunately, TODO we don't have a perfect way to find the right value for `BR2_TARGET_ROOTFS_EXT2_SIZE`. One good heuristic is: + +.... +du -hsx buildroot/output.arm-gem5~/target/parsec +.... + https://stackoverflow.com/questions/49211241/is-there-a-way-to-automatically-detect-the-minimum-required-br2-target-rootfs-ex -TODO: mount the benchmarks from host instead of installing them on guest. <<9p>> would be perfect for this, but we need to get it working on gem5 and arm first. +One way to overcome this problem is to mount benchmarks from host instead of adding them to the root filesystem, e.g. with: <<9p>> ====== PARSEC benchmark with parsecmgmt @@ -2353,7 +2403,16 @@ Don't forget to explicitly rebuild PARSEC with: You may also want to test if your patches are still functionally correct inside of QEMU first, which is a faster emulator. * sell your soul, and compile natively inside the guest. We won't do this, not only because it is evil, but also because Buildroot explicitly does not support it: https://buildroot.org/downloads/manual/manual.html#faq-no-compiler-on-target ARM employees have been known to do this: https://github.com/arm-university/arm-gem5-rsk/blob/aa3b51b175a0f3b6e75c9c856092ae0c8f2a7cdc/parsec_patches/qemu-patch.diff -TODO Buildroot is slow because of the `pkg-generic` `GLOBAL_INSTRUMENTATION_HOOKS` sanitation which go over the entire tree doing complex operations... I no like, in particular `check_bin_arch` and `check_host_rpath`. +[[rpath]] +====== Buildroot rebuild is slow when the root filesystem is large + +Buildroot is not designed for large root filesystem images, and the rebuild becomes very slow when we add a large package to it. + +This is due mainly to the `pkg-generic` `GLOBAL_INSTRUMENTATION_HOOKS` sanitation which go over the entire tree doing complex operations... I no like, in particular `check_bin_arch` and `check_host_rpath`, which get stuck for a long time on the message: + +.... +>>> Sanitizing RPATH in target tree +.... The pause is followed by: diff --git a/kernel_config_fragment b/kernel_config_fragment index c1b272a..f849b2e 100644 --- a/kernel_config_fragment +++ b/kernel_config_fragment @@ -1,8 +1,9 @@ # Changes to this file are automatically trigger kernel reconfigures # even without using the linux-reconfigure target. -CONFIG_MODULE_SRCVERSION_ALL=y CONFIG_BLK_DEV_INITRD=y +CONFIG_MODULE_SRCVERSION_ALL=y +CONFIG_OVERLAY_FS=y # GDB debugging. CONFIG_DEBUG_FS=y diff --git a/rootfs_overlay/overlayfs.sh b/rootfs_overlay/overlayfs.sh new file mode 100755 index 0000000..2608949 --- /dev/null +++ b/rootfs_overlay/overlayfs.sh @@ -0,0 +1,14 @@ +#!/bin/sh +set -ex +lower="${1:-/}" +mount="$2" +if [ -z "$mount" ]; then + mount="$lower" +fi +upper="${lower}/upper" +work="${lower}/work" +mkdir -p "$lower" "$upper" "$work" "$mount" +touch "${lower}/asdf" "${upper}/qwer" +mount -t overlay -o lowerdir="$lower",upperdir="$upper",workdir="$work" none "$mount" +ls "$lower" "$upper" +umount "$mount"