# Linux Kernel Module Cheat

### 13.1. OverlayFS

OverlayFS is a filesystem merged in the Linux kernel in 3.18.

As the name suggests, OverlayFS allows you to merge multiple directories into one. The following minimal runnable examples should give you an intuition on how it works:

- https://askubuntu.com/questions/109413/how-do-i-use-overlayfs/1075564#1075564

- https://stackoverflow.com/questions/31044982/how-to-use-multiple-lower-layers-in-overlayfs/52792397#52792397

We are very interested in this filesystem because we are looking for a way to make host cross compiled executables appear on the guest root `/` without reboot.

This would have several advantages:

- makes it faster to test modified guest programs

  - not rebooting is fundamental for gem5, where the reboot is very costly.

  - no need to regenerate the root filesystem at all and reboot

  - overcomes the `check_bin_arch` problem as shown at: Section 26.8, “Buildroot rebuild is slow when the root filesystem is large”

- we could keep the base root filesystem very small, which implies:

  - less host disk usage, no need to copy the entire `./getvar out_rootfs_overlay_dir` to the image again

  - no need to worry about BR2_TARGET_ROOTFS_EXT2_SIZE

We can already make host files appear on the guest with 9P, but they appear on a subdirectory instead of the root.

If they would appear on the root instead, that would be even more awesome, because you would just use the exact same paths relative to the root transparently.

For example, we wouldn’t have to mess around with variables such as `PATH` and `LD_LIBRARY_PATH`.

The idea is to:

- 9P mount our overlay directory `./getvar out_rootfs_overlay_dir` on the guest, which we already do at `/mnt/9p/out_rootfs_overlay`

- then create an overlay with that directory and the root, and `chroot` into it.

  I was unable to mount directly to `/` avoid the `chroot`: https://stackoverflow.com/questions/41119656/how-can-i-overlayfs-the-root-filesystem-on-linux https://unix.stackexchange.com/questions/316018/how-to-use-overlayfs-to-protect-the-root-filesystem ** https://unix.stackexchange.com/questions/420646/mount-root-as-overlayfs

We already have a prototype of this running from `fstab` on guest at `/mnt/overlay`, but it has the following shortcomings:

- changes to underlying filesystems are not visible on the overlay unless you remount with `mount -r remount /mnt/overlay`, as mentioned on the kernel docs:

      Changes to the underlying filesystems while part of a mounted overlay
      filesystem are not allowed.  If the underlying filesystem is changed,
      the behavior of the overlay is undefined, though it will not result in
      a crash or deadlock.

  This makes everything very inconvenient if you are inside `chroot` action. You would have to leave `chroot`, remount, then come back.

- the overlay does not contain sub-filesystems, e.g. `/proc`. We would have to re-mount them. But should be doable with some automation.

Even more awesome than `chroot` would be to `pivot_root`, but I couldn’t get that working either:

- https://stackoverflow.com/questions/28015688/pivot-root-device-or-resource-busy

- https://unix.stackexchange.com/questions/179788/pivot-root-device-or-resource-busy