lkmc v2-rc

Unsquashed version at v2-rc-unsquashed, but that cannot be merged as it
breaks bisects at several points. All bugs will not bisect to this
humongous change.

It all started with a conversion of the Bash scripts to Python, mainly
because I couldn't stand not being able to properly use --options for
run which has a million options.

Then since that required a full testing, I decided to do all the
refactorings that I had in mind at once, and so I did and it became
v2-rc.

This is the largest patch I have ever done! OMG a few weeks of extra time.
I'm never writing a Bash script for anything that starts getting big again.

Some of the features are:

* separate build-qemu and build-gem5 commands
* common: convert scripts to python. Add --option for everything
* rename build to build-buildroot now that we are splitting all the build
  commands, Linux kernel to follow
* move all git submodules to submodules/ and all buildroot packages to
  packages/
* refactor the out/ structure. Keep projects on toplevel, because guest
  projects separate archs and host ones don't, making a toplevel arch wrong
* do-release: rename to just release
  https://stackoverflow.com/questions/16174992/cant-get-argparse-to-read-quoted-string-with-dashes-in-it
* run: add --terminal and explain gem5 pdb
* just track the lvimrc
* store CLI kernel config fragment inside buildlroot to avoid conflicts
* gem5: document m5 initparam
* readme: make a bunch of things awesomer
* readme: fix broken refs
* parsec-benchmark: update to 75d55ac446a43c47efb1044844a108c6c330184c
  Could not fetch otherwise.
* gem5: M5_OVERRIDE_PY_SOURCE

build-gem5

@@ -0,0 +1,83 @@
+#!/usr/bin/env python3
+
+import glob
+import multiprocessing
+import os
+import pathlib
+import shutil
+import subprocess
+
+import common
+
+parser = common.get_argparse()
+common.add_build_arguments(parser)
+parser.add_argument(
+    'extra_scons_args',
+    default=[],
+    metavar='extra-scons-args',
+    nargs='*'
+)
+args = common.setup(parser)
+binaries_dir = os.path.join(common.gem5_system_dir, 'binaries')
+disks_dir = os.path.join(common.gem5_system_dir, 'disks')
+if args.clean:
+    common.rmrf(common.gem5_build_dir)
+else:
+    os.makedirs(binaries_dir, exist_ok=True)
+    os.makedirs(disks_dir, exist_ok=True)
+    if  not os.path.exists(os.path.join(common.gem5_src_dir, '.git')):
+        subprocess.check_call([
+            'git',
+            '-C', common.gem5_default_src_dir,
+            'worktree', 'add',
+            '-b', os.path.join('wt', args.gem5_build_id),
+            common.gem5_src_dir
+        ])
+    if args.arch == 'x86_64':
+        dummy_img_path = os.path.join(disks_dir, 'linux-bigswap2.img')
+        with open(dummy_img_path, 'wb') as dummy_img_file:
+            zeroes = b'\x00' * (2 ** 16)
+            for i in range(2 ** 10):
+                dummy_img_file.write(zeroes)
+        subprocess.check_call(['mkswap', dummy_img_path])
+        with open(os.path.join(binaries_dir, 'x86_64-vmlinux-2.6.22.9'), 'w'):
+            # This file must always be present, despite --kernel overriding that default and selecting the kernel.
+            # I'm not even joking. No one has ever built x86 gem5 without the magic dist dir present.
+            pass
+    elif args.arch == 'arm' or args.arch == 'aarch64':
+        gem5_system_src_dir = os.path.join(common.gem5_src_dir, 'system')
+
+        # dtb
+        dt_src_dir = os.path.join(gem5_system_src_dir, 'arm', 'dt')
+        dt_build_dir = os.path.join(common.gem5_system_dir, 'arm', 'dt')
+        subprocess.check_call(['make', '-C', dt_src_dir])
+        os.makedirs(dt_build_dir, exist_ok=True)
+        for dt in glob.glob(os.path.join(dt_src_dir, '*.dtb')):
+            shutil.copy2(dt, dt_build_dir)
+
+        # Bootloader 32.
+        bootloader32_dir = os.path.join(gem5_system_src_dir, 'arm', 'simple_bootloader')
+        # TODO use the buildroot cross compiler here, and remove the dependencies from configure.
+        subprocess.check_call(['make', '-C', bootloader32_dir])
+        # bootloader
+        shutil.copy2(os.path.join(bootloader32_dir, 'boot_emm.arm'), binaries_dir)
+
+        # Bootloader 64.
+        bootloader64_dir = os.path.join(gem5_system_src_dir, 'arm', 'aarch64_bootloader')
+        # TODO cross_compile is ignored because the make does not use CC...
+        subprocess.check_call(['make', '-C', bootloader64_dir])
+        shutil.copy2(os.path.join(bootloader64_dir, 'boot_emm.arm64'), binaries_dir)
+    assert common.run_cmd([
+            'scons',
+            # TODO factor with build.
+            '-j', str(multiprocessing.cpu_count()),
+            '--ignore-style',
+            common.gem5_executable
+        ] +
+        args.extra_scons_args,
+        cwd=common.gem5_src_dir,
+        extra_env={'PATH': '/usr/lib/ccache:' + os.environ['PATH']},
+    ) == 0
+    term_src_dir = os.path.join(common.gem5_src_dir, 'util/term')
+    subprocess.check_call(['make', '-C', term_src_dir])
+    shutil.copy2(os.path.join(term_src_dir, 'm5term'), common.gem5_m5term)