diff --git a/.gitignore b/.gitignore index 75aa743..0de9d0e 100644 --- a/.gitignore +++ b/.gitignore @@ -8,6 +8,7 @@ *~ .tmp_versions /rootfs_overlay/etc/init.d/S99 +/rootfs_overlay/ignore.sh Module.symvers README.html modules.order diff --git a/.gitmodules b/.gitmodules index 8dd0a80..5da02ec 100644 --- a/.gitmodules +++ b/.gitmodules @@ -11,3 +11,6 @@ [submodule "gem5/gem5"] path = gem5/gem5 url = https://gem5.googlesource.com/public/gem5 +[submodule "parsec-benchmark/parsec-benchmark"] + path = parsec-benchmark/parsec-benchmark + url = https://github.com/cirosantilli/parsec-benchmark diff --git a/README.adoc b/README.adoc index a690bb0..45da849 100644 --- a/README.adoc +++ b/README.adoc @@ -23,10 +23,7 @@ cd linux-kernel-module-cheat ./configure && ./build && ./run .... -The first build will take a while (https://stackoverflow.com/questions/10833672/buildroot-environment-with-host-toolchain[GCC], Linux kernel), e.g.: - -* 2 hours on a mid end 2012 laptop -* 30 minutes on a high end 2017 desktop +The first configure will take a while (30 minutes to 2 hours) to clone and build, see <> for more details. If you don't want to wait, you could also try to compile the examples and run them on your host computer as explained on at <>, but as explained on that section, that is dangerous, limited, and will likely not work. @@ -76,29 +73,36 @@ git ls-files | grep modulename === Rebuild -If you make changes to the kernel modules or most configurations tracked on this repository, you can just use again: +After making changes to a package, you must explicitly tell it to be rebuilt. + +For example, you you modify the kernel modules, you must rebuild with: .... -./build -./run +./build -k .... -and the modified files will be rebuilt. - -If you change any package besides `kernel_module`, you must also request those packages to be reconfigured or rebuilt with extra targets, e.g.: +which is just an alias for: .... -./build -t linux-reconfigure -t host-qemu-reconfigure +./build -- kernel_module-reconfigure .... -Those aren't turned on by default because they take quite a few seconds. +where `kernel_module` is the name of out Buildroot package that contains the kernel modules. -Linux and QEMU rebuilds are so common that we have dedicated shortcut flags for them: +Other important targets are: + +.... +./build -- linux-reconfigure host-qemu-reconfigure +.... + +which are aliased respectively to: .... ./build -l -q .... +We don't rebuild by default because, even with `make` incremental rebuilds, the timestamp check takes a few annoying seconds. + === Clean the build You did something crazy, and nothing seems to work anymore? @@ -158,6 +162,8 @@ the disk image gets overwritten by a fresh filesystem and you lose all changes. Remember that if you forcibly turn QEMU off without `sync` or `poweroff` from inside the VM, e.g. by closing the QEMU window, disk changes may not be saved. +When booting from <> however without a disk, persistency is lost. + === Message control We use `printk` a lot, and it shows on the QEMU terminal by default. If that annoys you (e.g. you want to see stdout separately), do: @@ -240,13 +246,23 @@ Instead, you can either run them from a minimal init: ./run -e 'init=/eval.sh - lkmc_eval="insmod /hello.ko;/poweroff.out"' -n .... +or if the script is large, add it to a gitignored file that will go into the guest: + +.... +echo ' +insmod /hello.ko +/poweroff.out +' > rootfs_overlay/ignore.sh +./run -e 'init=/ignore.sh' -n +.... + or run them at the end of the BusyBox init, which does things like setting up networking: .... ./run -e '- lkmc_eval="insmod /hello.ko;wget -S google.com;poweroff.out;"' .... -or add them to a new `init.d` entry: +or add them to a new `init.d` entry to run at the end o the BusyBox init: .... cp rootfs_overlay/etc/init.d/S98 rootfs_overlay/etc/init.d/S99 @@ -369,6 +385,32 @@ Just make sure that you never click inside the QEMU window when doing that, othe You can still send key presses to QEMU however even without the mouse capture, just either click on the title bar, or alt tab to give it focus. +=== What command was actually run? + +When asking for help on upstream repositories outside of this repository, you will need to provide the commands that you are running in detail without referencing our scripts. + +For example, QEMU developers will only want to see the final QEMU command that you are running. + +We make that easy by building commands as strings, and then echoing them before evaling. + +So for example when you run: + +.... +./run -a arm +.... + +Stdout shows a line with the full command of type: + +.... +./buildroot/output.arm~/host/usr/bin/qemu-system-arm -m 128M -monitor telnet::45454,server,nowait -netdev user,hostfwd=tcp::45455-:45455,id=net0 -smp 1 -M versatilepb -append 'root=/dev/sda nokaslr norandmaps printk.devkmsg=on printk.time=y' -device rtl8139,netdev=net0 -dtb ./buildroot/output.arm~/images/versatile-pb.dtb -kernel ./buildroot/output.arm~/images/zImage -serial stdio -drive file='./buildroot/output.arm~/images/rootfs.ext2.qcow2,if=scsi,format=qcow2' +.... + +This line is also saved to a file for convenience: + +.... +cat ./run.log +.... + [[gdb]] == GDB step debugging @@ -819,7 +861,7 @@ continue This is of least reliable setup as there might be other processes that use the given virtual address. -== Architecture +== Architectures The portability of the kernel and toolchains is amazing: change an option and most things magically work on completely different hardware. @@ -968,7 +1010,7 @@ To disable networking, use: To restore it, run: .... -./build -t initscripts-reconfigure +./build -- initscripts-reconfigure .... === The init environment @@ -1031,6 +1073,32 @@ Kernel modules built from the Linux mainline tree with `CONFIG_SOME_MOD=m`, are modprobe dummy-irq .... +== KVM + +You can make QEMU or gem5 <> by passing enabling KVM with: + +.... +./run -K +.... + +but it was broken in gem5 with pending patches: https://www.mail-archive.com/gem5-users@gem5.org/msg15046.html + +KVM uses the link:https://en.wikipedia.org/wiki/Kernel-based_Virtual_Machine[KVM Linux kernel feature] of the host to run most instructions natively. + +We don't enable KVM by default because: + +* only works if the architecture of the guest equals that of the host. ++ +We have only tested / supported it on x86, but it is rumoured that QEMU and gem5 also have ARM KVM support if you are link:https://www.youtube.com/watch?v=8ItXpmLsINs[running an ARM desktop for some weird reason] :-) +* limits visibility, since more things are running natively: +** can't use GDB +** can't do instruction tracing +* kernel boots are already fast enough without `-enable-kvm` + +The main use case for `-enable-kvm` in this repository is to test if something that takes a long time to run is functionally correct. + +For example, when porting a benchmark to Buildroot, you can first use QEMU's KVM to test that benchmarks is producing the correct results, before analysing them more deeply in gem5, which runs much slower. + == X11 Only tested successfully in `x86_64`. @@ -1038,7 +1106,7 @@ Only tested successfully in `x86_64`. Build: .... -./build -x +./build -i buildroot_config_fragment_x11 ./run .... @@ -1248,7 +1316,7 @@ One obvious use case is having an encrypted root filesystem: you keep the initrd I think GRUB then knows read common disk formats, and then loads that initrd to memory with a `/boot/grub/grub.cfg` directive of type: - initrd /initrd.img-4.4.0-108-generic + initrd /initrd.img-4.4.0-108-generic Related: https://stackoverflow.com/questions/6405083/initrd-and-booting-the-linux-kernel @@ -1485,12 +1553,14 @@ To get started, have a look at the "Hardware device drivers" secion under link:k == gem5 +=== gem5 getting started + gem5 is a system simulator, much <>: http://gem5.org/ For the most part, just add the `-g` option to the QEMU commands and everything should magically work: .... -./configure && ./build -a arm -g +./configure -gq && ./build -a arm -g ./run -a arm -g .... @@ -1555,7 +1625,7 @@ time ./run -a arm -e 'init=/poweroff.out' time ./run -a arm -e 'm5 exit' -g time ./run -a arm -e 'm5 exit' -g -- --caches --cpu-type=HPI time ./run -a x86_64 -e 'init=/poweroff.out' -time ./run -a x86_64 -e 'init=/poweroff.out' - -enable-kvm +time ./run -a x86_64 -e 'init=/poweroff.out' -- -enable-kvm time ./run -a x86_64 -e 'init=/poweroff.out' -g .... @@ -1572,12 +1642,7 @@ and the results were: |gem5 X86_64 |5 minutes 30 seconds| 82 |=== -on a Lenovo P51 laptop with: - -* Intel Core i7-7820HQ Processor (8MB Cache, up to 3.90GHz) (4 cores 8 threads) -* 32GB(16+16) DDR4 2400MHz SODIMM -* 512GB SSD PCIe TLC OPAL2 -* Ubuntu 17.10 +tested on the <>. === gem5 run benchmark @@ -1599,6 +1664,8 @@ It works like this: * the first commond boots linux with the default simplified `AtomicSimpleCPU`, and generates a <> after the kernel boots and before running the benchmark * the second command restores the checkpoint with the more detailed `HPI` CPU model, and runs the benchmark. We don't boot with it because that is much slower. +ARM employees have just been modifying benchmarking code with instrumentation directly: https://github.com/arm-university/arm-gem5-rsk/blob/aa3b51b175a0f3b6e75c9c856092ae0c8f2a7cdc/parsec_patches/xcompile-patch.diff#L230 + A few imperfections of our benchmarking method are: * when we do `m5 resetstats` and `m5 exit`, there is some time passed before the `exec` system call returns and the actual benchmark starts and ends @@ -1819,6 +1886,146 @@ External open source benchmarks. We will try to create Buildroot packages for th * http://parsec.cs.princeton.edu/ Mentioned on docs: http://gem5.org/PARSEC_benchmarks * http://www.m5sim.org/Splash_benchmarks +===== PARSEC benchmark + +We have ported parts of the link:http://parsec.cs.princeton.edu[PARSEC benchmark] for cross compilation at: https://github.com/cirosantilli/parsec-benchmark See the documentation on that repo to find out which benchmarks have been ported. + +This repo makes it trivial to get started with it: + +.... +configure -gpq && ./build -a arm -g -i buildroot_config_fragment_parsec +./run -a arm -g +.... + +Once inside the guest, we could in theory launch PARSEC exactly as you would launch it on the host: + +.... +cd /parsec/ +bash +. env.sh +parsecmgmt -a run -p splash2x.fmm -i test +.... + +TODO: `splash2x.barnes` segfaults on `arsecmgmt -a run -p splash2x.fmm -i simsmall` inside QEMU. Why? Other benchmarks ran fine. + +.... +[PARSEC] [---------- Beginning of output ----------] +Generating input file input_1... +Running /parsec/ext/splash2x/apps/barnes/inst/arm-linux.gcc/bin/barnes 1 < input_1: +reading input file : + +Segmentation fault +.... + +However, while this is fine inside QEMU, it is not practical in gem5, since the `parsecmgmt` Bash scripts just takes too long to run in that case! + +So instead, you must find out the raw executable command, and run it manually yourself. + +This command can be found from the `Running` line that `parsecmgmt` outputs when running the programs. + +"Luckily", we run the run scripts while creating the image to extract the inputs, so you can just do a find in your shell history to find the run command and find a line of type: + +.... +Running /parsec/ext/splash2x/apps/fmm/inst/arm-linux.gcc/bin/fmm 1 < input_1: +.... + +which teaches you that you can run `fmm` as: + +.... +cd /parsec/ext/splash2x/apps/fmm/run +../inst/arm-linux.gcc/bin/fmm 1 < input_1 +.... + +We are also collecting more raw commands for testing at: link:parsec-benchmark/test.sh[] + +And so inside of `gem5`, you likely want to do: + +.... +cd /parsec/ext/splash2x/apps/fmm/run +m5 checkpoint +m5 resetstats && /parsec/ext/splash2x/apps/fmm/inst/arm-linux.gcc/bin/fmm 1 < input_1 && m5 dumpstats +.... + +You will always want to `cd` into the `run` directory first, which is where the input is located. + +====== PARSEC change the input size + +One limitation is that only one input size is available on the guest for a given build. + +To change that, edit link:buildroot_config_fragment_parsec[] to contain for example: + +.... +BR2_PACKAGE_PARSEC_BENCHMARK_INPUT_SIZE=simsmall +.... + +and then rebuild with: + +.... +./build -a arm -g -i buildroot_config_fragment_parsec -- parsec-benchmark-reconfigure +.... + +This limitation exists because `parsecmgmt` generates the input files just before running via the Bash scripts, but we can't run `parsecmgmt` on gem5 as it is too slow! + +One option would be to do that inside the guest with QEMU, but this would required a full rebuild due to <>. + +Also, we can't generate all input sizes at once, because many of them have the same name and would overwrite one another... Parsec clearly needs a redesign for embedded, maybe we will patch it later. + +====== PARSEC uninstall + +If you want to remove PARSEC later, Buildroot doesn't provide an automated package removal mechanism as documented at: link:https://github.com/buildroot/buildroot/blob/2017.08/docs/manual/rebuilding-packages.txt#L90[], but the following procedure should be satisfactory: + +.... +rm -rf \ + ./buildroot/dl/parsec-* \ + ./buildroot/output.arm-gem5~/build/parsec-* \ + ./buildroot/output.arm-gem5~/build/packages-file-list.txt \ + ./buildroot/output.arm-gem5~/images/rootfs.* \ + ./buildroot/output.arm-gem5~/target/parsec-* \ +; +./build -a arm -g +.... + +====== PARSEC benchmark hacking + +If you end up going inside link:parsec-benchmark/parsec-benchmark[] to hack up the benchmark (you will!), these tips will be helpful. + +Buildroot was not designed to deal with large images, and currently cross rebuilds are a bit slow, due to some image generation and validation steps. + +A few workarounds are: + +* develop in host first as much as you can. Our PARSEC fork supports it. ++ +If you do this, don't forget to do a: ++ +.... +cd parsec-benchmark/parsec-benchmark +git clean -xdf . +.... +before going for the cross compile build. ++ +* patch Buildroot to work well, and keep cross compiling all the way. This should be totally viable, and we should do it. ++ +Don't forget to explicitly rebuild PARSEC with: ++ +.... +./build -a arm -g -i buildroot_config_fragment_parsec parsec-benchmark-reconfigure +.... ++ +You may also want to test if your patches are still functionally correct inside of QEMU first, which is a faster emulator. +* sell your soul, and compile natively inside the guest. We won't do this, not only because it is evil, but also because Buildroot explicitly does not support it: https://buildroot.org/downloads/manual/manual.html#faq-no-compiler-on-target ARM employees have been known to do this: https://github.com/arm-university/arm-gem5-rsk/blob/aa3b51b175a0f3b6e75c9c856092ae0c8f2a7cdc/parsec_patches/qemu-patch.diff + +TODO Buildroot is slow because of the `pkg-generic` `GLOBAL_INSTRUMENTATION_HOOKS` sanitation which go over the entire tree doing complex operations... I no like, in particular `check_bin_arch` and `check_host_rpath`. + +The pause is followed by: + +.... +buildroot/output.arm~/build/parsec-benchmark-custom/.stamp_target_installed +.... + +so which shows that the whole delay is inside our install itself. + +I put an `echo f` in `check_bin_arch`, and it just loops forever, does not stop on a particular package. + === gem5 kernel command line parameters Analogous <>: @@ -2050,7 +2257,7 @@ info: Entering event queue @ 0. Starting simulation... and the `telnet` at: .... -2017-12-28-11-59-51@ciro@ciro-p51$ ./gem5-shell +$ ./gem5-shell Trying 127.0.0.1... Connected to localhost. Escape character is '^]'. @@ -2337,7 +2544,13 @@ diff .config.olg .config Copy and paste the diff additions to `buildroot_config_fragment`. -==== What is making my build so slow? +==== Benchmarking this repo + +In this section document how fast the build and clone are, and how to investigate them. + +Send a pull request if you try it out on something significantly different. + +===== What is making my build so slow? .... cd buildroot/output.x86_64~ @@ -2350,6 +2563,41 @@ Our philosophy is: * if something adds little to the build time, build it in by default * otherwise, make it optional +The biggest time hog is always GCC, can we use a precompiled one? https://stackoverflow.com/questions/10833672/buildroot-environment-with-host-toolchain + +===== Benchmark machines + +The build times are calculated after doing link:https://buildroot.org/downloads/manual/manual.html#_offline_builds[`make source`], which downloads the sources, and basically benchmarks the Internet. + +====== P51 + +Build time at 2c12b21b304178a81c9912817b782ead0286d282: 28 minutes + +Lenovo ThinkPad link:https://www3.lenovo.com/gb/en/laptops/thinkpad/p-series/P51/p/22TP2WPWP51[P51 laptop]: + +* 2500 USD in 2018 (high end) +* Intel Core i7-7820HQ Processor (8MB Cache, up to 3.90GHz) (4 cores 8 threads) +* 32GB(16+16) DDR4 2400MHz SODIMM +* 512GB SSD PCIe TLC OPAL2 +* Ubuntu 17.10 + +====== T430 + +Build time: 2 hours. + +TODO specs, SHA. + +===== Benchmark internets + +====== 38Mbps + +2c12b21b304178a81c9912817b782ead0286d282: + +* shallow clone of all submodules: 4 minutes. +* `make source`: 2 minutes + +Google M-lab speed test: 36.4Mbps + === About This project is for people who want to learn and modify low level system components: diff --git a/build b/build index 13e9d0f..d22092e 100755 --- a/build +++ b/build @@ -1,48 +1,53 @@ #!/usr/bin/env bash -set -e +set -eu arch=x86_64 -extra_targets='' +configure=true +config_fragments=buildroot_config_fragment +extra_make_args='' gem5=false j="$(($(nproc) - 2))" post_script_args='' qemu_sdl='--enable-sdl --with-sdlabi=2.0' -x11=false v=0 -while getopts 'a:gj:lp:qSt:v' OPT; do +while getopts 'a:Cgj:i:klp:qS:v' OPT; do case "$OPT" in a) arch="$OPTARG" ;; + C) + configure=false + ;; g) gem5=true ;; j) j="$OPTARG" ;; + i) + config_fragments="$config_fragments $OPTARG" + ;; + k) + extra_make_args="$extra_make_args kernel_module-reconfigure" + ;; l) - extra_targets="$extra_args linux-reconfigure" + extra_make_args="$extra_make_args linux-reconfigure" ;; p) post_script_args="$OPTARG" ;; q) - extra_targets="$extra_args host-qemu-reconfigure" + extra_make_args="$extra_make_args host-qemu-reconfigure" ;; S) qemu_sdl='' ;; - t) - extra_targets="$extra_args $OPTARG" - ;; - x) - x11=true - ;; v) v=1 ;; esac done shift $(($OPTIND - 1)) +extra_make_args="$extra_make_args $@" case "$arch" in x86_64) defconfig=qemu_x86_64_defconfig @@ -62,60 +67,71 @@ if "$gem5"; then arch_dir="${arch}-gem5" # Networking was not working, so disable it to speed things up. post_script_args="$post_script_args -n" +else + config_fragments="$config_fragments buildroot_config_fragment_qemu" +fi +root_dir="$(pwd)" +buildroot_dir="${root_dir}/buildroot" +out_dir="${buildroot_dir}/output.${arch_dir}~" +config_file="${out_dir}/.config" + +if "$configure"; then + cd "${buildroot_dir}" + for p in $(find "${root_dir}/buildroot_patches/" -maxdepth 1 -name '*.patch' -print); do + patch -N -r - -p 1 <"$p" || : + done + make O="$out_dir" BR2_EXTERNAL="${root_dir}/kernel_module:${root_dir}/gem5:${root_dir}/parsec-benchmark" "$defconfig" + # TODO Can't get rid of these for now. + # http://stackoverflow.com/questions/44078245/is-it-possible-to-use-config-fragments-with-buildroots-config + for config_fragment in $config_fragments; do + cat "../$config_fragment" >> "${config_file}" + done + printf " +BR2_JLEVEL=$j +BR2_ROOTFS_POST_SCRIPT_ARGS=\"$post_script_args\" +" >> "${config_file}" + if "$gem5"; then + printf "\ +BR2_LINUX_KERNEL_CUSTOM_CONFIG_FILE=\"../kernel_config_${arch_dir}\" +BR2_PACKAGE_GEM5=y +" >> "${config_file}" + else + printf "\ +BR2_LINUX_KERNEL_CONFIG_FRAGMENT_FILES=\"../kernel_config_fragment\" +" >> "${config_file}" + fi + if [ "$arch" = 'mips64' ]; then + # Workaround for: + # http://lists.busybox.net/pipermail/buildroot/2017-August/201053.html + sed -Ei 's/^BR2_PACKAGE_LINUX_TOOLS_GPIO/BR2_PACKAGE_LINUX_TOOLS_GPIO=n/' "${config_file}" + fi + make O="$out_dir" olddefconfig fi -cd kernel_module +cd "${root_dir}/kernel_module" ./make-host.sh -j "$j" clean -cd ../buildroot -for p in $(find '../buildroot_patches/' -maxdepth 1 -name '*.patch' -print); do - patch -N -r - -p 1 <"$p" || : -done -outdir="output.${arch_dir}~" -make O="$outdir" BR2_EXTERNAL="$(pwd)/../kernel_module:$(pwd)/../gem5" "$defconfig" -# TODO Can't get rid of this for now. -# http://stackoverflow.com/questions/44078245/is-it-possible-to-use-config-fragments-with-buildroots-config -cat ../buildroot_config_fragment >> "${outdir}/.config" -if "$gem5"; then - echo "\ -BR2_LINUX_KERNEL_CUSTOM_CONFIG_FILE=\"../kernel_config_${arch_dir}\" -" >> "${outdir}/.config" -else - echo "\ -BR2_LINUX_KERNEL_CONFIG_FRAGMENT_FILES=\"../kernel_config_fragment\" -" >> "${outdir}/.config" -fi -if "$x11"; then - cat ../buildroot_config_fragment_x11 >> "${outdir}/.config" -fi -if ! "$gem5"; then - cat ../buildroot_config_fragment_qemu >> "${outdir}/.config" -fi -if [ "$arch" = 'mips64' ]; then - # Workaround for: - # http://lists.busybox.net/pipermail/buildroot/2017-August/201053.html - sed -Ei 's/^BR2_PACKAGE_LINUX_TOOLS_GPIO/BR2_PACKAGE_LINUX_TOOLS_GPIO=n/' "${outdir}/.config" -fi -make O="$outdir" olddefconfig +cd "${buildroot_dir}" # HOST_QEMU_OPTS is a hack that happens to work because the QEMU package luckly uses += at all times. # It shouldn't be necessary in the first place: https://bugs.busybox.net/show_bug.cgi?id=9936 # # Even if were an autotools package, there is no general way currently to pass extra configs to it: # https://stackoverflow.com/questions/44341188/how-to-pass-extra-custom-configure-autotools-options-to-a-buildroot-package/44341225#44341225 -time \ +# +# BR2_ options may be given on the command line here, and they do have direct "define" effects. +# But this is generally bad, as it skips the Kconfig mechanism, e.g. it does not set defaults properly. +cmd="time \ env \ -u LD_LIBRARY_PATH \ make \ - O="$outdir" \ - BR2_JLEVEL="$j" \ - BR2_PACKAGE_GEM5="$("$gem5" && echo y || echo n)" \ - BR2_ROOTFS_POST_SCRIPT_ARGS="$post_script_args" \ - HOST_QEMU_OPTS="--enable-debug --extra-cflags='-DDEBUG_PL061=1' --enable-trace-backends=simple $qemu_sdl" \ - V="$v" \ - kernel_module-rebuild \ - $extra_targets \ + O='$out_dir' \ + HOST_QEMU_OPTS='--enable-debug --extra-cflags=-DDEBUG_PL061=1 --enable-trace-backends=simple $qemu_sdl' \ + V='$v' \ + $extra_make_args \ all \ -; -cd .. +" +echo "$cmd" | tee "${root_dir}/build.log" +eval "$cmd" if "$gem5"; then + cd "${root_dir}" ./build-gem5 -a "$arch" fi diff --git a/buildroot_config_fragment b/buildroot_config_fragment index 1107574..67009ba 100644 --- a/buildroot_config_fragment +++ b/buildroot_config_fragment @@ -1,11 +1,13 @@ -BR2_ENABLE_LOCALE=y +# Custom packages +BR2_PACKAGE_KERNEL_MODULE=y + BR2_GCC_ENABLE_GRAPHITE=y BR2_GCC_ENABLE_LTO=y BR2_GCC_ENABLE_OPENMP=y BR2_GLOBAL_PATCH_DIR="../global_patch_dir" BR2_PACKAGE_BUSYBOX_CONFIG_FRAGMENT_FILES="../busybox_config_fragment" BR2_PACKAGE_DHRYSTONE=y -BR2_PACKAGE_KERNEL_MODULE=y +BR2_PACKAGE_FILE=y BR2_PACKAGE_OVERRIDE_FILE="../buildroot_override" # For qemu-ga on guest. TODO: do something with it, and document it. BR2_PACKAGE_QEMU=y diff --git a/buildroot_config_fragment_parsec b/buildroot_config_fragment_parsec new file mode 100644 index 0000000..896066e --- /dev/null +++ b/buildroot_config_fragment_parsec @@ -0,0 +1,8 @@ +BR2_PACKAGE_PARSEC_BENCHMARK=y +#BR2_PACKAGE_PARSEC_BENCHMARK_BUILD_LIST="splash2x" +#BR2_PACKAGE_PARSEC_BENCHMARK_INPUT_SIZE="simsmall" +# Because PARSEC + its data are huge. TODO: can't we automate calculating the size? +# Problems will arise if someone tries to use two such benchmarks. +# Cannot be selected automatically from Kconfig: +# https://stackoverflow.com/questions/40309054/how-to-select-the-value-of-a-string-option-from-another-option-in-kbuild-kconfig/49096538#49096538 +BR2_TARGET_ROOTFS_EXT2_SIZE="128M" diff --git a/buildroot_override b/buildroot_override index cf57077..2a8fdaf 100644 --- a/buildroot_override +++ b/buildroot_override @@ -1,3 +1,4 @@ HOST_QEMU_OVERRIDE_SRCDIR = ../qemu LINUX_OVERRIDE_SRCDIR = ../linux +PARSEC_BENCHMARK_OVERRIDE_SRCDIR = ../parsec-benchmark/parsec-benchmark QEMU_OVERRIDE_SRCDIR = ../qemu diff --git a/configure b/configure index 85a0e27..65c5816 100755 --- a/configure +++ b/configure @@ -1,9 +1,21 @@ #!/usr/bin/env bash set -e interactive_pkgs=libsdl2-dev +gem5=false +qemu=true +submodules='buildroot linux' y='' -while getopts t OPT; do +while getopts gqpt OPT; do case "$OPT" in + g) + gem5=true + ;; + q) + qemu=false + ;; + p) + submodules="$submodules parsec-benchmark/parsec-benchmark" + ;; t) interactive_pkgs='' y='-y' @@ -12,45 +24,56 @@ while getopts t OPT; do done shift $(($OPTIND - 1)) +## Submodules + +if "$qemu"; then + submodules="$submodules qemu" +fi +if "$gem5"; then + submodules="$submodules gem5/gem5" +fi ( set -e - # Shallow clonning saves a considerable ammount of time, specially because of the linux kernel. - # However, git submodules are buggy as usual, and this is the best way i've found to get it done: + # Shallow cloning saves a considerable amount of time, specially because of the linux kernel. + # However, git submodules are buggy as usual, and this is the best way I've found to get it done: # https://stackoverflow.com/questions/2144406/git-shallow-submodules/47374702#47374702 # In particular: # - `shallow = true` on the submodule has no effect for the non default educational branches of our submodules # - QEMU's submodules point to commits that are neither under branches nor tags, and so `--shallow-submodules` fails - git submodule update --depth 1 --jobs 4 --init - cd qemu - git submodule update --init + git submodule update --depth 1 --jobs 4 --init -- $submodules + if "$qemu"; then + cd qemu + git submodule update --init + fi ) & -wait $! || git submodule update --init +wait $! || git submodule update --init -- $submodules + +## apt-get pkgs="\ automake \ build-essential \ coreutils \ " - -# GEM5 -pkgs="$pkgs \ -g++-6 \ -gcc-6 \ -gcc-aarch64-linux-gnu \ -gcc-arm-linux-gnueabi \ -libgoogle-perftools-dev \ -protobuf-compiler \ -" - +if "$gem5"; then + pkgs="$pkgs \ + g++-6 \ + gcc-6 \ + gcc-aarch64-linux-gnu \ + gcc-arm-linux-gnueabi \ + libgoogle-perftools-dev \ + protobuf-compiler \ + " +fi command -v apt-get >/dev/null 2>&1 || { - cat < /proc/sys/kernel/randomize_va_space. # printk.time=y: log in format: "[time ] msg" for all printk messages. @@ -19,7 +20,7 @@ gem5=false gem5opts='' initrd=false root='' -while getopts a:c:Dde:G:giknt:x OPT; do +while getopts a:c:Dde:G:giKknt:x OPT; do case "$OPT" in a) arch="$OPTARG" @@ -36,6 +37,9 @@ while getopts a:c:Dde:G:giknt:x OPT; do e) extra_append="$extra_append $OPTARG" ;; + K) + kvm=true + ;; k) extra_append="$extra_append kgdbwait" # For those who want to try KDB. @@ -66,6 +70,9 @@ if $gem5; then outdir="$(pwd)/buildroot/output.${arch}-gem5~" gem5_dir="$(pwd)/gem5/gem5" if [ "$arch" = x86_64 ]; then + if "$kvm"; then + extra_flags="$extra_flags --cpu-type=X86KvmCPU" + fi cmd="\ M5_PATH='$(pwd)/gem5/gem5-system' \ '${gem5_dir}/build/X86/gem5.opt' \ @@ -95,6 +102,9 @@ $extra_flags \ fi else buildroot_out_dir="./buildroot/output.${arch}~" + if "$kvm"; then + extra_flags="$extra_flags -enable-kvm" + fi extra_flags="$extra_flags_qemu $extra_flags" images_dir="$buildroot_out_dir/images" qemu_common="\