qemu: allow turning -O0 debug build on or off, use -O0 by default

OMG, I was wasting time in ARM all along!!!

README.adoc

@@ -9786,25 +9786,27 @@ This is of course trivial since they are just regular userland programs on the h
 ./run --debug-vm
 ....
 
-Then you could:
-
-....
-break edu_mmio_read
-run
-....
-
-And in QEMU:
-
-....
-./qemu_edu.sh
-....
-
 Or for a faster development loop:
 
 ....
-./run --debug-vm-args '-ex "break edu_mmio_read" -ex "run"'
+./run --debug-vm-args '-ex "break qemu_add_opts" -ex "run"'
 ....
 
+Our default emulator builds are optimized with `gcc -O2 -g`. To use `-O0` instead, build and run with:
+
+....
+./build-qemu --qemu-build-type debug --verbose
+./run --debug-vm
+./build-gem5 --gem5-build-type debug --verbose
+./run --debug-vm --emulator-gem5
+....
+
+The `--verbose` is optional, but shows clearly each GCC build command so that you can confirm what `--*-build-type` is doing.
+
+The build outputs are automatically stored in a different directories for optimized and debug builds, which prevents `debug` files from overwriting `opt` ones. Therefore, `--gem5-build-id` is not required: 
+
+The price to pay for debuggability is high however: a Linux kernel boot was about 3x slower in QEMU and 14 times slower in gem5 debug compared to opt, see benchmarks at: xref:benchmark-linux-kernel-boot[xrefstyle=full]
+
 When in <<qemu-text-mode>>, using `--debug-vm` makes Ctrl-C not get passed to the QEMU guest anymore: it is instead captured by GDB itself, so allow breaking. So e.g. you won't be able to easily quit from a guest program like:
 
 ....
@@ -9828,19 +9830,28 @@ echo 'kernel.perf_event_paranoid=1' | sudo tee -a /etc/sysctl.conf
 sudo sysctl -p
 ....
 
-Then use it as:
+Then use it with your content of interest, for example:
 
 ....
-./run --debug-vm-rr
+./run --debug-vm-rr --userland userland/c/hello.c
 ....
 
-Once you are left in the RR GDB shell, just continue:
+This will first run the program once until completion, and then restart the program at the very first instruction at `_start` and leave you in a GDB shell.
+
+From there, run the program until your point of interest, e.g.:
 
 ....
+break qemu_add_opts
 continue
 ....
 
-you can now reiably use reverse debugging commands such as `reverse-continue` and `reverse-next`!
+and you can now reiably use reverse debugging commands such as `reverse-continue`, `reverse-finish` and `reverse-next`!
+
+To restart debugging again after quitting `rr`, simlpy run on your host terminal:
+
+....
+rr replay
+....
 
 ==== Debug gem5 Python scripts
 
@@ -12158,25 +12169,7 @@ In order to use different build options, you might also want to use <<gem5-build
 
 ==== gem5 debug build
 
-The `gem5.debug` executable has optimizations turned off unlike the default `gem5.opt`, and provides a much better <<debug-the-emulator,debug experience>>:
+Explained at: xref:debug-the-emulator[xrefstyle=full].
-
-....
-./build-gem5 --arch aarch64 --gem5-build-type debug
-./run --arch aarch64 --debug-vm --emulator gem5 --gem5-build-type debug
-....
-
-The build outputs are automatically stored in a different directory from other build types such as `.opt` build, which prevents `.debug` files from overwriting `.opt` ones.
-
-Therefore, `--gem5-build-id` is not required.
-
-The price to pay for debuggability is high however: a Linux kernel boot was about 14 times slower than opt at 71e927e63bda6507d5a528f22c78d65099bdf36f between the commands:
-
-....
-./run --arch aarch64 --eval 'm5 exit' --emulator gem5 --linux-build-id v4.16
-./run --arch aarch64 --eval 'm5 exit' --emulator gem5 --linux-build-id v4.16 --gem5-build-type debug
-....
-
-so you will likely only use this when it is unavoidable. This is also benchmarked at: xref:benchmark-linux-kernel-boot[xrefstyle=full]
 
 ==== gem5 clang build
 

build-qemu

@@ -26,14 +26,19 @@ class Main(common.BuildCliFunction):
             target_list = '{}-linux-user'.format(self.env['arch'])
         else:
             target_list = '{}-softmmu'.format(self.env['arch'])
+        if self.env['qemu_build_type'] == 'debug':
+            # https://stackoverflow.com/questions/4689136/debug-qemu-with-gdb
+            build_type_cmd = ['--enable-debug', LF]
+        else:
+            build_type_cmd = []
         self.sh.run_cmd(
             [
                 os.path.join(self.env['qemu_source_dir'], 'configure'), LF,
-                '--enable-debug', LF,
                 '--enable-trace-backends=simple', LF,
                 '--target-list={}'.format(target_list), LF,
                 '--enable-sdl', LF,
             ] +
+            build_type_cmd +
             self.sh.add_newlines(self.env['extra_config_args']),
             extra_paths=[self.env['ccache_dir']],
             cwd=build_dir

common.py

@@ -144,6 +144,13 @@ for key in consts['emulator_short_to_long_dict']:
     consts['emulator_choices'].add(consts['emulator_short_to_long_dict'][key])
 consts['host_arch'] = platform.processor()
 consts['guest_lkmc_home'] = os.sep + consts['repo_short_id']
+consts['build_type_choices'] = [
+    # -O2 -g
+    'opt',
+    # -O0 -g
+    'debug'
+]
+consts['build_type_default'] = 'opt'
 
 class ExitLoop(Exception):
     pass
@@ -325,7 +332,8 @@ Default: {}
         )
         self.add_argument(
             '--gem5-build-type',
-            default='opt',
+            choices=consts['build_type_choices'],
+            default=consts['build_type_default'],
             help='gem5 build type, most often used for "debug" builds.'
         )
         self.add_argument(
@@ -441,11 +449,16 @@ Use the docker download Ubuntu root filesystem instead of the default Buildroot
 
         # QEMU.
         self.add_argument(
-            '-Q',
             '--qemu-build-id',
             default=consts['default_build_id'],
             help='QEMU build ID. Allows you to keep multiple separate QEMU builds.'
         )
+        self.add_argument(
+            '--qemu-build-type',
+            choices=consts['build_type_choices'],
+            default=consts['build_type_default'],
+            help='QEMU build type, most often used for "debug" vs optimized builds.'
+        )
         self.add_argument(
             '--qemu-which',
             choices=[consts['repo_short_id'], 'host'],
@@ -743,7 +756,12 @@ Incompatible archs are skipped.
         env['linux_buildroot_build_dir'] = join(env['buildroot_build_build_dir'], 'linux-custom')
 
         # QEMU
-        env['qemu_build_dir'] = join(env['out_dir'], 'qemu', env['qemu_build_id'])
+        env['qemu_build_dir'] = join(
+            env['out_dir'],
+            'qemu',
+            env['qemu_build_id'],
+            env['qemu_build_type']
+        )
         env['qemu_img_basename'] = 'qemu-img'
         env['qemu_img_executable'] = join(env['qemu_build_dir'], env['qemu_img_basename'])
         if env['userland'] is None: