Fixed QEMU record/replay script and manual

README.adoc

@@ -11013,62 +11013,33 @@ By comparing the terminal output of both runs, we can see that they are the exac
 * timestamps of dmesg output
 * <<rand-check-out>> output
 
-The record and replay feature was revived around QEMU v3.0.0. It existed earlier but it rot completely. As of v3.0.0 it is still flaky: sometimes we get deadlocks, and only a limited number of command line arguments are supported.
+The record and replay feature was revived around QEMU v3.0.0. In v5.2.0 it is quite usable, almost all peripherals and vCPUs are supported.
 
-Documented at: https://github.com/qemu/qemu/blob/v2.12.0/docs/replay.txt
+Documented at: https://github.com/qemu/qemu/blob/v5.2.0/docs/replay.txt
 
-TODO: using `-r` as above leads to a kernel warning:
+replay may be used with with network:
-
-....
-rcu_sched detected stalls on CPUs/tasks
-....
-
-TODO: replay deadlocks intermittently at disk operations, last kernel message:
-
-....
-EXT4-fs (sda): re-mounted. Opts: block_validity,barrier,user_xattr
-....
-
-TODO replay with network gets stuck:
 
 ....
 ./qemu-rr --eval-after 'ifup -a;wget -S google.com;./linux/poweroff.out;'
 ....
 
-after the message:
+`arm` and `aarch64` targets can also be used with rr:
 
 ....
-adding dns 10.0.2.3
+./qemu-rr --arch aarch64 --eval-after './linux/rand_check.out;./linux/poweroff.out;'
+./qemu-rr --arch aarch64 --eval-after 'ifup -a;wget -S google.com;./linux/poweroff.out;'
 ....
 
-There is explicit network support on the QEMU patches, but either it is buggy or we are not using the correct magic options.
+Replay also supports <<initrd>> and no disk:
-
-Solved on unmerged c42634d8e3428cfa60672c3ba89cabefc720cde9 from https://github.com/ispras/qemu/tree/rr-180725
-
-TODO `arm` and `aarch64` only seem to work with initrd since I cannot plug a working IDE disk device? See also: https://lists.gnu.org/archive/html/qemu-devel/2018-02/msg05245.html
-
-Then, when I tried with <<initrd>> and no disk:
 
 ....
 ./build-buildroot --arch aarch64 --initrd
 ./qemu-rr --arch aarch64 --eval-after './linux/rand_check.out;./linux/poweroff.out;' --initrd
 ....
 
-QEMU crashes with:
-
-....
-ERROR:replay/replay-time.c:49:replay_read_clock: assertion failed: (replay_file && replay_mutex_locked())
-....
-
-I had the same error previously on x86-64, but it was fixed: https://bugs.launchpad.net/qemu/+bug/1762179 so maybe the forgot to fix it for `aarch64`?
-
-Solved on unmerged c42634d8e3428cfa60672c3ba89cabefc720cde9 from https://github.com/ispras/qemu/tree/rr-180725
-
 ===== QEMU reverse debugging
 
-TODO get working.
+QEMU replays support checkpointing, and this allows for a simplistic "reverse debugging" implementation since v5.2.0:
-
-QEMU replays support checkpointing, and this allows for a simplistic "reverse debugging" implementation proposed at https://lists.gnu.org/archive/html/qemu-devel/2018-06/msg00478.html on the unmerged https://github.com/ispras/qemu/tree/rr-180725[]:
 
 ....
 ./run --eval-after './linux/rand_check.out;./linux/poweroff.out;' --record
@@ -11093,6 +11064,8 @@ reverse-continue
 
 and we are back at `start_kernel`
 
+`reverse-continue` proceeds to the latest of the earlier breakpoints or to the very beginning if there were no breakpoints before.
+
 ==== QEMU trace multicore
 
 TODO: is there any way to distinguish which instruction runs on each core? Doing:

run

@@ -817,23 +817,20 @@ Extra options to append at the end of the emulator command line.
                     if rr:
                         driveif = 'none'
                         rrid = ',id=img-direct'
+                        rrid2 = ',id=img-direct2'
                         root = 'root=/dev/sda'
-                        snapshot = ''
+                        snapshot = ',snapshot'
                     else:
                         driveif = 'virtio'
                         root = 'root=/dev/vda'
                         rrid = ''
+                        rrid2 = ''
                         snapshot = ',snapshot'
                     if not self.env['baremetal']:
                         if not os.path.exists(self.env['qcow2_file']):
                             if not os.path.exists(self.env['rootfs_raw_file']):
                                 raise_rootfs_not_found()
                             self.raw_to_qcow2(qemu_which=self.env['qemu_which'])
-                        if rr:
-                            extra_emulator_args.extend([
-                            '-drive', 'driver=blkreplay,if=none,image=img-direct,id=img-blkreplay', LF,
-                            '-device', 'ide-hd,drive=img-blkreplay', LF,
-                        ])
                     if use_disk_image:
                         if os.path.splitext(self.env['disk_image'])[1] == '.qcow2':
                             disk_format = 'qcow2'
@@ -850,6 +847,15 @@ Extra options to append at the end of the emulator command line.
                             ),
                             LF,
                         ])
+                        hd_dev = 'ide-hd'
+                        if self.env['is_arm']:
+                            hd_dev = 'virtio-blk-device'
+                            root = 'root=/dev/vda'
+                        if rr:
+                            extra_emulator_args.extend([
+                            '-drive', 'driver=blkreplay,if=none,image=img-direct,id=img-blkreplay', LF,
+                            '-device', '{},drive=img-blkreplay'.format(hd_dev), LF,
+                        ])
                     if os.path.exists(self.env['disk_image_2']):
                         extra_emulator_args.extend([
                             '-drive',
@@ -858,10 +864,15 @@ Extra options to append at the end of the emulator command line.
                                 'raw',
                                 driveif,
                                 snapshot,
-                                rrid
+                                rrid2
                             ),
                             LF,
                         ])
+                        if rr:
+                            extra_emulator_args.extend([
+                            '-drive', 'driver=blkreplay,if=none,image=img-direct2,id=img-blkreplay', LF,
+                            '-device', '{},drive=img-blkreplay2'.format(hd_dev), LF,
+                        ])
                 if rr:
                     extra_emulator_args.extend([
                         '-object', 'filter-replay,id=replay,netdev=net0', LF,