# Linux Kernel Module Cheat

#### 24.3.2. gem5 cache size

https://stackoverflow.com/questions/49624061/how-to-run-gem5-simulator-in-fs-mode-without-cache/49634544#49634544

A quick `./run --emulator gem5 -- -h` leads us to the options:

    --caches
    --l1d_size=1024
    --l1i_size=1024
    --l2cache
    --l2_size=1024
    --l3_size=1024

But keep in mind that it only affects benchmark performance of the most detailed CPU types as shown at: Table 2, “gem5 cache support in function of CPU type”.

Table 2. gem5 cache support in function of CPU type
arch | CPU type | caches used
--- | --- | ---
X86 | `AtomicSimpleCPU` | no
X86 | `DerivO3CPU` | ?*
ARM | `AtomicSimpleCPU` | no
ARM | `HPI` | yes

*: couldn’t test because of:

- https://stackoverflow.com/questions/49011096/how-to-switch-cpu-models-in-gem5-after-restoring-a-checkpoint-and-then-observe-t

Cache sizes can in theory be checked with the methods described at: https://superuser.com/questions/55776/finding-l2-cache-size-in-linux:

    lscpu
    cat /sys/devices/system/cpu/cpu0/cache/index2/size

and on Ubuntu 20.04 host but not Buildroot 1.31.1:

    getconf -a | grep CACHE

and we also have an easy to use userland executable using sysconf at userland/linux/sysconf.c:

    ./run --emulator gem5 --userland userland/linux/sysconf.c

but for some reason the Linux kernel is not seeing the cache sizes:

- https://stackoverflow.com/questions/49008792/why-doesnt-the-linux-kernel-see-the-cache-sizes-in-the-gem5-emulator-in-full-sy

- http://gem5-users.gem5.narkive.com/4xVBlf3c/verify-cache-configuration

Behaviour breakdown:

- arm QEMU and gem5 (both `AtomicSimpleCPU` or `HPI`), x86 gem5: `/sys` files don’t exist, and `getconf` and `lscpu` value empty

- x86 QEMU: `/sys` files exist, but `getconf` and `lscpu` values still empty

The only precise option is therefore to look at gem5 config.ini as done at: gem5 event queue TimingSimpleCPU syscall emulation freestanding example analysis with caches.

Or for a quick and dirty performance measurement approach instead:

    ./gem5-bench-cache -- --arch aarch64
    cat "$(./getvar --arch aarch64 run_dir)/bench-cache.txt"

which gives:

    cmd ./run --emulator gem5 --arch aarch64 --gem5-readfile "dhrystone 1000" --gem5-restore 1 -- --caches --l2cache --l1d_size=1024   --l1i_size=1024   --l2_size=1024   --l3_size=1024   --cpu-type=HPI --restore-with-cpu=HPI
    time 23.82
    exit_status 0
    cycles 93284622
    instructions 4393457

    cmd ./run --emulator gem5 --arch aarch64 --gem5-readfile "dhrystone 1000" --gem5-restore 1 -- --caches --l2cache --l1d_size=1024kB --l1i_size=1024kB --l2_size=1024kB --l3_size=1024kB --cpu-type=HPI --restore-with-cpu=HPI
    time 14.91
    exit_status 0
    cycles 10128985
    instructions 4211458

    cmd ./run --emulator gem5 --arch aarch64 --gem5-readfile "dhrystone 10000" --gem5-restore 1 -- --caches --l2cache --l1d_size=1024   --l1i_size=1024   --l2_size=1024   --l3_size=1024   --cpu-type=HPI --restore-with-cpu=HPI
    time 51.87
    exit_status 0
    cycles 188803630
    instructions 12401336

    cmd ./run --emulator gem5 --arch aarch64 --gem5-readfile "dhrystone 10000" --gem5-restore 1 -- --caches --l2cache --l1d_size=1024kB --l1i_size=1024kB --l2_size=1024kB --l3_size=1024kB --cpu-type=HPI --restore-with-cpu=HPI
    time 35.35
    exit_status 0
    cycles 20715757
    instructions 12192527

    cmd ./run --emulator gem5 --arch aarch64 --gem5-readfile "dhrystone 100000" --gem5-restore 1 -- --caches --l2cache --l1d_size=1024   --l1i_size=1024   --l2_size=1024   --l3_size=1024   --cpu-type=HPI --restore-with-cpu=HPI
    time 339.07
    exit_status 0
    cycles 1176559936
    instructions 94222791

    cmd ./run --emulator gem5 --arch aarch64 --gem5-readfile "dhrystone 100000" --gem5-restore 1 -- --caches --l2cache --l1d_size=1024kB --l1i_size=1024kB --l2_size=1024kB --l3_size=1024kB --cpu-type=HPI --restore-with-cpu=HPI
    time 240.37
    exit_status 0
    cycles 125666679
    instructions 91738770

We make the following conclusions:

- the number of instructions almost does not change: the CPU is waiting for memory all the extra time. TODO: why does it change at all?

- the wall clock execution time is not directionally proportional to the number of cycles: here we had a 10x cycle increase, but only 2x time increase. This suggests that the simulation of cycles in which the CPU is waiting for memory to come back is faster.