# Linux Kernel Module Cheat

##### 24.21.1.1. gem5 functional vs atomic vs timing memory requests

gem5 memory requests can be classified in the following broad categories:

- functional: get the value magically, do not update caches, see also: gem5 functional requests

- atomic: get the value now without making a separate event, but do not update caches. Cannot work in Ruby due to fundamental limitations, mentioned in passing at: https://gem5.atlassian.net/browse/GEM5-676

- timing: get the value simulating delays and updating caches

This trichotomy can be notably seen in the definition of the MasterPort class:

    class MasterPort : public Port, public AtomicRequestProtocol,
        public TimingRequestProtocol, public FunctionalRequestProtocol

and the base classes are defined under `src/mem/protocol/`.

Then, by reading the rest of the class, we see that the send methods are all boring, and just forward to some polymorphic receiver that does the actual interesting activity:

        Tick
        sendAtomicSnoop(PacketPtr pkt)
        {
            return AtomicResponseProtocol::sendSnoop(_masterPort, pkt);
        }

        Tick
        AtomicResponseProtocol::sendSnoop(AtomicRequestProtocol *peer, PacketPtr pkt)
        {
            assert(pkt->isRequest());
            return peer->recvAtomicSnoop(pkt);
        }

The receive methods are therefore the interesting ones, and must be overridden on derived classes if they ever expect to receive such requests:

        Tick
        recvAtomicSnoop(PacketPtr pkt) override
        {
            panic("%s was not expecting an atomic snoop request\n", name());
            return 0;
        }

        void
        recvFunctionalSnoop(PacketPtr pkt) override
        {
            panic("%s was not expecting a functional snoop request\n", name());
        }

        void
        recvTimingSnoopReq(PacketPtr pkt) override
        {
            panic("%s was not expecting a timing snoop request.\n", name());
        }

One question that comes up now is: but why do CPUs need to care about snoop requests?

And one big answer is: to be able to implement LLSC atomicity as mentioned at: ARM LDXR and STXR instructions, since when other cores update memory, they could invalidate the lock of the current core.

Then, as you might expect, we can see that for example `AtomicSimpleCPU` does not override `recvTimingSnoopReq`.

Now let see which requests are generated by ordinary ARM LDR instruction. We run:

    ./run \
      --arch aarch64 \
      --debug-vm \
      --emulator gem5 \
      --gem5-build-type debug \
      --useland userland/arch/aarch64/freestanding/linux/hello.S \

and then break at the methods of the LDR class `LDRXL64_LIT`: gem5 `execute` vs `initiateAcc` vs `completeAcc`.

Before starting, we of course guess that:

- `AtomicSimpleCPU` will be making atomic accesses from `execute`

- `TimingSimpleCPU` will be making timing accesses from `initiateAcc`, which must generate the event which leads to `completeAcc`

so let’s confirm it.

We break on `ArmISAInst::LDRXL64_LIT::execute` which is what `AtomicSimpleCPU` uses, and that leads as expected to:

    MasterPort::sendAtomic
    AtomicSimpleCPU::sendPacket
    AtomicSimpleCPU::readMem
    SimpleExecContext::readMem
    readMemAtomic<(ByteOrder)1, ExecContext, unsigned long>
    readMemAtomicLE<ExecContext, unsigned long>
    ArmISAInst::LDRXL64_LIT::execute
    AtomicSimpleCPU::tick

Notably, `AtomicSimpleCPU::readMem` immediately translates the address, creates a packet, sends the atomic request, and gets the response back without any events.

And now if we do the same with `--cpu-type TimingSimpleCPU` and break at `ArmISAInst::LDRXL64_LIT::initiateAcc`, and then add another break for the next event schedule `b EventManager::schedule` (which we imagine is the memory read) we reach:

    EventManager::schedule
    DRAMCtrl::addToReadQueue
    DRAMCtrl::recvTimingReq
    DRAMCtrl::MemoryPort::recvTimingReq
    TimingRequestProtocol::sendReq
    MasterPort::sendTimingReq
    CoherentXBar::recvTimingReq
    CoherentXBar::CoherentXBarSlavePort::recvTimingReq
    TimingRequestProtocol::sendReq
    MasterPort::sendTimingReq
    TimingSimpleCPU::handleReadPacket
    TimingSimpleCPU::sendData
    TimingSimpleCPU::finishTranslation
    DataTranslation<TimingSimpleCPU*>::finish
    ArmISA::TLB::translateComplete
    ArmISA::TLB::translateTiming
    ArmISA::TLB::translateTiming
    TimingSimpleCPU::initiateMemRead
    SimpleExecContext::initiateMemRead
    initiateMemRead<ExecContext, unsigned long>
    ArmISAInst::LDRXL64_LIT::initiateAcc
    TimingSimpleCPU::completeIfetch
    TimingSimpleCPU::IcachePort::ITickEvent::process
    EventQueue::serviceOne

so as expected we have `TimingRequestProtocol::sendReq`.

Remember however that timing requests are a bit more complicated due to paging, since the page table walk can itself lead to further memory requests.

In this particular instance, the address being read with `ldr x2, =len` ARM LDR pseudo-instruction is likely placed just after the text section, and therefore the pagewalk is already in the TLB due to previous instruction fetches, and this is because the translation just finished immediately going through `TimingSimpleCPU::finishTranslation`, some key snippets are:

    TLB::translateComplete(const RequestPtr &req, ThreadContext *tc,
            Translation *translation, Mode mode, TLB::ArmTranslationType tranType,
            bool callFromS2)
    {
        bool delay = false;
        Fault fault;
        if (FullSystem)
            fault = translateFs(req, tc, mode, translation, delay, true, tranType);
        else
            fault = translateSe(req, tc, mode, translation, delay, true);
        if (!delay)
            translation->finish(fault, req, tc, mode);
        else
            translation->markDelayed();

and then `translateSe` does not use `delay` at all, so we learn that in syscall emulation, `delay` is always `false` and things progress immediately there. And then further down `TimingSimpleCPU::finishTranslation` does some more fault checking:

    void
    TimingSimpleCPU::finishTranslation(WholeTranslationState *state)
    {
        if (state->getFault() != NoFault) {
            translationFault(state->getFault());
        } else {
            if (!state->isSplit) {
                sendData(state->mainReq, state->data, state->res,
                         state->mode == BaseTLB::Read);

Tested in gem5 b1623cb2087873f64197e503ab8894b5e4d4c7b4.

- gem5 functional requests