# Linux Kernel Module Cheat

##### 3.5.1.1. arm64 secondary CPU entry point

In gem5 aarch64 Linux v4.18, experimentally the entry point of secondary CPUs seems to be `secondary_holding_pen` as shown at https://gist.github.com/cirosantilli2/34a7bc450fcb6c1c1a910369be1fdd90

What happens is that:

- the bootloader goes in in WFE

- the kernel writes the entry point to the secondary CPU (the address of `secondary_holding_pen`) with CPU0 at the address given to the kernel in the `cpu-release-addr` of the DTB

- the kernel wakes up the bootloader with a SEV, and the bootloader boots to the address the kernel told it

The CPU0 action happens at: https://github.com/cirosantilli/linux/blob/v5.7/arch/arm64/kernel/smp_spin_table.c:

Here’s the code that writes the address and does SEV:

    static int smp_spin_table_cpu_prepare(unsigned int cpu)
    {
    	__le64 __iomem *release_addr;

    	if (!cpu_release_addr[cpu])
    		return -ENODEV;

    	/*
    	 * The cpu-release-addr may or may not be inside the linear mapping.
    	 * As ioremap_cache will either give us a new mapping or reuse the
    	 * existing linear mapping, we can use it to cover both cases. In
    	 * either case the memory will be MT_NORMAL.
    	 */
    	release_addr = ioremap_cache(cpu_release_addr[cpu],
    				     sizeof(*release_addr));
    	if (!release_addr)
    		return -ENOMEM;

    	/*
    	 * We write the release address as LE regardless of the native
    	 * endianess of the kernel. Therefore, any boot-loaders that
    	 * read this address need to convert this address to the
    	 * boot-loader's endianess before jumping. This is mandated by
    	 * the boot protocol.
    	 */
    	writeq_relaxed(__pa_symbol(secondary_holding_pen), release_addr);
    	__flush_dcache_area((__force void *)release_addr,
    			    sizeof(*release_addr));

    	/*
    	 * Send an event to wake up the secondary CPU.
    	 */
    	sev();

and here’s the code that reads the value from the DTB:

    static int smp_spin_table_cpu_init(unsigned int cpu)
    {
    	struct device_node *dn;
    	int ret;

    	dn = of_get_cpu_node(cpu, NULL);
    	if (!dn)
    		return -ENODEV;

    	/*
    	 * Determine the address from which the CPU is polling.
    	 */
    	ret = of_property_read_u64(dn, "cpu-release-addr",
    				   &cpu_release_addr[cpu]);