diff mbox series

[v2] aspeed: Add boot stub for smp booting

Message ID 20200409063137.803522-1-joel@jms.id.au
State Superseded
Headers show
Series [v2] aspeed: Add boot stub for smp booting | expand

Commit Message

Joel Stanley April 9, 2020, 6:31 a.m. UTC
This is a boot stub that is similar to the code u-boot runs, allowing
the kernel to boot the secondary CPU.

u-boot works as follows:

 1. Initialises the SMP mailbox area in the SCU at 0x1e6e2180 with default values

 2. Copies a stub named 'mailbox_insn' from flash to the SCU, just above the
    mailbox area

 3. Sets AST_SMP_MBOX_FIELD_READY to a magic value to indicate the
    secondary can begin execution from the stub

 4. The stub waits until the AST_SMP_MBOX_FIELD_GOSIGN register is set to
    a magic value

 5. Jumps to the address in AST_SMP_MBOX_FIELD_ENTRY, starting Linux

Linux indicates it is ready by writing the address of its entrypoint
function to AST_SMP_MBOX_FIELD_ENTRY and the 'go' magic number to
AST_SMP_MBOX_FIELD_GOSIGN. The secondary CPU sees this at step 4 and
breaks out of it's loop.

To be compatible, a fixed qemu stub is loaded into the mailbox area. As
qemu can ensure the stub is loaded before execution starts, we do not
need to emulate the AST_SMP_MBOX_FIELD_READY behaviour of u-boot. The
secondary CPU's program counter points to the beginning of the stub,
allowing qemu to start secondaries at step four.

Reboot behaviour is preserved by resetting AST_SMP_MBOX_FIELD_GOSIGN
when the secondaries are reset.

This is only configured when the system is booted with -kernel and qemu
does not execute u-boot first.

Reviewed-by: Cédric Le Goater <clg@kaod.org>

Tested-by: Cédric Le Goater <clg@kaod.org>

Signed-off-by: Joel Stanley <joel@jms.id.au>

---
v2: test for number of CPUs
---
 hw/arm/aspeed.c | 65 +++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 65 insertions(+)

-- 
2.25.1

Comments

Peter Maydell April 16, 2020, 4:48 p.m. UTC | #1
On Thu, 9 Apr 2020 at 07:31, Joel Stanley <joel@jms.id.au> wrote:
>

> This is a boot stub that is similar to the code u-boot runs, allowing

> the kernel to boot the secondary CPU.


> +static void aspeed_write_smpboot(ARMCPU *cpu,

> +                                 const struct arm_boot_info *info)

> +{

> +    static const uint32_t poll_mailbox_ready[] = {

> +        /*

> +         * r2 = per-cpu go sign value

> +         * r1 = AST_SMP_MBOX_FIELD_ENTRY

> +         * r0 = AST_SMP_MBOX_FIELD_GOSIGN

> +         */

> +        0xee100fb0,  /* mrc     p15, 0, r0, c0, c0, 5 */

> +        0xe21000ff,  /* ands    r0, r0, #255          */

> +        0xe59f201c,  /* ldr     r2, [pc, #28]         */

> +        0xe1822000,  /* orr     r2, r2, r0            */

> +

> +        0xe59f1018,  /* ldr     r1, [pc, #24]         */

> +        0xe59f0018,  /* ldr     r0, [pc, #24]         */

> +

> +        0xe320f002,  /* wfe                           */

> +        0xe5904000,  /* ldr     r4, [r0]              */

> +        0xe1520004,  /* cmp     r2, r4                */

> +        0x1afffffb,  /* bne     <wfe>                 */


Note that unlike "wfi", QEMU's "wfe" implementation is merely
a 'yield', so a secondary-CPU boot loop that has wfe in it
will basically be a busy-loop of those vcpu threads.
(This is why the smpboot code in hw/arm/boot.c uses wfi.)

I don't suppose the secondary boot protocol on these boards
is such that a wfi loop will work ? (Depends on what the
primary code in the kernel does to prod the secondary after
writing the magic value.)


> +        0xe591f000,  /* ldr     pc, [r1]              */

> +        AST_SMP_MBOX_GOSIGN,

> +        AST_SMP_MBOX_FIELD_ENTRY,

> +        AST_SMP_MBOX_FIELD_GOSIGN,

> +    };


thanks
-- PMM
Joel Stanley May 4, 2020, 8:18 a.m. UTC | #2
On Thu, 16 Apr 2020 at 16:48, Peter Maydell <peter.maydell@linaro.org> wrote:
>

> On Thu, 9 Apr 2020 at 07:31, Joel Stanley <joel@jms.id.au> wrote:

> >

> > This is a boot stub that is similar to the code u-boot runs, allowing

> > the kernel to boot the secondary CPU.

>

> > +static void aspeed_write_smpboot(ARMCPU *cpu,

> > +                                 const struct arm_boot_info *info)

> > +{

> > +    static const uint32_t poll_mailbox_ready[] = {

> > +        /*

> > +         * r2 = per-cpu go sign value

> > +         * r1 = AST_SMP_MBOX_FIELD_ENTRY

> > +         * r0 = AST_SMP_MBOX_FIELD_GOSIGN

> > +         */

> > +        0xee100fb0,  /* mrc     p15, 0, r0, c0, c0, 5 */

> > +        0xe21000ff,  /* ands    r0, r0, #255          */

> > +        0xe59f201c,  /* ldr     r2, [pc, #28]         */

> > +        0xe1822000,  /* orr     r2, r2, r0            */

> > +

> > +        0xe59f1018,  /* ldr     r1, [pc, #24]         */

> > +        0xe59f0018,  /* ldr     r0, [pc, #24]         */

> > +

> > +        0xe320f002,  /* wfe                           */

> > +        0xe5904000,  /* ldr     r4, [r0]              */

> > +        0xe1520004,  /* cmp     r2, r4                */

> > +        0x1afffffb,  /* bne     <wfe>                 */

>

> Note that unlike "wfi", QEMU's "wfe" implementation is merely

> a 'yield', so a secondary-CPU boot loop that has wfe in it

> will basically be a busy-loop of those vcpu threads.

> (This is why the smpboot code in hw/arm/boot.c uses wfi.)

>

> I don't suppose the secondary boot protocol on these boards

> is such that a wfi loop will work ? (Depends on what the

> primary code in the kernel does to prod the secondary after

> writing the magic value.)


Thanks for the review.

I chose wfe as it's what aspeed's u-boot has. I don't have a strong
understand of this area of ARM processors, nor do I have any insight
into why that choice was made.

I did some testing and wfi in the qemu stub works fine with Linux so I
will respin with that instead.

Cheers,

Joel
diff mbox series

Patch

diff --git a/hw/arm/aspeed.c b/hw/arm/aspeed.c
index 88bcd6ff3fbd..e363e495ef48 100644
--- a/hw/arm/aspeed.c
+++ b/hw/arm/aspeed.c
@@ -116,6 +116,58 @@  static const MemoryRegionOps max_ram_ops = {
     .endianness = DEVICE_NATIVE_ENDIAN,
 };
 
+#define AST_SMP_MAILBOX_BASE            0x1e6e2180
+#define AST_SMP_MBOX_FIELD_ENTRY        (AST_SMP_MAILBOX_BASE + 0x0)
+#define AST_SMP_MBOX_FIELD_GOSIGN       (AST_SMP_MAILBOX_BASE + 0x4)
+#define AST_SMP_MBOX_FIELD_READY        (AST_SMP_MAILBOX_BASE + 0x8)
+#define AST_SMP_MBOX_FIELD_POLLINSN     (AST_SMP_MAILBOX_BASE + 0xc)
+#define AST_SMP_MBOX_CODE               (AST_SMP_MAILBOX_BASE + 0x10)
+#define AST_SMP_MBOX_GOSIGN             0xabbaab00
+
+static void aspeed_write_smpboot(ARMCPU *cpu,
+                                 const struct arm_boot_info *info)
+{
+    static const uint32_t poll_mailbox_ready[] = {
+        /*
+         * r2 = per-cpu go sign value
+         * r1 = AST_SMP_MBOX_FIELD_ENTRY
+         * r0 = AST_SMP_MBOX_FIELD_GOSIGN
+         */
+        0xee100fb0,  /* mrc     p15, 0, r0, c0, c0, 5 */
+        0xe21000ff,  /* ands    r0, r0, #255          */
+        0xe59f201c,  /* ldr     r2, [pc, #28]         */
+        0xe1822000,  /* orr     r2, r2, r0            */
+
+        0xe59f1018,  /* ldr     r1, [pc, #24]         */
+        0xe59f0018,  /* ldr     r0, [pc, #24]         */
+
+        0xe320f002,  /* wfe                           */
+        0xe5904000,  /* ldr     r4, [r0]              */
+        0xe1520004,  /* cmp     r2, r4                */
+        0x1afffffb,  /* bne     <wfe>                 */
+        0xe591f000,  /* ldr     pc, [r1]              */
+        AST_SMP_MBOX_GOSIGN,
+        AST_SMP_MBOX_FIELD_ENTRY,
+        AST_SMP_MBOX_FIELD_GOSIGN,
+    };
+
+    rom_add_blob_fixed("aspeed.smpboot", poll_mailbox_ready,
+                       sizeof(poll_mailbox_ready),
+                       info->smp_loader_start);
+}
+
+static void aspeed_reset_secondary(ARMCPU *cpu,
+                                   const struct arm_boot_info *info)
+{
+    AddressSpace *as = arm_boot_address_space(cpu, info);
+    CPUState *cs = CPU(cpu);
+
+    /* info->smp_bootreg_addr */
+    address_space_stl_notdirty(as, AST_SMP_MBOX_FIELD_GOSIGN, 0,
+                               MEMTXATTRS_UNSPECIFIED, NULL);
+    cpu_set_pc(cs, info->smp_loader_start);
+}
+
 #define FIRMWARE_ADDR 0x0
 
 static void write_boot_rom(DriveInfo *dinfo, hwaddr addr, size_t rom_size,
@@ -270,6 +322,19 @@  static void aspeed_machine_init(MachineState *machine)
         }
     }
 
+    if (machine->kernel_filename && bmc->soc.num_cpus > 1) {
+        /* With no u-boot we must set up a boot stub for the secondary CPU */
+        MemoryRegion *smpboot = g_new(MemoryRegion, 1);
+        memory_region_init_ram(smpboot, OBJECT(bmc), "aspeed.smpboot",
+                               0x80, &error_abort);
+        memory_region_add_subregion(get_system_memory(),
+                                    AST_SMP_MAILBOX_BASE, smpboot);
+
+        aspeed_board_binfo.write_secondary_boot = aspeed_write_smpboot;
+        aspeed_board_binfo.secondary_cpu_reset_hook = aspeed_reset_secondary;
+        aspeed_board_binfo.smp_loader_start = AST_SMP_MBOX_CODE;
+    }
+
     aspeed_board_binfo.ram_size = ram_size;
     aspeed_board_binfo.loader_start = sc->memmap[ASPEED_SDRAM];
     aspeed_board_binfo.nb_cpus = bmc->soc.num_cpus;