64-bit boot stub?

[swarren]

If I write a "boot stub" for the VC FW to place into memory at address 0 in 64-bit mode, will you accept it into the VC FW? If so:

a) What license do you want it under? I'd be happy to make it MIT/BSD/X11 or assign copyright to the Pi Foundation; whatever is easiest for you.

b) Which address(es) does the VC FW write to? I assume at least 0x14 for the ATAGS/DTB pointer, but I'm not sure whether the initialization values for cntfrq or the Linux machine ID are written too. I'm discounting address 0x100 and on since that's just data and not part of the boot stub itself.

(As an aside, if this stub could be loaded from a separate .bin file, that might be helpful since it'd allow easy experimentation, but that would then get into a discussion of how to name the files since we'd need different files for bcm2835, bcm2836, bcm2837 32-bit, bcm2837 64-bit).

[swarren]

Oh, and it might turn out useful if the boot stub code could be included in e.g. Qemu for RPi emulation - both the existing 32-bit version and any new 64-bit version. Do you have any problems with my doing that, even for the existing 32-bit binary that you wrote? I wonder if you could publish the source under an open license to help the contribution chain to Qemu.

[popcornmix]

No problem with releasing the boot stub code. See: http://pastebin.com/raw/stxjiVVD
If you are happy with a similar licence for a 64-bit boot stub, we'd be happy to include it in the firmware.

Just let us know what addresses to modify. Currently we do:

         ((unsigned *)mem)[BCM2836 ? 5:13] = device_tree_address;

But we could do something different in 64-bit mode.
The machine ID and ctrfreq are not overwritten.

[capnm]

Hmm, it seems to work without issues, but at least according to the A53 TRM (p.328) the order of enabling the caches and SMP is wrong:

Data coherency is enabled only when the CPUECTLR.SMPEN bit is set. You must set the
SMPEN bit before enabling the data cache. If you do not, then the cache is not coherent
with other cores and data corruption could occur.

    mrc p15, 0, r0, c1, c0, 0 @ Read System Control Register
    orr r0, r0, #(1<<2)       @ cache enable
    orr r0, r0, #(1<<12)      @ icache enable
    mcr p15, 0, r0, c1, c0, 0 @ Write System Control Register
.if !BCM2710
    mrc p15, 0, r0, c1, c0, 1 @ Read Auxiliary Control Register
    orr r0, r0, #(1<<6)       @ SMP
    mcr p15, 0, r0, c1, c0, 1 @ Write Auxiliary Control Register

[swarren]

Writing to the SCR doesn't actually enable the icache/dache. To enable the cache, the MMU must also be configured/enabled, and the "ARM boot stub" doesn't do that (Linux/U-Boot/... does it during early boot).

[capnm]

Great, thanks! Do you know why is the SCR code in the stub at all?

I guess the next stop is the kernel entry point here
http://lxr.free-electrons.com/source/arch/arm64/kernel/head.S#L54
MMU=off is clear, but they actually mean the state of the SCR register bits, right?

 58  * The requirements are:
 59  *   MMU = off, D-cache = off, I-cache = on or off,

What about the IIRC VC/ARM shared L2 cache? Is it always enabled? I couldn't find anything special in the kernel code.

[swarren]

Correction: System Control Register is SCTLR, whereas SCR is Secure Configuration Register.

I think the write to the SCTLR in the stub is not necessary; At least the arch/arm64 kernel repeats this and I'm sure the arch/arm kernel must too; see setup of x0 here:
http://lxr.free-electrons.com/source/arch/arm64/mm/proc.S#L178
based on data here:
http://lxr.free-electrons.com/source/arch/arm64/mm/proc.S#L229
and x0 is written to SCTLR here:
http://lxr.free-electrons.com/source/arch/arm64/kernel/head.S#L644

I suspect the comment in the kernel code refers to the logical state of the caches being disabled. The fact this is implemented by writing bits in the SCTLR is true but simply not explicitly mentioned.

I don't know if the L2 cache can be enabled/disabled, but IIUC all control over that cache is from the VideoCore FW. I believe its state is irrelevant to ARM-based SW, except for interaction with peripherals doing DMA, since it's beyond the point of coherency/unification for the ARM cores themselves. As such I would not expect to find any code that controls it.

[swarren]

A first cut is available at https://github.com/swarren/rpi-3-aarch64-demo in armstub64.S. The most-recent-but-one includes a TODO list describing the outstanding changes.

[swarren]

I have pushed what I hope is a final version of the 64-bit stub to that github repo. The source is in armstub64.S.

I have tested this with U-Boot, and the test app in the same repo. I have not tested it with any Linux kernel yet, nor have I attempted to boot any Linux kernel from U-Boot. I believe @anholt will test it some time.

The DTB address should be written as a 64-bit value to address 0xf8.

The stub implements the standard RAM-based "spin table" method of SMP secondary CPU boot. This is a mechanism already supported by the mainline Linux kernel's arch/arm64 port, so should ease upstreaming kernel support; the arch/arm64 maintainers are unlikely to welcome a new SoC-specific SMP mechanism (besides spin table or PSCI) that uses the bcm283x mailbox registers.

I'd like to suggest creating a new git repo that contains the ARM stub source for all the Pis. I'd be happy to contribute patches to such a repo to fix the issues I filed with the 32-bit stubs if there was somewhere I could send a regular git patch against. I'd also be happy to create the initial content of such a repo if you want; just let me know (assuming the pastebin you linked above is all the versions; otherwise it might be best if you filled in the existing 32-bit stub code first).

[sukantoghosh]

Do I need any custom version of boot-firmware for this or is the HEAD of github.com/raspberrypi/firmware master branch supposed to work? Somehow this is not working for me, I did try to put prints at the beginning of armstub64.S as well, but it didn't help.

This might be worth documenting in the README as well as the firmwares shipped with 2016-03-18-raspbian-jessie don't respond to 'enable_uart=250'.

BTW, I really hope that when this stub is integrated to VC FWs it is picked from a bin file as Stephen suggested

[popcornmix]

I'm happy to add support to load:
armstub.bin (when on Pi1)
armstub7.bin (when on Pi2 or Pi3 in 32-bit mode)
armstub8.bin (when booting in 64-bit mode)

at address zero (up to 0x100 bytes). If armstub*.bin is not present it will use default ones (including current armstub64.S from @swarren's repo).

Also the presence of kernel8.img will enable 64-bit mode.
I can add existing armstub*.S files to a Pi repo (possibly in tools/mkimage).
Is that okay?

[swarren]

@sukantoghosh I tested with 046effa "firmware: arm_loader: emmc clock depends on core clock See: #572", which was HEAD of github.com/raspberrypi/firmware master branch when I last fetched a few days ago.

[swarren]

@popcornmix that all sounds good to me, thanks!

Just one query though: The stub code you posted had an ifdef for BCM2810-vs-not, so I think you'd end needing 4 FW files: ARMv6, ARMv7, ARMv8 32-bit mode, and ARMv8 64-bit mode, or am I misinterpreting something?

[swarren]

@sukantoghosh that should be enable_uart=1 not =250. I'll fix my repo in a minute.

[popcornmix]

Yes, you are right, the stub code is slightly different between Pi2 and Pi3 so we should have an extra stub file.

[sukantoghosh]

thanks @swarren it works for me

[anholt]

With @swarren's boot stub I have SMP enabled on 64-bit Linux, and I'm making it to the point of trying to NFS root mount before it fails because USB device probing isn't working.

On 2836, the boot stub was setting a 19.2Mhz CNTFREQ but not LOCAL_CONTROL/PRESCALER, so we set the LOCAL_* regs early in Linux boot. For this boot stub, I've disabled the LOCAL_* setup so that we run with the 1Mhz it sets in CNTFREQ. I'm wondering, though: Don't we want to run the timer at 19.2Mhz? If so, should the firmware/boot stub take over that responsibility, if it was already setting CNTFREQ previously?

[swarren]

BTW, there's some discussion re: DWC2 and broken DMA support in the kernel on page 3 (and later) of https://www.raspberrypi.org/forums/viewtopic.php?f=72&t=137963&start=50. I'm not sure if that's talking about the RPi Foundation kernel or mainline.

Where (which module) are the LOCAL_CONTROL/PRESCALER registers; I'm not familiar with those.

[capnm]

Other difference that I spotted during my experiments (and I guess the
fw should take care of) is the another arm64 kernel load-adress, currently
hardcoded in the makefile

http://lxr.free-electrons.com/source/arch/arm64/Makefile#L53

TEXT_OFFSET := 0x0008_0000 # 512K
was 0x8000 / 32k

The image should be idealy loaded at adress taken from the image
header (if magic == 0x644d5241 ?), not sure what to do with the
u-boot or another images or how to pass that to the boot blob.
So probably just simply load the arm64 image at 0x00080000?

The decompressed kernel image contains a 64-byte header as follows:
  u32 code0;            /* Executable code */
  u32 code1;            /* Executable code */
  u64 text_offset;      /* Image load offset, little endian */
  u64 image_size;       /* Effective Image size, little endian */
  u64 flags;            /* kernel flags, little endian */
  u64 res2  = 0;        /* reserved */
  u64 res3  = 0;        /* reserved */
  u64 res4  = 0;        /* reserved */
  u32 magic = 0x644d5241;   /* Magic number, little endian, "ARM\x64" */
  u32 res5;

[popcornmix]

I've added the arm stubs here: raspberrypi/tools@920c7ed
Pull requests for fixes are welcome.

Here is a (not hugely tested) firmware:
https://dl.dropboxusercontent.com/u/3669512/temp/firmware_armstub.zip
that supports, e.g.
armstub=armstub8-32.bin
to load a stub file from a file. The file should be 0x100 bytes (or less).

If you have a kernel8.img (and optionally an armstub8.bin) they should be used and enable 64-bit mode automatically. Supported kernel names are:
kernel8.img, kernel8-32.img, kernel7.img, kernel.img
If one of those is loaded it will by default look for:
armstub8.bin, armstub8-32.bin, armstub7.bin armstub.bin
although they can be overridden with kernel/armstub in config.txt.

[anholt]

drivers/irqchip/irq-bcm2836.c

[swarren]

@popcornmix I've tested firmware_armstub.zip and it works great, thanks! I tested:

1. Unified SD card containing kernel.img (U-Boot rpi build), kernel7.img (U-Boot rpi_2 build), kernel8.img (U-Boot rpi_3 64-bit build). I plugged the card into a B+, 2 B, and 3 B, and all worked. I didn't test if the 2 B board picked kernel.img or kernel7.img; both would have worked. I'm pretty sure the B+ correctly picked kernel.img not kernel7.img or it would have failed to execute the ARMv7 instructions in it.

2. An SD card containing just kernel8-32.img (U-Boot rpi_3_32 build) booted on the Pi 3. I also accidentally validated that fallback to kernel7.img works on the 3 B in 32-bit mode.

I didn't use mkknlimg in any of my testing. I should probably switch to that soon:-)

[swarren]

@capnm yes it looks like the FW and 64-bit ARM stub can't just hard-code the kernel address; it must be read from the kernel image header. I'll prepare a patch for the stub that:

• Defines a 64-bit location that the FW can over-write with the address of the kernel, to which the stub will jump.
• The default value at that address will still be 0x8000 for compatibility. When loading a Linux kernel, the FW will always have to read the value out of the kernel image header, so the default value isn't at all relevant for Linux, just custom other FW.
• I'll also specifically reserve a location for the CPU0 spin-table release address. This will never actually be used, but according to the DT binding documentation for ARM stream tables all CPUs need a release address defined.

[swarren]

@popcornmix I've also tested using custom ARM stubs (only on RPi3 in 64-bit mode). If I use config.txt option armstub=armstub8.bin, it works great. However, if I don't put any armstub= option in config.txt, the FW doesn't seem to automatically check for and load armstub8.bin (even though I put the "kernel" in kernel8.img). I thought from your description it would?

[popcornmix]

Yes, your expectations are correct. I'll push an updated firmware later today that fixes a few issues. Hopefully that will work.

Currently, not specifying armstub= and armstub8.bin being found enables 64-bit mode automatically.
Obviously armstub=mystub.bin shouldn't enable 64-bit mode.
What about armstub=armstub8.bin?
What about armstub=mystub8.bin?
Currently they don't, but I'm wondering if one or both should (*8.bin being the wildcard to enable 64-bit mode). Thoughts?

[swarren]

@popcornmix my inclination is to keep things simple. Have the FW decide whether to boot in 32- or 64-bit mode solely based on the kernel image filename, and don't take the stub filename into account. That's because the kernel is the primary piece of SW, and the stub is somewhat only there to support the kernel. Once that bit-size determination is made, use that to select the default stub filename. This way, the FW doesn't have to have rules to decide whether the bit size implied by the kernel filename overrides the bit size implied by the stub filename, or the other way around.

If someone wants to do something really unsusual and run a 32-bit OS (kernel) under a 64-bit secure monitor/OS, I assume they'd make that work by putting the secure monitor into kernel8.img or setting arm_control=0x200 (thus ensuring 64-bit booting), using a custom stub file (ths avoiding the built-in switch to HYP/EL2), and having the secure OS load the real kernel image (or perhaps concatenating it with the secure OS image). Hence, even with this simple scheme, doing something unusual is still possible.

I envisage something like:

bits=32
if arm_control 64-bit flag is set:
    bits=64
if kernel= not in config.txt and kernel8.img exists:
    bits=64

if bits == 64:
    armstub_fn=armstub8.bin
    kernel_fn=kernel8.img
else:
    armstub_fn=armstub8.bin
    kernel_fn=kernel8-32.img

if armstub= in config.txt:
    armstub_fn=value_from_config

if kernel= in config.txt:
    kernel_fn=value_from_config

Does that make sense?

[swarren]

@anholt the timer frequency setup is implemented in raspberrypi/tools#54, and tested w/ U-Boot's sleep command.

[popcornmix]

@swarren there are a number of "bare metal" users who just use a single block of arm code.
Currently they use a kernel.img with kernel_old=1.
It would probably make more sense for them to just use armstub.bin and no kernel file. We could then deprecate kernel_old=1. (The size limit on armstub.bin would be removed).
In that scenario determining the bits from armstub makes sense.
In theory it would be possible for armstub.bin to be 64-bit code and kernel.img to be 32-bit (with armstub switching back to 32-bit mode), which again would mean deriving the bits from the armstub would make more sense.

[popcornmix]

Here is latest test firmware. It includes latest armstub8.S from github:
https://dl.dropboxusercontent.com/u/3669512/temp/firmware_armstub2.zip

It is useful for the firmware to provide some information. Currently:
device_tree_address (dtb_ptr) and kernel_address (kernel_entry)
Possibly we'll also need device_tree_end.

Currently device_tree_address is (uint32_t *)0x34 for Pi1 and (uint32_t *)0x14 for Pi2/Pi3.
We are not currently writing it, but it looks like armstub8.S is using (uint64_t *)0xf8.

armstub8.S also uses (uint64_t *)0xd0 for kernel_entry

As armstub7.S is quite cramped in 0x100 bytes currently (we may be able to increase the size, but that could break some bare metal assumptions), using 0xd0 might be awkward.

Can we define an API for where to put these words? It seems 32-bit values will be enough as bus addresses are still 32-bits even in 64-bit mode. Using unused vectors (like 0x14) is convenient for 32-bit mode. Can we find suitable values that work for both 32-bit and 64-bit stubs?

We could always use different values for 32-bit and 64-bit, but if making them the same is possible, it simplifies things.

[pelwell]

device_tree_end shouldn't be needed, since the dtb contains a length and any serious OS will unflatten it into a more convenient form ASAP.

[capnm]

armstub8.bin only

[popcornmix]

So there are no kernel*.img files on sdcard?

[capnm]

I've had some stale files, sorry for the false alarm. A fresh checkout and suddenly it worked again.

I tested the essential ARM64 relevant cases, everything LGTM. Now it's time for useful development work. Thanks @swarren @popcornmix @pelwell !

tested (armstub8.bin, rm kernel8* config.txt)

2. 4. 9. read device_tree_address from the stub:

stub without magic no change, ok

0x00000108 -> 0x00000100 .. 0x00008000
0x00000180 -> 0x00000100 .. 0x00080000
0x10000080 -> 0x10000000 .. 0x10080000
dtb_magic present, ok

0xffffffff -> 0x0, ok
0x00000100 -> 0x0, ok

3. tested (kernel8.img, config.txt, rm armstub8*)
   dtb_magic present, ok
   device_tree=, x0 == 0x0, ok

4. tested (kernel8.img, config.txt, rm armstub8*)
   kernel8.img default@0x0008_0000, ok
   kernel_address=0x0080_0000, dtb_magic present, ok

5. tested (kernel8.img, armstub8.bin, rm config.txt)
   kernel: 0xffffffff -> 0x0, ok
   kernel: 0x0080_0000, atags: 0x108 -> 0x0080_0000, dtb_magic present, ok
   kernel: 0x0, atags: 0x108 -> 0x0008_0000, dtb_magic present, ok

[swarren]

@capnm

Good point. If the kernel image is a part of distribution, the people
would hesitate to delete it. It's also easy to forget to delete the kernel image
and for several hours be amazed what's going on.

Well, nothing is going to help if you delete the kernel accidentally. If you delete it deliberately, then having to update the stub as well is going to be a bit annoying.

I think it is also
a valuable information for the stub code, to be able to alert the user.
I think the firmware "kernel loading" state could be even more explicit.

Well, it could perhaps flash the LED on the board, but that's about all it could do. I wouldn't expect noticing that the kernel file is missing to take particularly long, but perhaps that's simply because I have scripts that copy everything to the SD card, so I'd probably just re-run them, find it works, be puzzled and move on.

@pelwell the 0xffffffff changes make sense to me.

I tested the latest FW with both rpi-3-aarch64-demo and U-Boot with just a kernel8.img and no armstub8.bin and no config.txt options to force 64-bit and everything worked as expected.

Since everything is basically working fine w.r.t. armstub8.bin, I'm closing this bug. If problems are found, we can open new bugs for those specific changes. Thanks!

[SylvainGarrigues]

@pelwell Setting kernel_address=0x100000 in config.txt doesn't work, is it intended? How do I modify the load address these days? Shall I provide my own stubs?

Right now the only solution I have is to have a jump to 0x100000 as the first instruction of my kernel, which I have padded with zeros between 8004 and 100000:

/usr/bin/printf "\001\366\240\343" > first_commands
dd of=kernel7.img bs=1m count=4 if=/dev/zero
dd of=kernel7.img bs=1 conv=notrunc if=first_commands
dd of=kernel7.img bs=32k oseek=31 conv=notrunc if=kernel.bin

[pelwell]

@SylvainGarrigues No, that is not what is intended, and it isn't immediately obvious how that could be happening. Are you using an external stub? Is the stub being patched, and if so with what?

[SylvainGarrigues]

@pelwell I am not using any stub, and my config.txt just reads:

device_tree=rpi2.dtb
gpu_mem=128
disable_overscan=1
max_usb_current=1
audio_pwm_mode=2

Padding the kernel with 0 like I mentioned earlier works with this config.txt.

If I add the line kernel_address=0x100000 in config.txt and do not pad with 0, nothing happens.

[pelwell]

That's not my experience. With this config.txt:

enable_uart=1
dtparam=audio=on
kernel_address=0x100000
device_tree=bcm2710-rpi-3-b.dtb

and after zeroing the first 2MB of memory (not a requirement, just to confirm that it isn't picking up anything from the previous attempt), it boots. This is with a standard 32-bit compressed kernel built to run at 0x8000, so the decompressor must be decompressing from 0x100000 back to 0x8000 (the memory at 0x8000 once the kernel is running looks like the start of a normal kernel).

[pelwell]

And to remove any doubt, I see the kernel being loaded to 0x100000 as expected. All modifications to the internal kernel_address variable, apart from the initialisation from config.txt, are conditional on it being non-zero, so I don't see how this can be failing for you.

[swarren]

FWIW, yes the 32-bit ARM zImage decompressor is position-independent and so will always decompress to 0x8000 (or wherever the Image is linked, I suppose) no matter where it's loaded. Well, IIRC the decompressor must be loaded within the first 128MiB of RAM for AUTO_ZRELADDR to work correctly. As an aside, when using zImage, 0x8000 (or thereabouts) is actually about the worst place the kernel could be loaded, since the decompressor must first copy itself somewhere else and then perform decompression, so that the decompressed data doesn't over-write the compressed data during decompression. If the zImage was loaded elsewhere (say, 32MB into RAM) that initial memcpy wouldn't be needed. Still, too late to change that default I suspect.

[SylvainGarrigues]

I am sorry, I was mistaken, I had a serial problem.

My kernel is a raw binary (therefore neither an ELF nor a zImage) and it appears my kernel is loaded at the address specified by kernel_address and it is run from there without any stub, just with the kernel_address line in config.txt. Which is just what I wanted.

Now I can boot FreeBSD with just two lines in config.txt:

device_tree=rpi2.dtb
kernel_address=0x100000

So cool! No need for mkknlimg anymore, no need for u-boot either, no need for a stub, no need for kernel_old.

Thanks!

[capnm]

@SylvainGarrigues
And I can boot linux with just one line in config.txt ;)

device_tree=bcm2837-rpi-3-b.dtb

scp arch/arm64/boot/Image $rpi3/kernel8.img
scp arch/arm64/boot/dts/broadcom/bcm2837-rpi-3-b.dtb $rpi3/
...
uname -a
Linux rpi3 4.6.0-rc3-next-20160413+ #1 SMP PREEMPT Wed Apr 20 13:50:53 CEST 2016 aarch64 GNU/Linux