I will describe here how I made FreeBSD running on some embedded board, particularly on the board that has Allwinner A10 SoC. Please note that this is not complete or official guide or documentation, rather steps that was needed to bring up FreeBSD on particular board. I also used andrew@'s pdf and ian@'s BootMemoryMap.txt in this regard. Allwinner A10 is a SoC that has Cortex A8 ARM processor. For more information about Cortex A8 please see ARM site. SoC documentation specially technical reference manual is very handy to have, otherwise it would be difficult to look for memory addresses, interrupts etc from different linux and u-boot sources and some are different and sometimes it is totally confusing and time consuming. First there is need for host machine (can be virtual machine) with FreeBSD head/CURRENT src. This is because we need to build FreeBSD kernel for the board. Typical boot sequence of some embedded board includes: 1. Chip firmware (Performs chip specific initialisation, Hard-wired in the chip) 2. Early boot (Performs board specific initialisation e.g. Set up SDRAM, simple disk drivers Examples include TI X-Loader, Raspberry-Pi bootcode.bin, U-Boot SPL) 3. U-Boot (Advanced executable loader Can load from storage, network, serial (e.g. zmodem), etc Like a BIOS++ Can be scripted, e.g. change the boot commands Can load U-Boot executables, some ELF files, raw binaries) 4. ubldr (U-Boot loader) - (Is the FreeBSD loader Calls back to U-Boot to access disk and network) 5. Kernel Some descriptions related kernel: kernel - Kernel with elf headers kernel.bin - Kernel without elf headers kernel.debug - Kernel with elf headers and full debugging info kernel.symbols - Just symbols for debugging a crash dump with gdb kernel.gz.tramp - Gzipped kernel+unzipper trampoline with elf headers kernel.gz.tramp.bin - Gzipped kernel+unzipper trampoline w/o elf headers kernel.tramp - Uncompressed kernel+trampoline with elf headers kernel.tramp.bin - Uncompressed kernel+trampoline w/o elf headers - For a kernel without trampoline code, the entry point is in locore.S. - The kernel entry point listed in the elf header is the label _start in the file sys/arm/locore.S. When entered directly from a bootloader the MMU is typically disabled and the pc is the physical address of the entry point. - The entry code crafts simple startup page tables, enables the MMU, and transitions the pc from physical to virtual addresses. 6. Userland (file system built by buildworld) ARM platform related codes are in /usr/src/sys/arm. I used as a template files that was in /usr/src/sys/arm/tegra. Later tegra support was removed from src tree as it wasn't complete. Steps: 1. Created allwinner directory /usr/src/sys/arm/allwinner. 2. Then basically depending from SoC we need few files first in that directory. Please change a10 maybe to SoC name you have. a10_common.c a10_machdep.c files.a10 std.a10 Depending from SoC, maybe needs additional drivers if there are no existing drivers like timer, or interrupt controller, or UART. 3. In a10_common.c there should be decode code that checks compatibility of interrupt controller to FDT (Flattened Device Tree) 4. files.a10 should contain necessary src files needed for the SoC. These will be used when compiling kernel. 5. std.a10 should contain CPU type, if kernel.bin will be used then it needs to have PHYSADDR, KERNPHYSADDR. Also this file should have KERNVIRTADDR. KERNVIRTADDR is the virtual address at which the kernel is loaded (and linked). KERNPHYSADDR is the physical address at which the kernel is loaded. PHYSADDR is the physical address of the RAM. SoC Technical Reference Manual should contain these information. 6. a10_machdep.c should contain virtual address mapping for devices that will be used during boot. For instance: ... arm_devmap_add_entry(0x01C00000, 0x00400000); /* 4MB */ ... Here 0x01C00000 is the physical address of uart. It should be 1mb aligned. This file also should contain cpu reset call if exists such code. 7. Next we need kernel config file. For this case I used /usr/src/sys/arm/conf/BEAGLEBONE as a template for CUBIEBOARD. 8. Next we need FDT files in /usr/src/sys/boot/fdt/dts/arm/. First it was named as cubieboard.dts, but later separated common device informations to sun4i-a10.dtsi file and then included it in cubieboard.dts file. Device tree files should have necessary device informations such as physical addresses of device registers etc. For more information please look http://www.devicetree.org/Main_Page 9. Necessary steps to build kernel and world to test on board is described here: https://wiki.freebsd.org/FreeBSD/arm/Cubieboard Above wiki is for running and testing FreeBSD on Cubieboard, and it would be similar for other ARM SoC cases. If necessary drivers exist and physical/virtual addresses related to SoC and its device register addresses are correct FreeBSD should boot successfully on that board. If there isn't existing uart driver for the SoC, no output will be printed on display. In best case, you have an ns16550 compatible UART. Worst case, you have to write a custom driver without using the uart framework. There could be other problems related to boot. Common problems: In: serial Out: serial Err: serial Hit any key to stop autoboot: 0 soc# fatload mmc 0 0x40200000 kernel reading kernel 4598153 bytes read in 261 ms (16.8 MiB/s) soc# go 0x40200000 ## Starting application at 0x40200000 ... And then nothing. . . Things that can go wrong: 1. Load the kernel to the wrong location 2. Branch to the wrong address: - An address outside the kernel - The start of the kernel load address - A random address within the kernel 3. Incorrect PHYSADDR, KERNPHYSADDR or KERNVIRTADDR. Depends whether kernel or kernel.bin is used. 4. No Flattened Device Tree blob 5. Incorrect UART driver 6. Maybe more Solution for 1: # readelf -h \ /usr/obj/arm.armv6/src/sys/CUBIEBOARD/kernel ELF Header: Magic: 7f454c46010101000000000000000000 ... Entry point address: 0xc0200100 The physical entry point is: entrypoint = 0xc0200100 − KERNVIRTADDR + KERNPHYSADDR = 0xc0200100 − 0xc0200000 + 0x40200000 = 0x40200100 Solution for 2: Fix by jumping to the correct location In: serial Out: serial Err: serial Hit any key to stop autoboot: 0 soc# fatload mmc 0 0x40200000 kernel reading kernel 4598153 bytes read in 261 ms (16.8 MiB/s) soc# go 0x40200100 ## Starting application at 0x40200100 ... And then nothing. . . So solution would be to find some way to output data e.g. JTAG, UART, LED. When you know nothing about SoC and there isn't any codes (linux, u-boot etc) JTAG adapter maybe needed. It could be expensive and using it can be complicated. JTAG adapter is also helpful when you brick the board's bootloader. Making UART working could be challenging but there are some simple steps: 1. Ensure the UART clocks are setup (they most likely will be setup by U-Boot) 2 Find the UART output register 3 Write a character to the UART register For a ns16550 UART the data register, for writing, is at offset 0. I needed to put uart related line in /usr/src/sys/arm/arm/locore.S. #define DEBUG_LOC ldr r0, =0x01c28000; ldr r1, =0x65; str r1, [r0] where 0x65 is 'e' and 0x01c28000 is uart address (for writing). This should be put earlier in locore.S. Then DEBUG_LOC should be put after ASENTRY_NP(_start). After compiling and building kernel we need to try this kernel. If it prints 'e' on display then it means kernel is loaded. If it stops printing 'e' then we need to check further. There are options in kernel config that enables early printf. a10_machdep.c should contain some early printf codes added at the end: /* * Early putc routine for EARLY_PRINTF support. To use, add to kernel config: * option SOCDEV_PA=0x01c00000 * option SOCDEV_VA=0x01c00000 * option EARLY_PRINTF * Please note it should be aligned 1mb. */ static void a10_early_putc(int c) { volatile uint32_t *UART_USR = (uint32_t *)0x01c2807c; volatile uint32_t *UART_TX_REG = (uint32_t *)0x01c28000; const uint32_t UART_TRANSMIT_FIFO_NOT_FULL = (1 << 1); while((*UART_USR & UART_TRANSMIT_FIFO_NOT_FULL) == 0) continue; *UART_TX_REG = c; } early_putc_t *early_putc = a10_early_putc; Also add following to kernel config and compile: option SOCDEV_PA=0x01c00000 option SOCDEV_VA=0x01c00000 option EARLY_PRINTF Then you need to add some printf inside initarm() of /usr/src/sys/arm/arm/machdep.c, but before setttb(kernel_l1pt.pv_pa) as this switches pagetable so the uart map would be no longer valid. You can disable setttb() add printf after it, but printf may not work between setttb and cpu_tlb_flushID. Also printf may not work between setttb and cninit. cninit() inits console which means further it is related to uart. So all above means: - Early boot code is fragile - Easy to make mistakes - Can be difficult to debug - Not documented If early printf works and then somehow current ns16550 driver doesn't work then maybe something is missing from current driver. Like in case of Rockchip RK3288 SoC it uses Synopsys DesignWare ns16550 compatible uart which requires 32bit access to its registers and current uart.h has only 8bit access methods. In case of A10/A20 uart, it needed some special codes related to busy_detect, and broken_txfifo. If there is no existing uart driver then maybe easier to write simple console driver like /usr/src/sys/arm/allwinner/console.c. If register addresses are correct then simple console driver should print FreeBSD boot messages up to the following (this assumes root mount works, which means there should be either mmc/sd, usb ehci or network drivers (network mount) exist): ... warning: no time-of-day clock registered, system time will not be set accurately ... Or if no mmc/sd, usb ehci, or network drivers exist then it should stop showing mountroot prompt. This is in case if interrupt controller driver and timer driver exist and working. If not then it should stop saying there is no timer driver. If you wrote timer driver and if DELAY function is wrong then boot will stop at some places where DELAY is used. For instance USB uses a lot DELAY function. Timer driver usually has time counter and event timer and DELAY implementations. Once there are interrupt controller and timer working there is need to write uart driver or make current uart driver working. Once uart driver is working, as said before, there should be either mmc/sd, usb ehci or network drivers exist and working. That way it would be possible to mount rootfs and boot gets up to the login prompt. Depending from SoC, those controllers may need new drivers, or some clock and power related settings like GPIO related settings etc. USB devices usually require gpio pin setting to give power.