Setting up Arch Linux ARM64 for QEMU guide.

Getting x86_64 guests in QEMU is quite easy (with the PITA of setting up cmdline flags), but doing so for ARM(64) based guests does tend to get more involved with additional setup, and often times it’s not straightforward and confusing. Most tutorials involving setting up ARM64 guests is either non existent, or outdated… While we’ve seen tutorials for doing it on tools like UTM for Apple Silicon for running ARM guests, I do want to set it up myself via writing my own QEMU launch scripts.

I’ve learned, that despite most of the time you need to grab the kernel and initramfs images from guest to host and use -kernel and -initrd flags, you can just easily use efistub and EDK2 to facilitate the boot process as normal, without having to carry seperate kernel and init files.

Setting up however does involve offline bootstrapping which means we would need to directly mount and partition disk images within a running Linux installation, without using Arch ISOs (because its non existent for ARM64), and one time dealing with running EFI shell commands.

Warnings

#

Preparation and bootstrapping.

#

This section explains how I prepared the disk image and bootstrap alarm

Assuming we’re running on a Linux host, we need:

• QEMU NBD Package (This is what I’m using)
  • You also need a kernel with nbd support. This can be checked and loaded by running modprobe nbd max_part=8 (max_part means here is maximum partitions, in this case it’s 8 but we only need 3 partitions)
  • Alternatively, if you want to use a fixed disk setup, you can simply use qemu-img to create a raw image, losetup, cfdisk, and mount it directly. No need for qemu-nbd for this case.
• Partitioning tool of your choice, such as: cfdisk.
• Root privileges
• (FOR x86_64 hosts) qemu-user-static package and systemd-binfmt or binfmt-support package to make it easier to handle different binary formats and register QEMU binaries if the package has binfmt rule on it using services, on Arch, you also need qemu-user-static-binfmt.

Preparing the disk image

#

We first create an image using the command:

qemu-img create -f qcow2 arch.qcow2 40g # You can change the size but we recommend the minimum of 10GB in size

This creates a dynamically allocated disk (tiny file size and growing). Never choose any formats but qcow2 or raw, as any other formats would render the vdisk unmountable without using special tools which is rare.

After the disk is created, we mount it using qemu-nbd as root

sudo modprobe nbd max_part=8 # if you haven't already
sudo qemu-nbd -c /dev/nbd0 -f qcow2 arch.qcow2

Partitioning and formatting

#

After running qemu-nbd command, the /dev/nbd0 is now acting as a block device node which is similar to sda or nvme0n1, which can be manipulated using partitioning tools.

Pick your poison, in this case I’m using cfdisk.

It will ask you what type of disk, choose gpt.

Then create the following partitions with types sequentially:

• EFI System Partition (for booting) - 512MB - EFI system

  NOTE: Use EFI system type, not Linux Filesystem as mistaken in the screenshot.

• Swap (optional but recommended) - 1GB - Linux swap
• Root Partition - 38GB (or the rest of the disk space) - Linux Filesystem.

Write and quit.

Partitions now appear as:

• /dev/nbd0p1 - EFI system partition
• /dev/nbd0p2 - Swap
• /dev/nbd0p3 - Root Filesystem

Then format the partitions using the following commands:

• EFI system partition - sudo mkfs.fat -F 32 /dev/nbd0p1
• Swap - sudo mkswap /dev/nbd0p2
• Root filesystem - sudo mkfs.ext4 /dev/nbd0p3

Bootstrapping

#

After configuring partitions, we then provision ALarm rootfs to the root partition.

Instead of using pacstrap, you need to download and decompress a tarball. Download the latest generic Arch Linux ARM64 image here

Then before we extract the rootfs, we need to mount the root partition somewhere. In this case, /mnt/archroot. Then we can extract the rootfs there.

sudo mkdir -p /mnt/archroot
sudo mount /dev/nbd0p3 /mnt/archroot
sudo tar -xpvf ArchLinuxARM-aarch64-latest.tar.gz -C /mnt/archroot

After extraction, we might also need to setup ESP mountpoint, in accordance to archwiki you can mount the ESP partition of your liking, however in my case, it’s very simple and I don’t want to get into efistub-capable kernel updates sync just yet… which we mount the ESP to /esp and simply copy the kernel and initramfs.

We then mount the ESP

sudo mkdir /mnt/archroot/esp
sudo mount /dev/nbd0p1 /mnt/archroot/esp

Chrooting and system preparation

#

This part is where we chroot into a newly bootstrapped system. Rather than using arch-chroot, we simply mount and chroot as normal.

# Assuming you mounted an ESP partition before as following this *guide*
ROOTFS=/mnt/archroot
sudo -s
systemctl restart systemd-binfmt.service
mount -t proc proc $ROOTFS/proc
mount -t sysfs sys $ROOTFS/sys
mount --rbind /dev $ROOTFS/dev
chroot $ROOTFS /bin/su -l

chroot here.

Updating the system

#

Once you’re in arch chroot… update and install the essential packages

pacman-key --init
pacman-key --populate
pacman -Syu sudo nano networkmanager efibootmgr

This is, in most cases it will also update the linux package and rebuild initramfs which will take time especially using qemu-user.

After it’s done updating, you do need to copy the files from /boot with Image, initramfs-linux.img and initramfs-linux-fallback.img to esp directory, in my case.. These files will be placed at /esp/EFI/arch

System configuration

#

The same thing as configuring and personalizing your Arch Linux system, you can prepare the system as usual like locales, timezones, and packages.

There are things that you do need to configure.

Fstab

#

You do atleast need to configure fstab more importantly making sure that root partition / is mounted properly, and swap.

Optionally you can also set your ESP mountpoint within fstab, I prefer mounting /esp at boot so I can easily update the kernel with efistub used for efi matching the one installed from system (you can refer to guide here for more information). If you’re not planning to automatically mount ESP at boot, you can always manually mount partition 1 later.

Before you begin, you need to check the UUID of the partitions to make sure it’s always properly mounted, as long it can identify and load these partitions within the disk.

# Assuming you're in chroot, as a root user
blkid /dev/nbd0p1 # ESP (Optional, depending on how you set it up)
blkid /dev/nbd0p2 # Swap (Only if you setup swap)
blkid /dev/nbd0p3 # Root Partition

# Output
/dev/nbd0p1: UUID="DA4E-01DF" BLOCK_SIZE="512" TYPE="vfat" PARTUUID="d137ea7d-73a0-4136-8286-cf9e0241c520"
/dev/nbd0p2: UUID="46f66a8a-47e9-4106-af13-b286a7d2f91d" TYPE="swap" PARTUUID="1ed8c07a-0a75-4105-81e8-32d650132f69"
/dev/nbd0p3: UUID="9c61b6b2-0076-47cb-ade2-9d25e6f17820" BLOCK_SIZE="4096" TYPE="ext4" PARTUUID="9ac65e83-4a15-42f1-8452-6bd8465749f2"

Once you obtained the UUIDs, you can configure fstab as:

UUID=DA4E-01DF /esp vfat defaults 0 2
UUID=46f66a8a-47e9-4106-af13-b286a7d2f91d none swap defaults 0 0
UUID=9c61b6b2-0076-47cb-ade2-9d25e6f17820 / ext4 defaults 0 1

You can always customize options

Networking

#

Always enable NetworkManager to automatically setup network at boot. Natively recognizes virtio-net-pci

systemctl enable NetworkManager

Serial Console Access

#

In some cases, ARM64 guests may not be able to properly use a display like virtio-gpu-pci, ramfb, as it may not work in all cases due to missing drivers shipped. As we are required to use -nographic, we do need systemd to be bought up to the terminal during boot, in this case we must utilize ttyS0 or ttyAMA0 serial devices which is connected to the primary QEMU serial0 interface (or nographic), so we need to make sure we can login to Arch properly.

Enable serial-getty for ttyS0 and ttyAMA0

systemctl enable serial-getty@ttyS0.service
systemctl enable serial-getty@ttyAMA0.service

Not done yet, this is only to make sure systemd can be bought to the serial0 interface, but the kernel must have cmdline option console=ttyS0,115200n8 console=ttyAMA0,115200n8 which I’ll explain later.

The reason we need to enable both is despite ttyAMA0 is primarily for ARM hardware, in most cases within a virt machine type the serial0 is exposed as ttyS0. To make things confusing, some distributions like Debian would typically use ttyAMA0 interface, we can’t be sure about everything.

SSH

#

If serial console access may not work out, you can also configure SSH… and you can login to as alarm user

systemctl enable sshd.service

Note that root password authentication is disabled, it’s recommended to allow wheel group in sudoers using EDITOR=nano visudo. You’ll be able to login as n:alarm/p:alarm with sudo access through SSH.

This works only if you do port forwarding exposing port 22 mapping to higher port number for user networking or bridge/TAP interface that lets you access the port directly through interface.

YOU DO NOT

#

• Configure GRUB bootloader, instead… we use efistub which is simply placing the kernel and initramfs to the ESP.

First boot setup

#

Once you configured your arch system with the prerequistes above. We then

Boot

#

Download EDK2 images, I recommend extracting the package qemu-efi-aarch64 deb file using ar or file-roller

You only need to grab files from /usr/share/AAVMF/ the following:

• AAVMF_CODE.no-secboot.fd
• AAVMF_VARS.fd

Both files must have 64M-67M file size, you do not need to perform dd commands. The latter file is important as this is where we persist the boot settings, consider the variable file as nvram and unique to this installation. Never reuse the EFI variables file.

First boot quirks

#

One thing to realize is… we haven’t setup EFI boot entry using efibootmgr during chroot step (which is important to prevent tampering your existing EFI settings).

By default, we won’t be able to get past through Arch boot process… However:

This means, we need to interact with EFI shell for once to manually boot (If you came back here then congrats, efibootmgr failed for some reason)

We need to use -display gtk so we can access serial0 through` View > serial0 radio toggle.

Here is my launch command line:

# Replace files like firmware and qcow2 with actual ones
# Access SSH via ssh -p 8022 alarm@localhost

..\QEMU\qemu-system-aarch64.exe `
    -M virt `
    -m 2048 `
    -cpu cortex-a72 `
    -accel tcg `
    -netdev user,hostfwd=tcp::8022-:22,id=eth0 `
    -device virtio-net-pci,netdev=eth0 `
    -display gtk `
    -device virtio-gpu-pci `
    -drive if=pflash,format=raw,readonly=on,file=Firmware\AAVMF_CODE.no-secboot.fd `
    -drive if=pflash,format=raw,file=Firmware\AAVMF_VARS.fd `
    -device virtio-scsi-pci,id=scsi0 `
    -drive file=arch.qcow2,format=qcow2,if=none,media=disk,id=scsi0 `
    -device scsi-hd,drive=scsi0,bus=scsi0.0,bootindex=1 `
    -device nec-usb-xhci,id=xhci `
    -device usb-kbd,bus=xhci.0 

Navigating the calm before the storm

#

Once you ran QEMU with GTK window… you will see Tianocore logo, and it’s very normal to not boot into anything.. BUT don’t panic, you need to enter the EFI shell by pressing any key and selecting EFI shell from the boot menu

Within the EFI shell, you will need to manually execute commands. You need to navigate the Image or kernel to be executed. The fs0: drive points to the found first ESP on the disk.

Depending on how you setup ESP, in my case, it’s \efi\arch\ where the Image EFI executable is located, you will need to adjust paths.

Once you’re in EFI shell, you typically navigate using DOS-like syntax to your ESP where the kernal is located.

fs0:
cd efi
cd arch

This is where Image and initramfs-linux.img is located… once you’re in CWD with both files. You run the Linux kernel as

:: Replace root=UUID=9c61b6b2-0076-47cb-ade2-9d25e6f17820 with your own partition UUID
:: Replace initrd=\efi\arch\initramfs-linux.img to the actual ESP structure where your initramfs is located
Image root=UUID=9c61b6b2-0076-47cb-ade2-9d25e6f17820 rw initrd=\efi\arch\initramfs-linux.img console=ttyS0,115200n8 console=ttyAMA0,115200n8 systemd.tty.term.ttyS0=xterm-256color systemd.tty.term.ttyAMA0=xterm-256color
:: We use xterm-256color so later we can use serial directly to our favorite terminal emulators like Ghostty or Alacritty.

And once you execute it, switch to serial0 interface using View > serial0 to see the actual boot process. There should be no kernel panics, it only happens for misconfigured initramfs and potentially misconfigured fstab/root partition.

You should see a systemd boot sequence.

You can alternatively use EFI shell to directly add boot entries to EFI variables, but I haven’t tested yet. See: https://blog.stigok.com/2018/06/04/set-efi-boot-entries-uefi-shell.html and skip the efibootmgr part. Then proceed to here

After it asks me to login

#

Congrats! You can either login directly to serial0 or use ssh -p 8022 alarm@localhost

We’re not done yet

#

One last thing is we configure the boot entry for Arch Linux ARM using efibootmgr, so that we don’t have to manually deal with entering EFI shell ever again and typing everything.

Note that EFI vars file is needed as explained above.

As alarm user, assuming with sudo access granted.. run:

sudo efibootmgr -c \
  -d /dev/sda \
  -p 1 \
  -l '\EFI\ARCH\Image' \
  -u 'root=UUID=9c61b6b2-0076-47cb-ade2-9d25e6f17820 rw initrd=\EFI\ARCH\initramfs-linux.img console=ttyS0,115200n8 console=ttyAMA0,115200n8 systemd.tty.term.ttyS0=xterm-256color systemd.tty.term.ttyAMA0=xterm-256color'

If you’re using virtio-blk-pci, replace sda with vda
Again replace \EFI\ARCH path to actual ESP structure you setup

Reboot and see if it can automatically boot

#

Do not do system_reset, shutdown the guest first then run the launch script again unchanged, and check if EFI doesn’t complain missing boot / you able to access serial0 and automatically see systemd logs.

If you see the system boots, congrats! You now have ready to boot Arch ARM64 Image.

If however you still had to enter EFI shell again, check here

When everything goes well

#

If you’re able to boot automatically and have your Arch system set up. You can now directly use the emulated system through terminal.

With the launch script using nographic, supports multiplexer, and passes signals like CTRL-C to guest rather than qemu process.

#!/bin/bash

qemu-system-aarch64 \
    -M virt \
    -m 2048 \
    -cpu cortex-a72 \
    -accel tcg \
    -netdev user,hostfwd=tcp::8022-:22,id=eth0 \
    -device virtio-net-pci,netdev=eth0 \
    -device virtio-gpu-pci \
    -object rng-random,filename=/dev/urandom,id=rng0 \
    -device virtio-rng-pci,rng=rng0 \
    -nographic \
    -chardev stdio,mux=on,id=char0,signal=off \
    -serial chardev:char0 \
    -mon chardev=char0,mode=readline \
    -drive if=pflash,format=raw,readonly=on,file=Firmware/AAVMF_CODE.no-secboot.fd \
    -drive if=pflash,format=raw,file=Firmware/AAVMF_VARS.fd \
    -device virtio-scsi-pci,id=scsi0 \
    -drive file=arch.qcow2,format=qcow2,if=none,media=disk,id=scsi0 \
    -device scsi-hd,drive=scsi0,bus=scsi0.0,bootindex=1 \
    -device nec-usb-xhci,id=xhci \
    -device usb-kbd,bus=xhci.0
    #-drive file=alpine-virt-3.22.2-aarch64.iso,if=none,media=cdrom,id=cdrom0 `
    #-device scsi-cd,drive=cdrom0,bus=scsi0.0,bootindex=2 `

Here’s the boot process and successful login:

Some tips

#

• When the terminal output looks garbled, you need resize command which you can do by installing xorg-xterm package and running resize; reset which this command ensures it aligns with the serial terminal state such as height and width. As much as possible, use SSH.

• If you have M-series Mac, you can use QEMU with hvf accelerator or kvm for Linux.

• For graphical applications, we recommend simply using tigervnc (with port forwarding thru QEMU or SSH)

In summary

#

• Installation and bootstrapping is typically done in any online Linux installation with root access.

• EDK2 firmware is used to be able to boot into systems with EFI boot structure which often involves setting up ESP and placing boot files there, it’s not possible to use MBR/BIOS as the concept lives in x86 systems. Since Grub is uncommon we instead use a kernel with EFIstub which is enabled in linux package and using the kernel as efi bootlader binary. First time boot is not straightforward.

• Direct graphical display through SDL is not possible, since using display drivers like ramfb or virtio-gpu-pci are often a hit or miss depending how OSes ships display drivers, ARM boards typically doesn’t have VGA, in this case, serial is necessary to login to system or SSH.