Linux "Hello world" in Rust with GNU libc

Let's write a Rust program that outputs "Hello world" to stdout and compile it to a RISC-V ELF binary. Bonus points: actually run it.

If you have a RISC-V Linux installation, congratulations, everything is handled by the default configuration and toolchains (cargo init && cargo run should just work).

The following instructions assume cross-compilation on a x86_64 host machine with Ubuntu 22.04.

Packages and tools for cross-compilation

Ensure that rustup is installed. This is needed to manage rustc targets.

Make sure that riscv64gc-unknown-linux-gnu rustc target is installed:

rustup target add riscv64gc-unknown-linux-gnu

It seems to be the case that rustc targets do not try to bring their own GCC toolchains with them or guess what system packages provide it (which is reasonable, since each Linux distribution has its own non-standard packaging for cross-compilation toolchains and glibc; I wish it was not so).

For Ubuntu, make sure that the following packages are installed:

• gcc-riscv64-linux-gnu, the cross-compilation GCC toolchain for RISCV.
• libc6-riscv64-cross for a dynamically-linked RISCV version of glibc
• qemu-user to run a RISCV binary on x86_64
• patchelf in case you want to run a dynamically-linked binary on x86_64

Documentation and references:

• Cargo configuration
• Rustc command-line arguments
• The GNU linker Manual

Compilation

Create a Cargo project template in an empty directory:

$ cargo init --name hello-libc

$ cat src/main.rs
fn main() {
    println!("Hello world!");
}

Now, adjust .cargo/config.toml:

[build]
target = "riscv64gc-unknown-linux-gnu"      # build for this target by default

[target.riscv64gc-unknown-linux-gnu]        # settings for this target
linker = "riscv64-linux-gnu-gcc"            # mandatory to link the binary
runner = "qemu-riscv64"                     # to make `cargo run` work on x86\_64

linker = "riscv64gc-unknown-linux-gnu" is crucial for cross-compilation, without it rustc just tries to use whichever ld it finds in $PATH and fails miserably. I still don't understand why different GCC toolchains require mostly-the-same, but different GNU linkers.

Also, you cannot use linker = "riscv64-unknown-linux-ld" directly, since it will not be able to find -lgcc_s on its own. There might be a way to tweak this, but GNU toolchain options are pain</rant>

Now cargo build (--release) should produce a RISCV executable in target/riscv64gc-unknown-linux-gnu/debug/hello-libc.

Compiling with a statically-linked glibc

Since Rust 1.19, it is possible to link glibc statically. To make a statically-linked executable, use a somewhat cryptically named target-feature=+crt-static to rustc flags in .cargo/config.toml:

[target.riscv64gc-unknown-linux-gnu]
...
rustflags = [
    "-C", "target-feature=+crt-static",     # link glibc statically
]

Running it

Let's assume the following shell variables are set, to make snippets more human-friendly:

$ BIN_DIR=./target/riscv64gc-unknown-linux-gnu/debug
$ RV_SYS_DIR=/usr/riscv64-linux-gnu

The compiled binary, $BIN_DIR/hello-libc, can be copied to a RISC-V Linux installation and it should be able to run there (I did not verify this).

If it is statically linked, it should just work with qemu-riscv64.

If you want to run a dynamically linked RISC-V executable on a x86_64 machine, things get complicated:

$ qemu-riscv64 $BIN_DIR/hello-libc
qemu-riscv64: Could not open '/lib/ld-linux-riscv64-lp64d.so.1': No such file or directory

There is no such ELF loader ("ELF interpeter"), /lib/ld-linux-riscv64-lp64d.so.1, installed, but:

$ apt-file search ld-linux-riscv64-lp64d.so
libc6-riscv64-cross: /usr/riscv64-linux-gnu/lib/ld-linux-riscv64-lp64d.so.1

One (dirty) way to solve this is to symlink $RV_SYS_DIR/lib/ld-linux-riscv64-lp64d.so.1 into /lib/ manually and use LD_LIBRARY_PATH=$RV_SYS_DIR/lib to override system libraries.

A better way is to patch the ELF interpreter and RPATH in (a copy of) the executable:

$ patchelf $BIN_DIR/hello-libc \
    --set-interpreter $RV_SYS_DIR/lib/ld-linux-riscv64-lp64d.so.1 \
    --set-rpath $RV_SYS_DIR/lib

$ qemu-riscv64 $BIN_DIR/hello-libc
Hello world!

Troubleshooting the linking

These are some tricks I found useful to understand what was (not) going on:

• troubleshooting link failures with cargo build with -vv.

  The actual rustc command is still a hostile lump of text.

  $ cargo build -vv
     Compiling hello-libc v0.1.0 (/home/user/code/lang/arch/riscv/hello-libc)
       Running `CARGO=/home/user/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/bin/cargo CARGO_BIN_NAME=hello-libc CARGO_CRATE_NAME=hello_libc CARGO_MANIFEST_DIR=/home/user/code/lang/arch/riscv/hello-libc CARGO_PKG_AUTHORS='' CARGO_PKG_DESCRIPTION='' CARGO_PKG_HOMEPAGE='' CARGO_PKG_LICENSE='' CARGO_PKG_LICENSE_FILE='' CARGO_PKG_NAME=hello-libc CARGO_PKG_REPOSITORY='' CARGO_PKG_RUST_VERSION='' CARGO_PKG_VERSION=0.1.0 CARGO_PKG_VERSION_MAJOR=0 CARGO_PKG_VERSION_MINOR=1 CARGO_PKG_VERSION_PATCH=0 CARGO_PKG_VERSION_PRE='' CARGO_PRIMARY_PACKAGE=1 LD_LIBRARY_PATH='/home/user/code/lang/arch/riscv/hello-libc/target/debug/deps:/home/user/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib:/home/user/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib' rustc --crate-name hello_libc --edition=2021 src/main.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type bin --emit=dep-info,link -C embed-bitcode=no -C debuginfo=2 -C metadata=cac59addb6dfe60a -C extra-filename=-cac59addb6dfe60a --out-dir /home/user/code/lang/arch/riscv/hello-libc/target/riscv64gc-unknown-linux-gnu/debug/deps --target riscv64gc-unknown-linux-gnu -C linker=riscv64-linux-gnu-gcc -C incremental=/home/user/code/lang/arch/riscv/hello-libc/target/riscv64gc-unknown-linux-gnu/debug/incremental -L dependency=/home/user/code/lang/arch/riscv/hello-libc/target/riscv64gc-unknown-linux-gnu/debug/deps -L dependency=/home/user/code/lang/arch/riscv/hello-libc/target/debug/deps`
      Finished dev [unoptimized + debuginfo] target(s) in 0.62s
  

  Copying the command into $EDITOR and breaking into human-digestible lines helps.

  Alternatively, if you're in the mood for a shell vibe from the 80s, use some `sed`

  $ sed -e 's/ \(CARGO_\|LD_\|-C\|--\|-L\|rustc\|src\)/\n\1/g' < tmp/link-command.txt
  CARGO=/home/user/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/bin/cargo
  CARGO_BIN_NAME=hello-libc
  CARGO_CRATE_NAME=hello_libc
  CARGO_MANIFEST_DIR=/home/user/code/lang/arch/riscv/hello-libc
  CARGO_PKG_AUTHORS=''
  CARGO_PKG_DESCRIPTION=''
  CARGO_PKG_HOMEPAGE=''
  CARGO_PKG_LICENSE=''
  CARGO_PKG_LICENSE_FILE=''
  CARGO_PKG_NAME=hello-libc
  CARGO_PKG_REPOSITORY=''
  CARGO_PKG_RUST_VERSION=''
  CARGO_PKG_VERSION=0.1.0
  CARGO_PKG_VERSION_MAJOR=0
  CARGO_PKG_VERSION_MINOR=1
  CARGO_PKG_VERSION_PATCH=0
  CARGO_PKG_VERSION_PRE=''
  CARGO_PRIMARY_PACKAGE=1
  LD_LIBRARY_PATH='/home/user/code/lang/arch/riscv/hello-libc/target/debug/deps:/home/user/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib:/home/user/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib'
  rustc
  --crate-name hello_libc
  --edition=2021
  src/main.rs
  --error-format=json
  --json=diagnostic-rendered-ansi,artifacts,future-incompat
  --crate-type bin
  --emit=dep-info,link
  -C embed-bitcode=no
  -C debuginfo=2
  -C metadata=cac59addb6dfe60a
  -C extra-filename=-cac59addb6dfe60a
  --out-dir /home/user/code/lang/arch/riscv/hello-libc/target/riscv64gc-unknown-linux-gnu/debug/deps
  --target riscv64gc-unknown-linux-gnu
  -C linker=riscv64-linux-gnu-gcc
  -C incremental=/home/user/code/lang/arch/riscv/hello-libc/target/riscv64gc-unknown-linux-gnu/debug/incremental
  -L dependency=/home/user/code/lang/arch/riscv/hello-libc/target/riscv64gc-unknown-linux-gnu/debug/deps
  -L dependency=/home/user/code/lang/arch/riscv/hello-libc/target/debug/deps
  

  Tip: rustc -C help is your friend.

• getting verbose output from gcc with rustflags = ["-C", "link-arg=-v"] in [target.riscv64gc-unknown-linux-gnu] section of .cargo/config.toml.

  rustflags = [
    "-C", "link-arg=-v",                # make gcc more talkative
    "-C", "link-arg=-Wl,--verbose",     # make linker more talkative
  ]
  

• getting the ELF interpreter via file:

  $ file $BIN_DIR/hello-libc
  ./target/riscv64gc-unknown-linux-gnu/debug/hello-libc: ELF 64-bit LSB pie executable, UCB RISC-V, RVC, double-float ABI, version 1 (SYSV), 
  dynamically linked, interpreter /usr/riscv64-linux-gnu/lib/ld-linux-riscv64-lp64d.so.1, ...