How To Make Rust Multi-Arch Release Easy

At Qovery, we're using Rust for 10+ projects. If you are familiar with Rust, you know how painful it is to release a binary for multiple architectures. Even if the toolchain natively supports it.

Pierre Mavro

October 10, 2023 · 6 min read

How To Make Rust Multi-Arch Release Easy - Qovery

Written by

Pierre Mavro

Pierre is an SRE and CTO of Qovery. He has 15+ years of experience in R&D, from the financial to the Ad-Tech industry; he has a strong knowledge of distributed and highly-reliable systems. He's also the MariaDB High-Performance book author.

See all articles

Engineering

Table of Contents

Rust project
GitHub Action pipeline
Goreleaser
Bonus
Conclusion

With Go, there is a fantastic tool called GoReleaser, which allows you to:

Compile for all architectures you want binaries
Create a release on your favorite Git provider
Automatically generate the changelog based on commits between the current the previous version
Manage repositories for Linux (Arch), MacOs (HomeBrew), and Windows (Scoop/Winget).
Generate binaries checksums
And many other things

Once you’ve used GoReleaser, it’s hard to go back. I looked at Rust alternatives but unfortunately didn’t find one with the expected level of features. This is why I tried to use GoReleaser with Rust and naïvely thought it would be as simple as go, as I knew Rust was also made to be multi-architecture. Unfortunately, the reality is more complex than I expected. I found several tutorials, blog posts, and discussions around it, but nothing ready to set up in 5 min.

This is why I started to make a template to build multi-architecture binaries on Rust, with GitHub Action and GoReleaser. The goal is to build binaries for:

Linux (Aarch64 & x86_64)
MacOs (Aarch64 & x86_64)
Windows (x86_64)

#Rust project

Before digging into GitHub action, I'm using a fresh, newly created Git repository, create a Rust hello world (example), and move the source into the repository:

git clone git@github.com:<username>/<repo>.git
cargo new example
mv example/* .
rmdir example

The Cargo.toml can be tuned to optimize the binary size on the release build:

[package]
name = "example"
version = "0.1.0"
edition = "2021"

[dependencies]

[profile.release]
strip = true
opt-level = "z"
codegen-units = 1

#GitHub Action pipeline

First, create a DockerHub token to pull (and avoid access being denied) a MacOS image containing the required libs for cross-compilation (thanks a lot to this project for the work).

Now that I have a Rust code example, I’m using a GitHub workflow as follows (.github/workflows/release.yaml):

on:
  # Indicates I want to run this workflow on all branches, PR, and tags
  push:
    branches: ["**"]
    tags: ["*"]
  pull_request:
    branches: [ "main" ]

env:
  # Define the rust version to use
  RUST_VERSION: 1.72.1
  # Rust build arguments
  BUILD_ARGS: "--release --all-features"
  # The binary name
  BIN_NAME: "example"
  # Docker token required to pull images from DockerHub
  DOCKER_LOGIN: ${{ secrets.DOCKER_LOGIN }}
  DOCKER_TOKEN: ${{ secrets.DOCKER_TOKEN }}

jobs:
  build:
    name: Build - ${{ matrix.platform.name }}
    # By default, runs on Ubuntu, otherwise, override with the desired os
    runs-on: ${{ matrix.platform.os || 'ubuntu-22.04' }}
    strategy:
      matrix:
        # Set platforms you want to build your binaries on
        platform:
          # Linux
            # The name is used for pretty print
          - name: Linux x86_64
            # The used Rust target architecture
            target: x86_64-unknown-linux-gnu
          - name: Linux aarch64
            target: aarch64-unknown-linux-gnu

          # Mac OS
          - name: MacOS x86_64
            target: x86_64-apple-darwin
          - name: MacOS aarch64
            target: aarch64-apple-darwin

          # Windows
          - name: Windows x86_64
            # Use another GitHub action OS
            os: windows-latest
            target: x86_64-pc-windows-msvc

    steps:
    - name: Checkout Git repo
      uses: actions/checkout@v3

    # Linux & Windows
    - name: Install rust toolchain
      if: ${{ !contains(matrix.platform.target, 'apple') }}
      uses: actions-rs/toolchain@v1
      with:
        # We setup Rust toolchain and the desired target
        profile: minimal
        toolchain: "${{ env.RUST_VERSION }}"
        override: true
        target: ${{ matrix.platform.target }}
        components: rustfmt, clippy
    - name: Build ${{ matrix.platform.name }} binary
      if: ${{ !contains(matrix.platform.target, 'apple') }}
      uses: actions-rs/cargo@v1
      # We use cross-rs if not running on x86_64 architecture on Linux
      with:
        command: build
        use-cross: ${{ !contains(matrix.platform.target, 'x86_64') }}
        args: ${{ env.BUILD_ARGS }} --target ${{ matrix.platform.target }}

    # Mac OS
    - name: Login to DockerHub
      if: contains(matrix.platform.target, 'apple')
      # We log on DockerHub
      uses: docker/login-action@v3
      with:
        username: ${{ env.DOCKER_LOGIN }}
        password: ${{ env.DOCKER_TOKEN }}
    - name: Build ${{ matrix.platform.name }} binary
      if: contains(matrix.platform.target, 'apple')
      # We use a dedicated Rust image containing required Apple libraries to cross-compile on multiple archs
      run: |
        docker run --rm --volume "${PWD}":/root/src --workdir /root/src joseluisq/rust-linux-darwin-builder:$RUST_VERSION \
        sh -c "CC=o64-clang CXX=o64-clang++ cargo build $BUILD_ARGS --target ${{ matrix.platform.target }}"
    
    - name: Store artifact
      uses: actions/upload-artifact@v3
      with:
        # Finally, we store the binary as GitHub artifact for later usage
        name: ${{ matrix.platform.target }}-${{ env.BIN_NAME }}
        path: target/${{ matrix.platform.target }}/release/${{ env.BIN_NAME }}${{ contains(matrix.platform.target, 'windows') && '.exe' || '' }}
        retention-days: 1

  release:
    name: Release
    needs: [build]
    # We run the release job only if a tag starts with 'v' letter
    if: startsWith( github.ref, 'refs/tags/v' )
    runs-on: ubuntu-22.04
    steps:
    - name: Checkout Git repo
      uses: actions/checkout@v3

    # Download all artifacts
    - uses: actions/download-artifact@v3
      with:
        path: artifacts

    # Goreleaser  
    - name: Set up Go
      uses: actions/setup-go@v4
    - name: Run GoReleaser
      uses: goreleaser/goreleaser-action@v5
      with:
        distribution: goreleaser
        version: latest
        # Run goreleaser and ignore non-committed files (downloaded artifacts)
        args: release --clean --skip=validate
      env:
        GITHUB_TOKEN: ${{ secrets.GH_TOKEN_RUST_CROSS }}

#Goreleaser

Now, the setup is made; the last part is Goreleaser. As you certainly understand, we first build binaries, then use GoReleaser to perform the packaging and release. As GoReleaser is not able to build Rust code and requires to build something, we have to create an empty Go code and build it (goreleaser.go):

package main
func main() {
}

Then, replace the generated binaries with the Rust ones with a hook calling a shell script (.goreleaser_hook.sh):

#!/usr/bin/env bash

go_arch=$1
go_os=$2
project_name=$3

# Make Go -> Rust arch/os mapping
case $go_arch in
    amd64) rust_arch='x86_64' ;;
    arm64) rust_arch='aarch64' ;;
    *) echo "unknown arch: $go_arch" && exit 1 ;;
esac
case $go_os in
    linux) rust_os='linux' ;;
    darwin) rust_os='apple-darwin' ;;
    windows) rust_os='windows' ;;
    *) echo "unknown os: $go_os" && exit 1 ;;
esac

# Find artifacts and uncompress in the corresponding directory
find artifacts -type f -name "*${rust_arch}*${rust_os}*" -exec unzip -d dist/${project_name}_${go_os}_${go_arch} {} \;

To finish, we set the GoReleaser configuration to the same architectures:

project_name: example
builds:
  - main: goreleaser.go
    goos:
      - linux
      - darwin
      - windows
    goarch:
      - amd64
      - arm64
    binary: example
    ignore:
      - goos: windows
        goarch: arm64
    hooks:
      post:
        - ./.goreleaser_hook.sh {{ .Arch }} {{ .Os }} {{ .ProjectName }}
checksum:
  name_template: "checksums.txt"
changelog:
  sort: asc
  filters:
    exclude:
      - "^docs:"
      - "^test:"

Now, push the code to the repository. Create and push a tag, and you'll see your GitHub Action processing:

You can admire the result in the Release tab when everything is finished.

As you can see, building a hello world with cross-compilation takes a longer time (Mac OS and Linux aarch64) than if you would build directly on the target architecture and operating system (Linux and Windows x86_64).

It’s up to you to find a good balance between easy repeatability and speed, depending on your needs.

#Bonus

To give you a concrete example of how GoReleaser is powerful on the packaging part, here is a configuration example of how to let it manage the HomeBrew repository. Append this configuration to your .goreleaser.yaml file:

brews:
  - name: <example>
    goarm: 6
    tap:
      owner: <tap-owner>
      name: <git-repository-name>
    url_template: "https://github.com/<username>/<example-repository-name>/releases/download/{{ .Tag }}/{{ .ArtifactName }}"
    commit_author:
      name: <Author Name>
      email: <email@address.com>
    folder: Formula
    homepage: "https://github.com/<username>/<example-repository-name>"
    description: "A description"
    skip_upload: false

#Conclusion

Combining GoReleaser, Rust, and GitHub Action makes multi-architecture compilation simpler. It still requires some work, so we have made a template to clone/fork and save time. We hope this will bring value to the beloved Rust and GoReleaser community. We made this because we love Rust at Qovery and want Rust to be easily adopted :)