Transforming Development Workflow: Cross-Building Go Binaries with GitHub Actions

Introduction

In this article, we’ll explore how to set up a robust GitHub Actions workflow to automate the cross-building of Go binaries with CGO enabled. We’ll cover the essential steps, from configuring your environment to handling dependencies, ensuring you can effortlessly generate executables for multiple operating systems and architectures. By the end of this tutorial, you’ll have a solid understanding of leveraging GitHub Actions for cross-building Go applications, making your development process more efficient and reliable.

So, let’s dive in and see how we can easily achieve this!

Problem

Golang has gained substantial popularity for its performance, simplicity, and robustness, particularly in cross-platform development. Its concurrency model, based on goroutines, and its powerful standard library make it an excellent choice for a wide range of applications, from web servers to network tools and even cloud-based solutions.

However, challenges arise when projects depend on C libraries or need to integrate existing C codebases. This is where CGo comes into play, enabling the inclusion and calling of C code directly from Go programs. While CGo extends Go’s capabilities significantly, it introduces complexities such as a more complicated build process, potential performance overhead, and memory safety concerns. Thus, developers must carefully weigh the benefits and drawbacks of using CGo to ensure they maintain the simplicity, portability, and safety that Go is known for. Here’s a short example:

// #include <stdio.h>

// #include <errno.h>

import “C”

This commonly creates an issue when trying to build the binary for a different architecture or OS, as some of the build toolchains may not be present on a machine (moreover, if we’re talking about containerizing and using CI/CD pipelines) like in this example:

➜  awesomeProject env GOOS=linux GOARCH=arm64 CGO_ENABLED=1 go build ./main.go

# runtime/cgo

gcc_arm64.S: Assembler messages:

gcc_arm64.S:30: Error: no such instruction: `stp x29,x30,[sp,’

gcc_arm64.S:34: Error: too many memory references for `mov’

gcc_arm64.S:36: Error: no such instruction: `stp x19,x20,[sp,’

gcc_arm64.S:39: Error: no such instruction: `stp x21,x22,[sp,’

…and so on

This is the exact issue we faced on one of our projects in DEDICATTED, and here we will describe the solution we engineered to ensure the best performance and efficiency.

Challenge

The primary goal is to create a GitHub Actions workflow that efficiently cross-builds Golang binaries with CGO_ENABLED=1. We need to ensure the process is optimized for speed and reliability. Here’s a breakdown of the tasks we’ll tackle:

• Investigate a couple of approaches to deal with this issue
• Write a GitHub Actions workflow to cross-build the project for multiple architectures.
• Implement base images with dependencies (and some cross-build tools) to speed up the build process.

Solution

The containerized approach

Depending on the environment where such an approach would be used, there are a couple of ways to deal with this kind of issue. We needed the automated pipeline to create binaries for multiple OSes and architectures for each new app version. For this small showcase, we chose these most commonly used ARCHs:

• windows/amd64 (Windows)
• linux/amd64 (Linux)
• darwin/amd64 (MacOS)
• darwin/arm64 (MacOS, ARM)

The most concise solution we found is using elastic/golang-crossbuild: a repo that builds a set of Docker images that include all necessary build toolchains inside them for each widely used OS and architecture:

Great! We have a solution right in front of us! But how does it work?

According to the README, the executable inside a container will try to launch a Makefile entry (make build) inside the given project root and build it for a specified OS/ARCH string. Here’s how it looks:

docker run -it –rm \

  -v $GOPATH/src/github.com/user/go-project:/go/src/github.com/user/go-project \

  -w /go/src/github.com/user/go-project \

  -e CGO_ENABLED=1 \

  docker.elastic.co/beats-dev/golang-crossbuild:1.16.7-armhf \

  –build-cmd “make build” \

  -p “linux/armv7”

Let’s pick the required image from the list. golang-crossbuild:main fits perfectly for linux/amd64 and windows/amd64, so we’ll use that one. 

However, as for the MacOS, a list specifies that golang-crossbuild:version-darwin only works for darwin/amd64, and there’s no mention of darwin/arm64 either. Luckily, the maintainers of this repo also create a set of Debian-based images. That list of images works quite well (although with a tradeoff of the image being a little more significant due to being Debian-based).

Let’s create a simple GitHub Actions workflow to incorporate this approach:

The cross-build binary inside the container also requires a command that it will use to build our code. For that, we made a small Makefile entry to include all our needs in one place:

Creating base images with dependencies to speed up the build process

There’s still room for improvement. If we have multiple gigabytes of dependencies, we’d not like to try to download them one with go mod download, as it would be a slow process. Instead, we can create a base image for our cross-build images containing all the required packages needed for the app to compile.

Let’s configure a Dockerfile for this:

Here, we make arguments for our required Golang version and the target image tag so that we can choose it in GitHub Actions. Let’s not forget that we also want to push our dependency images to an external image registry, and we’re using ECR. Here’s an example job for building our binaries with this approach:

This GitHub Actions job automates the process of building and pushing Docker images to ECR for different platforms. It starts by checking out the repository to ensure access to the source code, configuring AWS credentials using secrets stored in the repository’s settings, and logging in to Amazon ECR to obtain an authentication token. It builds, tags, and pushes Docker images for both Linux/Windows and Darwin (macOS) platforms. By adding build-args we can tell our Dockerfile to use our desired version and base image tag to build the image. Finally, the job logs out of Amazon ECR to clean up the session.

We also need a switch not to build these dependency images each workflow run. We can do it by tracking changes in certain files (namely go.mod, go.sum, and Dockerfile.deps) to launch a job using workflow_call. Here’s a snapshot of it in a dependency image build workflow:

Option for local builds: Using xgo package

Before discovering elastic/golang-crossbuild, we stumbled upon a neat solution called karalabe/xgo. The approach used runs the Docker images for cross-build automatically and builds the binaries for you. Unfortunately, this repository is now archived and lives on through a couple of forks, so we decided that it would be a nice solution for our local test builds but certainly not for GitHub Actions.

Generally, the usage is straightforward, as you can use xgo instead of go in the build command. We created a Makefile that would do it for us:

Conclusion

You can get complete code and see an example run in our public GitHub repository (there’s also other stuff there; check it out!): https://github.com/dedicatted/golang-cross-build-ghaction

Here’s the resulting pipeline. To speed things up even more, you can create parallel jobs for each OS and Arch! 

This solution would be helpful for many who need to integrate the Golang binary build process into the GitHub Actions pipeline.

DEDICATTED Team decided to share our knowledge with the community, and that’s a result.