Go Docker Build with Internal Shared Package

 

In the post Go Shared Library, we have reviewed a method to use a shared Go library in a single git repository. This is a case where we have multiple Go libraries in a single git repository, and some share libraries that are used by the modules. A possible folders structure for this use case is:

In this example, we have 3 modules and 2 common libraries. 

The go.mod file for module-a, includes the replace section for the internal shared libraries.

module my.company.com/example/modulea

go 1.19

replace (
   my.company.com/example/commonlib1 => ../common-lib-1
   my.company.com/example/commonlib2 => ../common-lib-2
)

require (
   my.company.com/example/commonlib1 v0.0.0-00010101000000-000000000000
   my.company.com/example/commonlib2 v0.0.0-00010101000000-000000000000
)

To build a docker image for this module, we need to add the sources of all the requirements. We can do it manually for each of module-a, module-b, and module-c. A better approach is to automatically handle this. 

We present now a script to automatically build a docker image for a Go module, including the internal module source includes. This automation is done based on the parsing of the related module go.mod file.

#!/usr/bin/env bash

#-------------------------------------
# This script builds a single GO image
#-------------------------------------

set -e

DockerRegistry="${DockerRegistry:-my-project-snapshot-local}"
ProjectVersion="${ProjectVersion:-/dev:latest}"
ScriptsFolder=$(dirname "$0")
Project=$(basename ${PWD})
EntryPoint=\\/${Project}
ArtifactName="my-project-${FolderName}"
DockerTag="${DockerRegistry}/my-project-${Project}${ProjectVersion}"


AddKubectl=false
PushImage=false

HandleCommonPackage(){
  commonPackage=$1
  sourceFolder=$(echo "${commonPackage}" | cut -c 9-)

  echo "common module ${sourceFolder}"

  if [[ ! -d ./temp-commons-go-manifest/${sourceFolder} ]]; then
    mkdir -p ./temp-commons-go-manifest/${sourceFolder}
    cp ../${sourceFolder}/go.mod ./temp-commons-go-manifest/${sourceFolder}/go.mod
    cp ../${sourceFolder}/go.sum ./temp-commons-go-manifest/${sourceFolder}/go.sum
  fi

  if [[ ! -d ./temp-commons-go-all/${sourceFolder} ]]; then
    mkdir -p ./temp-commons-go-all/${sourceFolder}
    sourceParent=$(dirname ./temp-commons-go-all/${sourceFolder})
    cp -r ../${sourceFolder} ${sourceParent}
  fi
}

ReplaceVariables(){
  sed -i "s/___PROJECT___/${Project}/g" temp-Dockerfile
  sed -i "s/___ENTRYPOINT___/${EntryPoint}/g" temp-Dockerfile

  grep "=" ./src/go.mod | cut -d= -f2 | while read line ; do HandleCommonPackage "$line" ; done
}

CleanTemp(){
  rm -f temp-Dockerfile
  rm -rf ./temp-commons-go-manifest
  rm -rf ./temp-commons-go-all
}

Build(){
  CleanTemp
  mkdir ./temp-commons-go-manifest
  mkdir ./temp-commons-go-all

  cat ${ScriptsFolder}/Dockerfile_stage1 >> temp-Dockerfile
  ReplaceVariables
  echo "COPY files /images/${Project}/files" >> temp-Dockerfile
  echo "RUN go test -race -timeout 300s ./..." >> temp-Dockerfile

  TagCompileStage="${DockerTag}-stage1"
  docker build -t "${DockerTag}" -c ${TagCompileStage} --cache-from=${DockerTag} -f temp-Dockerfile

  cat ${ScriptsFolder}/Dockerfile_stage2 >> temp-Dockerfile
  ReplaceVariables

  if [[ -d files ]]; then
      echo "COPY files /" >> temp-Dockerfile
  fi

  docker build -t "${DockerTag}" -c ${TagCompileStage} --cache-from=${DockerTag} -f temp-Dockerfile

  CleanTemp
}

Build

The script uses a two-stages build. These stages docker files templates located in Dockerfile_stage1, and Dockerfile_stage2. Notice that the script automatically replaces ___VARIABLE___ strings in the docker file.

Dockerfile_stage1

FROM golang:1.19.1 AS go-compiler

ENV GOPATH=/go GOBIN=/go/bin

# get dependencies
COPY ./src/go.mod /images/___PROJECT___/src/
COPY ./src/go.sum /images/___PROJECT___/src/
ADD ./temp-commons-go-manifest /images
WORKDIR /images/___PROJECT___/src

RUN go mod download

# compile source
ADD ./src /images/___PROJECT___/src
ADD ./temp-commons-go-all /images

RUN go build -o /output/___PROJECT___

Dockerfile_stage2

FROM ubuntu:20.04
RUN apt update
RUN apt install -y net-tools ca-certificates curl iputils-ping dnsutils
RUN update-ca-certificates

COPY --from=go-compiler /output/___PROJECT___ /___PROJECT___
WORKDIR /
ENTRYPOINT ["___ENTRYPOINT___"]

Final Note

Using this simple script, handling of the docker image creation is automatic and simple. It is a great tool to automate Go modules and Go internal libraries usage.