If you would like to use the new Make flow (we dropped Bazel!), here is a tutorial on how test cert-manager on a non-kind cluster such as GKE or OpenShift. In this tutorial, I assume that you have cloned cert-manager/cert-manager and that you have a shell session open in that folder. I also assume that your GKE or OpenShift cluster has access to the Docker Hub registry (i.e., you cluster has access to the internet).
You will need docker, helm, and crane installed. If you are on macOS or on Linux, you
can install helm and crane using Homebrew:
brew install helm craneFirst, build the images:
make -j8 bin/containers/cert-manager-{controller,acmesolver,cainjector,webhook,ctl}-linux-amd64.tarNow, go to https://hub.docker.com and create a registry named cm (for "cert-manager"). We will be pushing the
5 images to that registry under different tags.
🌟 For information, the image names look like this:
maelvls/cm:controller-v1.8.0-alpha.0-17-g719a48996b1b1c maelvls/cm:acmesolver-v1.8.0-alpha.0-17-g719a48996b1b1c maelvls/cm:cainjector-v1.8.0-alpha.0-17-g719a48996b1b1c maelvls/cm:webhook-v1.8.0-alpha.0-17-g719a48996b1b1c maelvls/cm:ctl-v1.8.0-alpha.0-17-g719a48996b1b1c
Let us set a variable with the repository name in Docker Hub:
REPO=maelvls/cmLog into Docker Hub:
docker login docker.io -u maelvlsPush the images (the REPO env var that we have set will be used here):
for i in bin/containers/cert-manager-{controller,acmesolver,cainjector,webhook,ctl}-linux-amd64.tar; do \
crane push $i $REPO:$(tar xfO $i manifest.json | jq ".[0].RepoTags[0]" -r | perl -ne '/cert-manager-(.*)-amd64:(.*)/ && print "$1-$2"'); \
doneFinally, let us build the Helm chart, and then install cert-manager with Helm (REPO is also required here):
# First, build the Helm chart tarball:
make bin/cert-manager.tgz
# Then, use that Helm chart:
helm upgrade --install cert-manager bin/cert-manager.tgz --create-namespace -n cert-manager --wait \
--set image.repository=$REPO \
--set cainjector.image.repository=$REPO \
--set webhook.image.repository=$REPO \
--set startupapicheck.image.repository=$REPO \
--set image.tag="$(tar xfO bin/containers/cert-manager-controller-linux-amd64.tar manifest.json | jq '.[0].RepoTags[0]' -r | perl -ne '/cert-manager-(.*)-amd64:(.*)/ && print "$1-$2"')" \
--set cainjector.image.tag="$(tar xfO bin/containers/cert-manager-cainjector-linux-amd64.tar manifest.json | jq '.[0].RepoTags[0]' -r | perl -ne '/cert-manager-(.*)-amd64:(.*)/ && print "$1-$2"')" \
--set webhook.image.tag="$(tar xfO bin/containers/cert-manager-webhook-linux-amd64.tar manifest.json | jq '.[0].RepoTags[0]' -r | perl -ne '/cert-manager-(.*)-amd64:(.*)/ && print "$1-$2"')" \
--set startupapicheck.image.tag="$(tar xfO bin/containers/cert-manager-ctl-linux-amd64.tar manifest.json | jq '.[0].RepoTags[0]' -r | perl -ne '/cert-manager-(.*)-amd64:(.*)/ && print "$1-$2"')" \
--set installCRDs=true \
--set featureGates="AdditionalCertificateOutputFormats=true\,ExperimentalCertificateSigningRequestControllers=true\,ServerSideApply=true" \
--set "webhook.extraArgs={--feature-gates=AdditionalCertificateOutputFormats=true}" \
--set "cainjector.extraArgs={--feature-gates=}" \
--set "extraArgs={--acme-http01-solver-image=$REPO:$(tar xfO bin/containers/cert-manager-acmesolver-linux-amd64.tar manifest.json | jq '.[0].RepoTags[0]' -r | perl -ne '/cert-manager-(.*)-amd64:(.*)/ && print "$1-$2"')}" >/dev/nullAt this point, cert-manager should be ready to be used.
Imagine that you are trying to implement a new feature in cert-manager. Since the end-to-end suite takes a long time to run (approximately 30 minutes), you would like to run a specific end-to-end test. In this section, you will learn:
- To figure out which end-to-end test might be of interest to you,
- To run this single test.
You might be wondering: where can I find where the go build command is
being run, and where is docker build? Let us take the example of building
the controller image tarball that was shown above:
make bin/containers/cert-manager-acmesolver-linux-amd64.tarTo see the list of intermediate "targets" (i.e., the intermediate steps)
that make is calling to get to that image tarball, you can run:
make clean
make bin/containers/cert-manager-controller-linux-amd64.tar --dry-run --debug \
| perl -pe 's/^( *(?:Must remake target|Successfully remade target|File .* does not exist))/# \1/g'The output shows the steps (the character # was added to every "debug"
line for readability purposes):
# File 'bin/containers/cert-manager-controller-linux-amd64.tar' does not exist.
# File 'bin/scratch/build-context/cert-manager-controller-linux-amd64/controller' does not exist.
# File 'bin/server/controller-linux-amd64' does not exist.
# File 'bin/server' does not exist.
# Must remake target 'bin/server'.
mkdir -p bin/server
# Successfully remade target file 'bin/server'.
# Must remake target 'bin/server/controller-linux-amd64'.
GOOS=linux GOARCH=amd64 CGO_ENABLED=0 GOMAXPROCS= \
go build -o bin/server/controller-linux-amd64 -ldflags '-w -s' -trimpath cmd/controller/main.go
# Successfully remade target file 'bin/server/controller-linux-amd64'.
# File 'bin/scratch/build-context/cert-manager-controller-linux-amd64' does not exist.
# Must remake target 'bin/scratch/build-context/cert-manager-controller-linux-amd64'.
mkdir -p bin/scratch/build-context/cert-manager-controller-linux-amd64
# Successfully remade target file 'bin/scratch/build-context/cert-manager-controller-linux-amd64'.
# Must remake target 'bin/scratch/build-context/cert-manager-controller-linux-amd64/controller'.
ln -f bin/server/controller-linux-amd64 bin/scratch/build-context/cert-manager-controller-linux-amd64/controller
# Successfully remade target file 'bin/scratch/build-context/cert-manager-controller-linux-amd64/controller'.
# File 'bin/scratch/build-context/cert-manager-controller-linux-amd64/cert-manager.license' does not exist.
# File 'bin/scratch/cert-manager.license' does not exist.
# File 'bin/scratch/license.yaml' does not exist.
# File 'bin/scratch' does not exist.
# Must remake target 'bin/scratch'.
mkdir -p bin/scratch
# Successfully remade target file 'bin/scratch'.
# Must remake target 'bin/scratch/license.yaml'.
sed -e "s/YEAR/2021/g" < hack/boilerplate/boilerplate.sh.txt > bin/scratch/license.yaml
# Successfully remade target file 'bin/scratch/license.yaml'.
# File 'bin/scratch/license-footnote.yaml' does not exist.
# Must remake target 'bin/scratch/license-footnote.yaml'.
echo -e "# To view licenses for cert-manager dependencies, see the LICENSES file in the\n# cert-manager repo: https://github.com/cert-manager/cert-manager/blob/491223c93cb863d49efe54c09daedcaea13aaa22/LICENSES" > bin/scratch/license-footnote.yaml
# Successfully remade target file 'bin/scratch/license-footnote.yaml'.
# Must remake target 'bin/scratch/cert-manager.license'.
cat bin/scratch/license.yaml bin/scratch/license-footnote.yaml > bin/scratch/cert-manager.license
# Successfully remade target file 'bin/scratch/cert-manager.license'.
# Must remake target 'bin/scratch/build-context/cert-manager-controller-linux-amd64/cert-manager.license'.
ln -f bin/scratch/cert-manager.license bin/scratch/build-context/cert-manager-controller-linux-amd64/cert-manager.license
# Successfully remade target file 'bin/scratch/build-context/cert-manager-controller-linux-amd64/cert-manager.license'.
# File 'bin/scratch/build-context/cert-manager-controller-linux-amd64/cert-manager.licenses_notice' does not exist.
# File 'bin/scratch/cert-manager.licenses_notice' does not exist.
# Must remake target 'bin/scratch/cert-manager.licenses_notice'.
cp bin/scratch/license-footnote.yaml bin/scratch/cert-manager.licenses_notice
# Successfully remade target file 'bin/scratch/cert-manager.licenses_notice'.
# Must remake target 'bin/scratch/build-context/cert-manager-controller-linux-amd64/cert-manager.licenses_notice'.
ln -f bin/scratch/cert-manager.licenses_notice bin/scratch/build-context/cert-manager-controller-linux-amd64/cert-manager.licenses_notice
# Successfully remade target file 'bin/scratch/build-context/cert-manager-controller-linux-amd64/cert-manager.licenses_notice'.
# File 'bin/release-version' does not exist.
# File 'FORCE' does not exist.
# Must remake target 'FORCE'.
# Successfully remade target file 'FORCE'.
# Must remake target 'bin/release-version'.
test "v1.7.0-beta.0-258-g491223c93cb863" == "" || echo v1.7.0-beta.0-258-g491223c93cb863 > bin/release-version
# Successfully remade target file 'bin/release-version'.
# File 'bin/containers' does not exist.
# Must remake target 'bin/containers'.
mkdir -p bin/containers
# Successfully remade target file 'bin/containers'.
# Must remake target 'bin/containers/cert-manager-controller-linux-amd64.tar'.
docker build --quiet \
-f hack/containers/Containerfile.controller \
--build-arg BASE_IMAGE=gcr.io/distroless/static@sha256:49f33fac9328ac595cb74bd02e6a186414191c969de0d8be34e6307c185acb8e \
-t cert-manager-controller-amd64:v1.7.0-beta.0-258-g491223c93cb863 \
bin/scratch/build-context/cert-manager-controller-linux-amd64/ >/dev/null
docker save cert-manager-controller-amd64:v1.7.0-beta.0-258-g491223c93cb863 -o bin/containers/cert-manager-controller-linux-amd64.tar >/dev/null
# Successfully remade target file 'bin/containers/cert-manager-controller-linux-amd64.tar'.The following corresponds to the controller binary:
# Must remake target 'bin/server/controller-linux-amd64'.
GOOS=linux GOARCH=amd64 CGO_ENABLED=0 GOMAXPROCS= \
go build -o bin/server/controller-linux-amd64 -ldflags '-w -s' -trimpath cmd/controller/main.goNow, where is the target bin/server/controller-linux-amd64 located in the
Makefile? It is possible to search for it. This target is defined in the
file make/server.mk:
$ grep bin/server/controller-linux-amd64 -R Makefile **/*.mk --files-with-matches
make/server.mk