Google Cloud Build Maven Build with Cache

cloudbuild.yaml

steps:
  # Grab cached M2 repo
  - name: gcr.io/cloud-builders/gsutil
    id: Get M2 Cache
    args: ['cp', 'gs://BUCKET_NAME/m2.tar.gz', 'm2.tar.gz']
  # See https://github.com/GoogleCloudPlatform/cloud-builders-community to get the tar command
  - name: gcr.io/PROJECT_ID/tar
    id: Expand M2 Cache
    args: ['xpzf', 'm2.tar.gz']
  # Optional, we had some extra settings to include
  - name: 'gcr.io/cloud-builders/gsutil'
    id: Get setting.xml
    args: ['cp', 'gs://BUCKET_NAME/settings.xml', 'custom-settings.xml']
  # The maven.repo.local setting was key
  - name: 'gcr.io/cloud-builders/mvn'
    id: Test and Build
    args: ['-Dmaven.repo.local=/workspace/.m2/repository', '--settings', 'custom-settings.xml', 'deploy']
  # Update M2 repo cache
  - name: gcr.io/PROJECT_ID/tar
    id: Compress M2 Cache
    args: ['cpzf', 'm2.tar.gz', '.m2']
  - name: gcr.io/cloud-builders/gsutil
    id: Save M2 Cache
    args: ['cp', 'm2.tar.gz', 'gs://BUCKET_NAME/m2.tar.gz']

You have to seed your bucket with a m2.tar.gz file. Adding the .m2 cache via this method sped up our build about 50%. We have about 180MB of jar files that were being downloaded from central every build.

This concept is addressed in the Google Cloud Build docs but there isn't a great example of it: https://cloud.google.com/cloud-build/docs/speeding-up-builds

Check that also: https://github.com/GoogleCloudPlatform/DataflowTemplates/blob/master/cloudbuild.yaml

thank you for this.

tried gsutil cp and the gsutil rsync .

This method is way faster (6 - 10min) vs 2 mins for a 160MB repo.

though I'm still not sure why it takes 24s to copy a 160MB, 1 zip file, intra-region. I've heard 10-20MB/s and usually around 60MB/s should be normal GCS to Cloud Build environment... any comments?

Yeah, not really sure but we see the same thing. I suspect it's some sort of process startup/cleanup time vs the actual copy. Our last build shows a 34 seconds for a 545MB zip.