How to parameterize a Hashicorp's Nomad Job for user-defined variables?

ParameterizeHashicorpNomadJob.md

How to parameterize a Hashicorp's Nomad Job for user-defined variables?

Description

This sample Nomad job attempts to clone a github repository from a github account, and then perform ls command to verify if it downloaded.

However, the cool thing is, it uses parameterized construct to parameterize the repository name so I can re-use this nomad job to clone any repository at runtime.

Motivation for writing

As a Nomad beginner, it took me quite some time (though, I must mention Nomad is super easy to get started with!) to fully wire together components required to parameterize a nomad job.

Therefore, writing it up so that folks like me can benefit from this.

Requirement

This tutorial assumes you have Nomad setup in your environment.

Another requirement is for job file to have the Scheduler Type specified as batch -

type = "batch"

Declare the parameterized block at the job level

You declare which variables are required and which ones are optional

parameterized {
    meta_required = ["REPOSITORY_NAME"]
    meta_optional = ["FIRST_NAME", "LAST_NAME"]
  }

Here, we have declared variable REPOSITORY_NAME as a required variable and FIRST_NAME and LAST_NAME as optional.

Reference the variables in your job file

This example attempts to download a repository whose name is parameterized. Therefore we need to reference the name like the following -

artifact {
  source = "[email protected]:rohitkothari/${NOMAD_META_REPOSITORY_NAME}"
  destination = "local/config/"
  options {
    sshkey = "_BASE64_SSH_KEY_STRING_"
  }
}

As you can see, the variable declared as REPOSITORY_NAME is available to be referenced as NOMAD_META_REPOSITORY_NAME.

This is because you Nomad treats all the user-defined variables as meta variables and thus the prefix NOMAD_META_.

Register the parameterized nomad job with Nomad server

$ nomad run ansible.nomad
Job registration successful

At this stage, the job has been registered, however it has not started any container or allocation yet.

Dispatch a new instance of registered, parameterized job with variable values

We finally dispatch the registered job (i.e. create a new running instance of our registered job) named "ansible" -

$ nomad job dispatch -meta REPOSITORY_NAME=Hello-World ansible
Dispatched Job ID = ansible/dispatch-1497641508-d0858f30
Evaluation ID     = f2fea7fe

==> Monitoring evaluation "f2fea7fe"
    Evaluation triggered by job "ansible/dispatch-1497641508-d0858f30"
    Allocation "d46fbe73" created: node "ec25b9c7", group "cache"
    Evaluation status changed: "pending" -> "complete"
==> Evaluation "f2fea7fe" finished with status "complete"

As you can see above, we dynamically inject the value of variable REPOSITORY_NAME while dispatching the job.

You can declare as many variables as you want and inject them at runtime like above.

This completes the demo of configuring a Hashicorp nomad job for user-defined variables and injecting their values dynamically at runtime.

Rest of this doc shows various status commands and the entire job file for reference.

Check the status of the dispatched job using the Dispatched Job ID obtained from above output -

$ nomad status ansible/dispatch-1497641508-d0858f30
ID            = ansible/dispatch-1497641508-d0858f30
Name          = ansible/dispatch-1497641508-d0858f30
Type          = batch
Priority      = 50
Datacenters   = dc1
Status        = dead
Periodic      = false
Parameterized = false

Summary
Task Group  Queued  Starting  Running  Failed  Complete  Lost
cache       0       0         0        0       1         0

Allocations
ID        Eval ID   Node ID   Task Group  Desired  Status    Created At
d46fbe73  f2fea7fe  ec25b9c7  cache       run      complete  06/16/17 19:31:48 UTC

As you can see, this shows you the list of "Allocations" (i.e. the executed container).

Check the status/logs of an allocation execution

You can check the status of a specific allocation ID -

$ nomad alloc-status d46fbe73
ID                  = d46fbe73
Eval ID             = f2fea7fe
Name                = ansible/dispatch-1497641508-d0858f30.cache[0]
Node ID             = ec25b9c7
Job ID              = ansible/dispatch-1497641508-d0858f30
Client Status       = complete
Client Description  = <none>
Desired Status      = run
Desired Description = <none>
Created At          = 06/16/17 19:31:48 UTC

Task "ansible" is "dead"
Task Resources
CPU      Memory   Disk     IOPS  Addresses
500 MHz  256 MiB  300 MiB  0     db: 10.78.1.169:39408

Recent Events:
Time                   Type                   Description
06/16/17 19:31:50 UTC  Terminated             Exit Code: 0
06/16/17 19:31:50 UTC  Started                Task started by client
06/16/17 19:31:48 UTC  Downloading Artifacts  Client is downloading artifacts
06/16/17 19:31:48 UTC  Task Setup             Building Task Directory
06/16/17 19:31:48 UTC  Received               Task received by client

You can also check the logs generated by an allocation ID (for this example, it performs command "ls" on downloaded github repository) -

$ nomad logs d46fbe73
README.md

Full example nomad job file for reference -

job "ansible" {
  datacenters = ["dc1"]

  type = "batch"
  parameterized {
      meta_required = ["REPOSITORY_NAME"]
      meta_optional = ["FIRST_NAME", "LAST_NAME"]
  }

  group "cache" {
    count = 1

    ephemeral_disk {
      size = 300
    }

    task "ansible" {
      driver = "docker"
      config {
        image = "ansible/centos7-ansible:stable"
        command = "ls"
        args = ["local/config/"]
      }
      artifact {
        source = "[email protected]:rohitkothari/${NOMAD_META_REPOSITORY_NAME}"
        destination = "local/config/"
        options {
          sshkey = "_BASE64_SSH_KEY_STRING_"
        }
      }

      resources {
        cpu    = 500 # 500 MHz
        memory = 256 # 256MB
        network {
          mbits = 10
          port "db" {}
        }
      }
    }
  }
}

Thanks for the writeup :)

Who are you and why are you so kind? :)
Thank you !!!!

That helped! :)