Skip to content

Instantly share code, notes, and snippets.

@tehranian

tehranian/gist:7123269

Created October 23, 2013 17:43
Download ZIP

Revisions

  1. tehranian revised this gist Oct 23, 2013. No changes.
  2. tehranian revised this gist Oct 23, 2013. No changes.
  3. tehranian created this gist Oct 23, 2013.
    67 changes: 67 additions & 0 deletions gistfile1.sh
    View file
    Original file line number Diff line number Diff line change
    @@ -0,0 +1,67 @@
    #!/bin/bash

    # High-performance, native SSH implementation for Jenkins SSH slaves. Jenkins'
    # standard turn-key SSH slave implementation uses an embedded, "pure Java"
    # implementation of SSH (https://github.com/jenkinsci/trilead-ssh2).
    # That standard implementation combines all of the convenience of a pure Java
    # implementation with all of the "performance" of a pure Java implementation :P

    # If your distributed build process generates large build artifacts like ISOs,
    # VM images, Vagrant boxes, etc, then you will see a substantial benefit from
    # switching to this custom slave launcher method. Ex: 5 MB/sec -> 40 MB/sec

    # Given a hostname, we copy over Jenkins's "slave.jar" to that host and launch
    # it with "java -jar ...". This script should be run from the Jenkins master
    # via a custom "Launch method" on the slave node's configuration page.

    # Background on distributed builds w/Jenkins:
    # @see: https://wiki.jenkins-ci.org/display/JENKINS/Distributed+builds

    set -ex

    JENKINS_SLAVE_HOSTNAME=

    JENKINS_DIR=/jenkins
    JENKINS_URL=http://`hostname -f`
    JENKINS_USER=jenkins

    while getopts "d:h:u:" opt; do
    case $opt in
    d) JENKINS_DIR="$OPTARG"
    ;;
    h) JENKINS_SLAVE_HOSTNAME="$OPTARG"
    ;;
    u) JENKINS_USER="$OPTARG"
    ;;
    *) echo "invalid option: $1" 1>&2;
    ;;
    esac
    done

    if [ -z $JENKINS_SLAVE_HOSTNAME ]; then
    echo "Error: Host name of slave must be specified with '-h <slave hostname>'"
    exit 100
    fi


    # Tune SSH for performance.
    # @see: http://www.damtp.cam.ac.uk/user/ejb48/sshspeedtests.html
    SSH_CIPHERS="arcfour256,arcfour128,arcfour,blowfish-cbc"
    SSH_MACS="[email protected],hmac-md5"
    SSH_SECURITY_OPTS="-o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no"
    SSH_COMMAND="ssh -c ${SSH_CIPHERS} -o MACs=${SSH_MACS} ${SSH_SECURITY_OPTS}"


    JENKINS_TMPDIR="${JENKINS_DIR}/tmp"
    DOWNLOAD_COMMAND="curl -o ${JENKINS_DIR}/slave.jar ${JENKINS_URL}/jnlpJars/slave.jar"
    LAUNCH_SLAVE_COMMAND="java -Djava.io.tmpdir=${JENKINS_TMPDIR} -jar ${JENKINS_DIR}/slave.jar"


    # download slave.jar & run it.
    #
    # NOTE: we need to do this all in one command otherwise Jenkins will fail the
    # launching of the slave with a communication exception. I think this is
    # because STDIN/STDOUT become polluted and Jenkins master is using those
    # streams for communication with the remote Java process?
    ${SSH_COMMAND} ${JENKINS_USER}@${JENKINS_SLAVE_HOSTNAME} \
    "cd ${JENKINS_DIR} && ${DOWNLOAD_COMMAND} && TMPDIR=${JENKINS_TMPDIR} ${LAUNCH_SLAVE_COMMAND}"