Angel Pizarro delagoya

Map of Task Execution Service (TES) to AWS

This document is an overview of how concepts from TES map to concepts in AWS Batch.

AWS Batch - Basic Concepts

AWS Batch ("Batch") has a few basic concepts that need to be understood before we can make a comparison to concepts in TES. Some relate directly to TES and others do not.

Job

Port 22 is BLOCKED! HALP!

The MCW auditorium wifi blocks the standard SSH network port, port 22. If you are to successfully connect to the EC2 instance from the MCW guest wifi, you will need to change the default port that the SSH service provides for connections. You will also need to allow network traffic to that port on the EC2 instance.

Step 1: the userdata script

The following Bash shell script will be used to add a port to the SSH service.

Implementing a simple AWS Lambda function

In this lab, we will use the chalice Python serverless web application framework for AWS to create a REST API endpoint to reverse complement a DNA string using AWS Lambda and Amazon API Gateway.

# install chalice
$ pip install chalice

	#!/bin/bash

	# Specify the desired volume size in GiB as a command line argument. If not specified, default to 20 GiB.
	SIZE=${1:-20}

	# Get the ID of the environment host Amazon EC2 instance.
	# IMDSv2 enabled
	TOKEN=`curl -X PUT "http://169.254.169.254/latest/api/token" -H "X-aws-ec2-metadata-token-ttl-seconds: 21600"`
	INSTANCEID=$(curl -H "X-aws-ec2-metadata-token: $TOKEN" http://169.254.169.254/latest/meta-data/instance-id)
	REGION=$(curl -H "X-aws-ec2-metadata-token: $TOKEN" -s http://169.254.169.254/latest/meta-data/placement/availability-zone \| sed 's/\(.*\)[a-z]/\1/')

	Description: Create a VS code-server instance with an Amazon CloudFront distribution for development of Nextflow pipelines. Version 4.0.0

	Parameters:
	VSCodeUser:
	Type: String
	Description: UserName for VS code-server
	Default: developer
	GitUserName:
	Type: String
	Description: Global Git user.name for repo commits

	{
	"jobDefinitionName": "dask-mnp-hello-world",
	"type": "multinode",
	"platformCapabilities": [
	"EC2"
	],
	"nodeProperties": {
	"numNodes": 3,
	"mainNode": 0,
	"nodeRangeProperties": [

	#!/bin/sh
	cd /tmp
	git clone https://github.com/mpitutorial/mpitutorial
	cd mpitutorial/tutorials/mpi-hello-world/code
	# grab the mpicc PATH locations
	source /etc/profile.d/efa.sh
	make all
	cp mpi_hello_world /shared/mpi_hello_world

	FROM rocker/r-base

	RUN apt-get update
	# install python tools
	RUN apt-get install -y python-dev python-pip
	# install AWS CLI
	RUN pip install -U awscli

	@namespace("com.example")
	/**
	This is a comment for the whole protocol
	*/

	protocol Example {

	/**

	The comment applies to the `NoSpaces` record, but is not indented to the