Skip to content

Instantly share code, notes, and snippets.

@evmasuta
evmasuta / Hsiao_Lab COMET
Last active June 3, 2018 17:18 — forked from KBlansit/SDSC Singularity Build
Hsiao_Lab COMET onboarding
By: Evan Masutani
As of 11 May 2018, the Hsiao lab has been granted computational time on XSEDE/SDSC (San Diego Supercomputer Cluster), specifically
on the COMET shared-gpu resource, which currently boasts a number of K80s and P100s (see https://portal.xsede.org/sdsc-comet).
The purpose of many projects in the Hsiao lab is to apply machine learning principles to solve medical problems, specifically in
the realm of imaging. At the time of writing, the preferred framework developed in-house uses keras with a tensorflow-gpu backend.
That many dependencies are required is an understatement. Furthermore, for reproducibility's sake and the ability to run lab
software across multiple platforms (notably transferring load from lab workstations to the supercomputer cluster), it became
necessary to seamlessly and intuitively port both custom software and associated dependencies between servers.
For experienced users, skip the following section (BACKGROUND) and proceed to TECHNICAL DETAILS
@evmasuta
evmasuta / Hsiao_Lab COMET
Last active August 1, 2020 00:43
Hsiao_Lab COMET onboarding
By: Evan Masutani
As of 11 May 2018, the Hsiao lab has been granted computational time on XSEDE/SDSC (San Diego Supercomputer Cluster), specifically
on the COMET shared-gpu resource, which currently boasts a number of K80s and P100s (see https://portal.xsede.org/sdsc-comet).
The purpose of many projects in the Hsiao lab is to apply machine learning principles to solve medical problems, specifically in
the realm of imaging. At the time of writing, the preferred framework developed in-house uses keras with a tensorflow-gpu backend.
That many dependencies are required is an understatement. Furthermore, for reproducibility's sake and the ability to run lab
software across multiple platforms (notably transferring load from lab workstations to the supercomputer cluster), it became
necessary to seamlessly and intuitively port both custom software and associated dependencies between servers.
For experienced users, skip the following section (BACKGROUND) and proceed to TECHNICAL DETAILS