Skip to content

Instantly share code, notes, and snippets.

@cmdcolin

cmdcolin/static_jbrowse.md

Last active February 4, 2025 14:47
Show Gist options
  • Save cmdcolin/bac94993c89ee8a22cc43318ff31e077 to your computer and use it in GitHub Desktop.
Save cmdcolin/bac94993c89ee8a22cc43318ff31e077 to your computer and use it in GitHub Desktop.
Download ZIP
Using JBrowse on a static file host
Raw
static_jbrowse.md

One thing that is important to note about JBrowse is that it is a "static site compatible application"

What is a "static site"?

A static site is just a set of plain files like HTML, JS, and CSS

Static sites don't use dynamic "server side" code like PHP, Node.js, Java Servlets, or things like that.

Brief history of static sites

The idea of plain HTML, JS, and CSS for websites goes back to the early days of the web

However, the term "static site" (as far as I know) became popular around the time "GitHub pages" was first created in ~2008 (https://en.wikipedia.org/wiki/Timeline_of_GitHub)

Github pages allowed users to host their own webpages on github, and a program called Jekyll (https://jekyllrb.com/), was released around the same time to help users to create blogs in a static site compatible way

The simple idea is that Jekyll would generate plain HTML, JS, and CSS files for your blog, and Github Pages would host them for free

Why static sites

Static sites are compelling because they don't have nearly as many potential security vulnerabilities because there is little to no server side code to exploit

What does it mean that "JBrowse is static site compatible"?

JBrowse (both 1 and 2) are released as a zip file of HTML, JS, and CSS

There is no server side for the program

Sometimes people are misled by the fact that jbrowse includes CLI perl programs (for JBrowse 1) and node.js programs (for JBrowse 2) which are used for preparing the config files. However, these are run-once tools, and do not continuously run on the server

How does JBrowse create it's visualizations without a server side

JBrowse does all the visualization using client side javascript, and can even do advanced things like parse BAM, CRAM, BigWig and other binary formats in client side javascript. It uses HTTP Range queries to access only small parts of the super large e.g. BAM files to only get the reads for a particular genomic region of interest

JBrowse's client side javascript visualization strategy has a lot of strengths (no need to babysit a server's CPU load or monitor crashes) but some weaknesses (it may increase the CPU and bandwidth usage on users computers)

Conceptually, if a user really wanted to make server side generated visualizations, they could make a JBrowse plugin to do so

Setting up JBrowse on nginx or Apache2

In many of our tutorials, we instruct users to downloaded JBrowse to a linux server

Then you download the JBrowse CLI tools (either the perl tools like bin/flatfile-to-json.pl for jbrowse 1, or the node.js based "jbrowse add-track" for jbrowse 2)

These CLI tools prepare the config file, and put the hosted files in e.g. the /var/www/html/ for serving by Apache2 or nginx

Users commonly do this on university computing resources, but Amazon EC2 is also common. EC2 is a virtual machine in the cloud, which you pay monthly for. But...JBrowse doesn't even need a EC2 virtual machine!

No need for EC2 - Setting up JBrowse as a "serverless" static site using Amazon S3

Amazon S3 is a file hosting service that does not have any actual server computing resources or virtual machine attached to it.

Instead of putting the config files in the /var/www/html/ folder, you can upload those same files to Amazon S3

jbrowse create myjbrowsefolder
samtools faidx myfile.fa
jbrowse add-assembly myfile.fa --out myjbrowsefolder --load copy
jbrowse sort-gff myfile.gff > myfile.sorted.gff
bgzip myfile.sorted.gff
tabix myfile.sorted.gff.gz
jbrowse add-track myfile.sorted.gff.gz --out myjbrowsefolder --load copy
aws s3 sync myjbrowsefolder s3://mybucket/myjbrowsefolder

In the above "myjbrowsefolder" is a working JBrowse 2 "instance" that contains a indexed fasta, a tabix indexed gff, and the jbrowse 2 index.html and javascript/css files

We upload it to amazon s3 with the "aws s3 sync" command

Those would then be accessible through a URL like s3.amazonaws.com/mybucket/myjbrowsefolder/index.html

Note that, with just the raw S3 bucket URLs like this, adding the index.html to th URL is required. It actually works without index.html the second time your visit the link, but the first time, you need it. This is annoying, so I suggest following the next couple steps

Enhancing the Amazon S3 static site with CloudFront or extra config

As mentioned above, the raw S3 links are a bit tricky, and need things like manually adding index.html

You can work around this though in a couple of ways

These both make the S3 bucket links feel more like normal websites

Here is a tutorial from Amazon about hosting a static site on Amazon S3

https://docs.aws.amazon.com/AmazonS3/latest/userguide/WebsiteHosting.html

Note that there is also an Option C that Amazon recommends to use called AWS Amplify

Some JBrowse users have also successfully used with JBrowse with AWS Amplify (@scottcain, WormBase/AGR)

Note that jbrowse.org uses Amazon S3 + CloudFront (option A)

What other static site hosting options are there

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment