Skip to content

Instantly share code, notes, and snippets.

@STRd6

STRd6/recursive-selection.md

Last active October 18, 2019 19:47
Show Gist options
  • Save STRd6/1dea28b9fda71276ca9e to your computer and use it in GitHub Desktop.
Save STRd6/1dea28b9fda71276ca9e to your computer and use it in GitHub Desktop.
Download ZIP
Weird Ideas
Raw
s3-cas-ecosystem.md

S3 CAS Ecosystem

A mircoservice would validate pings notifying it of data to ingest into the central CAS. Users get permission to upload to a namespace then ping the workers to validate and move the file over.

Append only so no difference between dev and production.

Structure

bucket
+ root
| | SHAs...
+ users
\ + some_duder
  \ | SHAs...

Worker

  • Needs credentials to bucket root
  • receives url to user submitted resource + server metadata payload
  • notifies server with metadata payload and exits if sha exists in root
  • validates SHA
  • copies to root
  • optionally deletes from users/username/sha
  • notifies server with metadata payload

Server

  • Receives metadata payload + SHA from worker
  • Creates DB entry referencing SHA and any metadata (creator, etc...)

Users

  • Get namespaced credentials from server so they can upload directly to S3
  • Compute SHA locally and upload to S3
  • Ping worker queue when an upload is complete
Raw
s3-repliwiki.md

Self-replicating S3 Hosted Wiki

  • Credentials stored only client localStorage
  • Contains tools to build and post files to S3, edits itself.
  • Post everything to a new bucket to replicate.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment