Last active
December 26, 2015 00:39
-
-
Save benzenwen/7066179 to your computer and use it in GitHub Desktop.
Windows powershell script for uploading a directory to Joyent Manta
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Param ( | |
[alias("nl")] | |
[switch] | |
$NoLinks, | |
[alias("v")] | |
[switch] | |
$Verbose, | |
[alias("q")] | |
[switch] | |
$Quiet, | |
[alias("f")] | |
[switch] | |
$Force, | |
[alias("d")] | |
[switch] | |
$DryRun | |
) | |
<# | |
.SYNOPSIS | |
gci -recurse | mupload [-NoLinks|nl] [-Verbose|v] [-Quiet|q] | |
[-DryRun|d] <basedirectory> | |
.DESCRIPTION | |
Upload a Windows directory hierarchy to Joyent Manta | |
Joyent Manta is a cloud-based object storage system, with an | |
integrated compute cluster. mupload uploads a directory | |
hierarchy to a particular manta account. | |
There is one required parameter, which is the base directory | |
in the remote Manta system. Those directories are of the form | |
"/<accountname>/[stor|public]/<user-selectable-names>". NOTE | |
that these are forward slashes. | |
It takes a piped set of items, like from gci -recurse, and | |
tries to determine each type. It uses the current working | |
Windows directory as the relative root to append to the Manta | |
base directory mentioned above. That is if you are in | |
c:\Users\Administrator, and you provide | |
/benwen20131018/stor/windowsbak001 as the base path, and you | |
do a -recurse to descend your local Windows filesystem, then | |
all filepaths relative to c:\Users\Administrator will | |
appear at /benwen20131018/stor/windowsbak001 as well. | |
If the item is a directory, it creates it. | |
If the item is a file, upload it to Manta in the relative | |
directory | |
Using -Force does NOT delete existing Manta objects if there is | |
a local file to overwrite it. | |
.PARAMETER | |
-NoLinks | n | |
Do not upload files that end in '.lnk', which is how soft | |
links. | |
-Verbose | v | |
Provide more output. | |
-Quiet | q | |
Suppress most output. | |
-Force | q | |
Ignore existing Manta base directory checks. Note that | |
existing objects will be overwritten, but non-overlapping | |
objects will still remain. | |
-DryRun | d | |
Show what commands would be run, but don't actually execute them. | |
.EXAMPLE | |
cgi -recurse | mpuload /benwen/stor/windowsbackup001/ | |
.NOTES | |
Getting started with Joyent Manta: | |
1) Download and Install Node.js as all of the client tools are | |
built on Node.js. http://nodejs.org/download/ | |
2) Run this in a Powershell propmt. It installes the Node | |
Package called "manta": npm install -g manta | |
3) Create a Joyent Cloud account at http://joyent.com | |
4) Create a private / public ssh key pair using the Joyent web | |
console (you should be prompted as you sign up to create and | |
download a key. Note that some versions of IE may not work | |
properly here). | |
5) Put the private key in your Windows home directory in a | |
folder called ".ssh", for example "/User/benwen/.ssh/ (the key | |
file should be called simply "id_rsa" in that directory. | |
6) Do the same with the public key, except it's named | |
"id_rsa.pub". Note that DSA keys don't work yet with Manta | |
(as of 2013-Oct). | |
7) Create these three environment variables: | |
$env:MANTA_USER = "benwen20131018" | |
$env:MANTA_URL = "https://us-east.manta.joyent.com" | |
$env:MANTA_KEY_ID = "11:de:e4:68:f1:b9:3c:1b:9a:9d:01:f5:8c:eb:b7:7c" | |
(substituting your Joyent username for benwen20131018, and | |
your key fingerprint (the 11:de:e4...) which can be found in | |
the Joyent web console in the same place. Click on the | |
generated key to expand the field to show both the fingerprint | |
and public key) | |
8) Run this command to test that Manta is working: "mls". | |
If you don't get an error message you're good to go. Visit: | |
http://apidocs.joyent.com/manta for more information. | |
TODO: figure out how to display the help text in the right | |
place. | |
TODO: what is this thing going to do with special characters | |
like quotes, etc? | |
.LINK | |
http://apidocs.joyent.com/manta | |
#> | |
function Manta-Exists ($mydir) { | |
$mycmd = 'mls' | |
$myexist = & $mycmd $mydir | |
## This is fragile | |
if ($myexist -match '.*ResourceNotFoundError.*') { | |
$False | |
} else { | |
$True | |
} | |
} | |
if (!( $args[0] -match '.*/$')) { | |
$base = $args[0] += "/" | |
} else { | |
$base = $args[0] | |
} | |
if ($Verbose) { "Basedir: " + $base | echo } | |
if (! $Force) { | |
if (Manta-Exists($base)) { | |
"Manta directory " + $base + " already exists. Aborting. Use -Force to override." | echo | |
exit 1 | |
} | |
} | |
foreach ($i in $input) { | |
if ($Verbose) {"Processing: " + $i.name + " is a container: " + $i.PSIsContainer | echo} | |
$tmp = Resolve-Path -relative $i.fullname | |
$tmp = $tmp -replace '\\', '/' | |
$tmp = $tmp -replace '\./', '' | |
$mdir = $base + $tmp | |
if ($Verbose) {"manta path: " + $mdir | echo} | |
if ($i.PSIsContainer) { | |
## mmkdir -p makes parent directories if needed | |
$mcmd = "mmkdir" | |
$mopts = "-p", $mdir | |
} else { | |
## Is this a soft link? | |
if (($i.name -match '.*.lnk$') -and ($NoLinks)) { | |
## do nothing | |
$mcmd = "" | |
$mopts = "" | |
} else { | |
## A regular file | |
$mcmd = "mput" | |
$mopts = "-f", $i.fullname, $mdir | |
} | |
} | |
if ($DryRun) { | |
"Dry Run: " + $mcmd + $mopts | echo | |
} else { | |
if (! $Quiet) { $mcmd + " " + $mopts | echo } | |
& $mcmd $mopts | |
} | |
} | |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment