Skip to content

Instantly share code, notes, and snippets.

@craibuc
Created January 25, 2022 19:57
Show Gist options
  • Save craibuc/ba46f295fc06a8df3812be3b865306b6 to your computer and use it in GitHub Desktop.
Save craibuc/ba46f295fc06a8df3812be3b865306b6 to your computer and use it in GitHub Desktop.
Recursively retrieve the files and folders from a remote resource.
<#
.SYNOPSIS
Recursively retrieve the files and folders from a remote resource.
.PARAMETER Uri
The URI of the remote item.
.PARAMETER Destination
The path to folder that will contain the files and folders that are fetched from the Uri. Default: ~/Downloads
.EXAMPLE
Get-RemoteItem -Uri 'https://physionet.org/files/deid/1.1/'
Gets the files and folders from resource and saves them to ~/Downloads
#>
function Get-RemoteItem
{
[CmdletBinding()]
param (
[Parameter(Position=0)]
[uri]$Uri,
[Parameter(Position=1)]
[string]$Destination = "$env:USERPROFILE\Downloads"
)
Write-Verbose "Requesting $Uri..."
$Response = Invoke-WebRequest -Uri $Uri.AbsoluteUri
# it's a file
if ($Response.Links.Count -eq 0)
{
$FilePath = Join-Path -Path $Destination -ChildPath ("{0}{1}" -f $Uri.Host, $Uri.LocalPath)
Write-Debug "FilePath: $FilePath"
$Parent = Split-Path $FilePath -Parent
if ( $False -eq (Test-Path $Parent) )
{
Write-Verbose "Creating directory: $Parent..."
New-Item -ItemType Directory -Path $Parent -Force
}
Write-Verbose "Saving file $FilePath..."
$Response.Content | Out-File -FilePath $FilePath -Force
}
# it's a directory
else
{
# skip parent directory (../)
$Response.Links | Where-Object { $_.href -ne '../' } | ForEach-Object {
Write-Debug "href: $( $_.href )"
Get-RemoteItem "$Uri$($_.href)" $Destination
}
}
}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment