Skip to content

Instantly share code, notes, and snippets.

@milanboers

milanboers/read_s3_python.py

Last active October 13, 2020 08:11
Show Gist options
  • Save milanboers/83eba1baaf1570517328cd2520c9bccb to your computer and use it in GitHub Desktop.
Save milanboers/83eba1baaf1570517328cd2520c9bccb to your computer and use it in GitHub Desktop.
Download ZIP
Streaming read lines from S3 in Python using boto
Raw
read_s3_python.py
s3 = boto3.client('s3')
obj = s3.get_object(Bucket=bucket, Key=key)
lines = map(lambda x: x.decode('utf-8'), obj['Body'].iter_lines()) # Iterator which yields lines in file
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment