Skip to content

Instantly share code, notes, and snippets.

@iht

iht/relay_options.py

Created January 25, 2021 17:57
Show Gist options
  • Save iht/dda4edef69678a882c4c4b0779afe674 to your computer and use it in GitHub Desktop.
Save iht/dda4edef69678a882c4c4b0779afe674 to your computer and use it in GitHub Desktop.
Download ZIP
Relay your custom runtime options to Dataflow, assuming you use Flex Templates
Raw
relay_options.py
def run_pipeline(argv):
opts: PipelineOptions = PipelineOptions(argv)
gcloud_opts: GoogleCloudOptions = opts.view_as(GoogleCloudOptions)
if opts.i_want_streaming_engine:
gcloud_opts.enable_streaming_engine = True
else:
gcloud_opts.enable_streaming_engine = False
...
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment