Skip to content

Instantly share code, notes, and snippets.

@mqzhang
Forked from laixintao/decent_request.py
Created December 21, 2023 12:22
Show Gist options
  • Save mqzhang/2d5814965db88ae5e3d16d71f509146f to your computer and use it in GitHub Desktop.
Save mqzhang/2d5814965db88ae5e3d16d71f509146f to your computer and use it in GitHub Desktop.
Send HTTP requests using python-requests with timeout, tcp reuse(session) and retry.
from requests.adapters import HTTPAdapter, Retry
from requests import Session
retries = Retry(
total=5, backoff_factor=1, status_forcelist=[502, 503, 504]
)
session = Session() # reuse tcp connection
session.mount("http://", HTTPAdapter(max_retries=retries))
session.mount("https://", HTTPAdapter(max_retries=retries))
session.get("https://example.com", timeout=5) # seconds
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment