Skip to content

Instantly share code, notes, and snippets.

@Nani-o

Nani-o/report.md

Last active February 4, 2019 15:41
Show Gist options

  • Select an option

    Learn more about clone URLs
  • Save Nani-o/ad39aabacb9e4b9bdc731c16339608ea to your computer and use it in GitHub Desktop.

Select an option

Learn more about clone URLs
Save Nani-o/ad39aabacb9e4b9bdc731c16339608ea to your computer and use it in GitHub Desktop.
Download ZIP
User agent not setting up correctly in some ansible modules when using proxy
Raw
report.md

Using curl with a proxy

command

curl -A "ansible-httpget" -L https://github.com/wkhtmltopdf/wkhtmltopdf/releases/download/0.12.4/wkhtmltox-0.12.4_linux-generic-amd64.tar.xz

tcpdump captured headers
CONNECT github.com:443 HTTP/1.1
Host: github.com:443
User-Agent: ansible-httpget
Proxy-Connection: Keep-Alive

CONNECT github-production-release-asset-2e65be.s3.amazonaws.com:443 HTTP/1.1
Host: github-production-release-asset-2e65be.s3.amazonaws.com:443
User-Agent: ansible-httpget
Proxy-Connection: Keep-Alive

Using ansible with a proxy

Ansible task
    - name: "WKHTMLTOPDF | Download and unarchiving"
      get_url:
        url: "https://github.com/wkhtmltopdf/wkhtmltopdf/releases/download/0.12.4/wkhtmltox-0.12.4_linux-generic-amd64.tar.xz"
        dest: /tmp/
Ansible result with validate_certs: True
The full traceback is:
WARNING: The below traceback may *not* be related to the actual failure.
  File "/tmp/ansible_get_url_payload_bgZvFJ/ansible_get_url_payload.zip/ansible/module_utils/urls.py", line 1259, in fetch_url
    client_key=client_key, cookies=cookies)
  File "/tmp/ansible_get_url_payload_bgZvFJ/ansible_get_url_payload.zip/ansible/module_utils/urls.py", line 1162, in open_url
    client_cert=client_cert, client_key=client_key, cookies=cookies)
  File "/tmp/ansible_get_url_payload_bgZvFJ/ansible_get_url_payload.zip/ansible/module_utils/urls.py", line 1070, in open
    r = urllib_request.urlopen(*urlopen_args)
  File "/usr/lib/python2.7/urllib2.py", line 154, in urlopen
    return opener.open(url, data, timeout)
  File "/usr/lib/python2.7/urllib2.py", line 435, in open
    response = meth(req, response)
  File "/usr/lib/python2.7/urllib2.py", line 548, in http_response
    'http', request, response, code, msg, hdrs)
  File "/usr/lib/python2.7/urllib2.py", line 467, in error
    result = self._call_chain(*args)
  File "/usr/lib/python2.7/urllib2.py", line 407, in _call_chain
    result = func(*args)
  File "/usr/lib/python2.7/urllib2.py", line 654, in http_error_302
    return self.parent.open(new, timeout=req.timeout)
  File "/usr/lib/python2.7/urllib2.py", line 427, in open
    req = meth(req)
  File "/tmp/ansible_get_url_payload_bgZvFJ/ansible_get_url_payload.zip/ansible/module_utils/urls.py", line 779, in http_request
    build_ssl_validation_error(self.hostname, self.port, paths_checked, e)
  File "/tmp/ansible_get_url_payload_bgZvFJ/ansible_get_url_payload.zip/ansible/module_utils/urls.py", line 580, in build_ssl_validation_error
    raise SSLValidationError(' '.join(msg) % (hostname, port, ", ".join(paths)))

fatal: [pmp-mpe1-prep]: FAILED! => {
    "changed": false,
    "invocation": {
        "module_args": {
            "attributes": null,
            "backup": null,
            "checksum": "",
            "client_cert": null,
            "client_key": null,
            "content": null,
            "delimiter": null,
            "dest": "/tmp/",
            "directory_mode": null,
            "follow": false,
            "force": false,
            "force_basic_auth": false,
            "group": null,
            "headers": null,
            "http_agent": "ansible-httpget",
            "mode": null,
            "owner": null,
            "regexp": null,
            "remote_src": null,
            "selevel": null,
            "serole": null,
            "setype": null,
            "seuser": null,
            "sha256sum": "",
            "src": null,
            "timeout": 10,
            "tmp_dest": null,
            "unsafe_writes": null,
            "url": "https://github.com/wkhtmltopdf/wkhtmltopdf/releases/download/0.12.3/wkhtmltox-0.12.3_linux-generic-amd64.tar.xz",
            "url_password": null,
            "url_username": null,
            "use_proxy": true,
            "validate_certs": true
        }
    },
    "msg": "Failed to validate the SSL certificate for github-production-release-asset-2e65be.s3.amazonaws.com:443. Make sure your managed systems have a valid CA certificate installed. You can use validate_certs=False if you do not need to confirm the servers identity but this is unsafe and not recommended. Paths checked for this platform: /etc/ssl/certs, /etc/pki/ca-trust/extracted/pem, /etc/pki/tls/certs, /usr/share/ca-certificates/cacert.org, /etc/ansible. The exception msg was: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:590)."
}
Ansible result with validate_certs: False
fatal: [pmp-mpe1-prep]: FAILED! => {
    "changed": false,
    "dest": "/tmp/",
    "gid": 0,
    "group": "root",
    "invocation": {
        "module_args": {
            "attributes": null,
            "backup": null,
            "checksum": "",
            "client_cert": null,
            "client_key": null,
            "content": null,
            "delimiter": null,
            "dest": "/tmp/",
            "directory_mode": null,
            "follow": false,
            "force": false,
            "force_basic_auth": false,
            "group": null,
            "headers": null,
            "http_agent": "ansible-httpget",
            "mode": null,
            "owner": null,
            "regexp": null,
            "remote_src": null,
            "selevel": null,
            "serole": null,
            "setype": null,
            "seuser": null,
            "sha256sum": "",
            "src": null,
            "timeout": 10,
            "tmp_dest": null,
            "unsafe_writes": null,
            "url": "https://github.com/wkhtmltopdf/wkhtmltopdf/releases/download/0.12.3/wkhtmltox-0.12.3_linux-generic-amd64.tar.xz",
            "url_password": null,
            "url_username": null,
            "use_proxy": true,
            "validate_certs": false
        }
    },
    "mode": "01777",
    "msg": "Request failed",
    "owner": "root",
    "response": "HTTP Error 403: Forbidden",
    "size": 4096,
    "state": "directory",
    "status_code": 403,
    "uid": 0,
    "url": "https://github.com/wkhtmltopdf/wkhtmltopdf/releases/download/0.12.3/wkhtmltox-0.12.3_linux-generic-amd64.tar.xz"
}
Captured headers
CONNECT github.com:443 HTTP/1.0
Connection: close

CONNECT github-production-release-asset-2e65be.s3.amazonaws.com:443 HTTP/1.0
Connection: close

Ansible without proxy (I used http instead of https to capture the headers using tcpdump)

Ansible task
    - name: "WKHTMLTOPDF | Download and unarchiving"
      get_url:
        url: "http://github.com/wkhtmltopdf/wkhtmltopdf/releases/download/0.12.4/wkhtmltox-0.12.4_linux-generic-amd64.tar.xz"
        dest: /tmp/
Ansible result (since the host has no access to internet, I have redirected github.com locally on the host to capture headers)
The full traceback is:
WARNING: The below traceback may *not* be related to the actual failure.
  File "/tmp/ansible_get_url_payload_Cj_yDx/ansible_get_url_payload.zip/ansible/module_utils/urls.py", line 1259, in fetch_url
    client_key=client_key, cookies=cookies)
  File "/tmp/ansible_get_url_payload_Cj_yDx/ansible_get_url_payload.zip/ansible/module_utils/urls.py", line 1162, in open_url
    client_cert=client_cert, client_key=client_key, cookies=cookies)
  File "/tmp/ansible_get_url_payload_Cj_yDx/ansible_get_url_payload.zip/ansible/module_utils/urls.py", line 1070, in open
    r = urllib_request.urlopen(*urlopen_args)
  File "/usr/lib/python2.7/urllib2.py", line 154, in urlopen
    return opener.open(url, data, timeout)
  File "/usr/lib/python2.7/urllib2.py", line 435, in open
    response = meth(req, response)
  File "/usr/lib/python2.7/urllib2.py", line 548, in http_response
    'http', request, response, code, msg, hdrs)
  File "/usr/lib/python2.7/urllib2.py", line 467, in error
    result = self._call_chain(*args)
  File "/usr/lib/python2.7/urllib2.py", line 407, in _call_chain
    result = func(*args)
  File "/usr/lib/python2.7/urllib2.py", line 654, in http_error_302
    return self.parent.open(new, timeout=req.timeout)
  File "/usr/lib/python2.7/urllib2.py", line 427, in open
    req = meth(req)
  File "/tmp/ansible_get_url_payload_Cj_yDx/ansible_get_url_payload.zip/ansible/module_utils/urls.py", line 781, in http_request
    raise ConnectionError('Failed to connect to %s at port %s: %s' % (self.hostname, self.port, to_native(e)))

fatal: [pmp-mpe1-prep]: FAILED! => {
    "changed": false,
    "invocation": {
        "module_args": {
            "attributes": null,
            "backup": null,
            "checksum": "",
            "client_cert": null,
            "client_key": null,
            "content": null,
            "delimiter": null,
            "dest": "/tmp/",
            "directory_mode": null,
            "follow": false,
            "force": false,
            "force_basic_auth": false,
            "group": null,
            "headers": null,
            "http_agent": "ansible-httpget",
            "mode": null,
            "owner": null,
            "regexp": null,
            "remote_src": null,
            "selevel": null,
            "serole": null,
            "setype": null,
            "seuser": null,
            "sha256sum": "",
            "src": null,
            "timeout": 10,
            "tmp_dest": null,
            "unsafe_writes": null,
            "url": "http://github.com/wkhtmltopdf/wkhtmltopdf/releases/download/0.12.3/wkhtmltox-0.12.3_linux-generic-amd64.tar.xz",
            "url_password": null,
            "url_username": null,
            "use_proxy": true,
            "validate_certs": true
        }
    },
    "msg": "Failed to connect to pmp-mpe1-prepwkhtmltopdf at port 443: [Errno -2] Name or service not known"
}
Captured headers
GET /wkhtmltopdf/wkhtmltopdf/releases/download/0.12.3/wkhtmltox-0.12.3_linux-generic-amd64.tar.xz HTTP/1.1
Accept-Encoding: identity
Host: github.com
Connection: close
User-Agent: ansible-httpget
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment