Skip to content

Instantly share code, notes, and snippets.

@uditagarwal97
Created May 23, 2017 11:55
Show Gist options
  • Save uditagarwal97/0a55d51d3cb4120312c247d6999bd22b to your computer and use it in GitHub Desktop.
Save uditagarwal97/0a55d51d3cb4120312c247d6999bd22b to your computer and use it in GitHub Desktop.
Error extraction using Travis API
In oreder to access Travis API, we need a client either via http , travis CLI (written in ruby), ruby(for making http headers) or travisPy(Python based travis API client).
I used TravisPy as API client. https://travispy.readthedocs.io/en/stable/
Now, in order to extract errors thrown by travis servers we need to somehow access log files generated after each job.
In order to extract logs, I followed the following route:
User(1) ---> Repository(2) ---> Build(3) ---> Job(4) ---> log(5) ---> log.body()(6)
1) After installation of TravisPy by "pip install travispy", we need to authenticate user.
For authentication, I prefer using github tokens.( view: https://github.com/settings/tokens ) with following permissions
read:org, repo:status, repo_deployment, user:email, write:repo_hook
Then, invoke
>>> from travispy import TravisPy
>>> t = TravisPy.github_auth(<your_github_token>)
>>> user = t.user()
>>> user.login
Now that should display your Github username.
Now, we need to access user repositories
2) Invoke,
>>> repos = t.repos(member=user.login)
>>> len(repos)
>>> repos[0].slug //should display the name of repository
>>> repo=t.repo('madaari/Test-code-base') //replace '*' with the content of repos[0].slug
Done, we now got repository we want.(This repository should be enable in travis.org user control panel + should contain valid .travis.yml file).
3) For Build info, Invoke
>>> build = t.build(repo.last_build_id)
>>> build.config //should display content of .travis.yml file
4) >>> job0=build.jobs[0]
5) >>> log0=job0.log
6) >>> log0.body
------------------------------------------------------------------------------------------------------------------------------
Logs:
>>> build.jobs[0]
<travispy.entities.job.Job object at 0x7f5296faff28>
>>> build.config
{u'dist': u'precise', u'language': u'python', u'script': u'python hellopy.py', u'python': [u'2.6', u'2.7', u'3.2'], u'.result': u'configured', u'group': u'stable'}
>>> job0=build.jobs[0]
>>> job0.config
{u'dist': u'precise', u'language': u'python', u'script': u'python hellopy.py', u'python': u'2.6', u'.result': u'configured', u'group': u'stable', u'os': u'linux'}
>>> job0.log
<travispy.entities.log.Log object at 0x7f52964670b0>
>>> log0=job0.log
>>> log0.body
u'travis_fold:start:worker_info\r\x1b[0K\x1b[33;1mWorker information\x1b[0m\nhostname: i-0720861-precise-production-2-worker-org-docker.travisci.net:f82f95c2-c3bd-48b0-85e0-aaccd80d6a48\nversion: v2.5.0 https://github.com/travis-ci/worker/tree/da3a43228dffc0fcca5a46569ca786b22991979f\ninstance: 3a9675d:travis:python\nstartup: 501.624364ms\ntravis_fold:end:worker_info\r\x1b[0Ktravis_fold:start:system_info\r\x1b[0K\x1b[33;1mBuild system information\x1b[0m\r\nBuild language: python\r\nBuild group: stable\r\nBuild dist: precise\r\nBuild id: 235129153\r\nJob id: 235129154\r\ntravis-build version: 97c7efd0c\r\n\x1b[34m\x1b[1mBuild image provisioning date and time\x1b[0m\r\nThu Feb 5 15:09:33 UTC 2015\r\n\x1b[34m\x1b[1mOperating System Details\x1b[0m\r\nDistributor ID:\tUbuntu\r\nDescription:\tUbuntu 12.04.5 LTS\r\nRelease:\t12.04\r\nCodename:\tprecise\r\n\x1b[34m\x1b[1mLinux Version\x1b[0m\r\n3.13.0-29-generic\r\n\x1b[34m\x1b[1mCookbooks Version\x1b[0m\r\na68419e https://github.com/travis-ci/travis-cookbooks/tree/a68419e\r\n\x1b[34m\x1b[1mGCC version\x1b[0m\r\ngcc (Ubuntu/Linaro 4.6.3-1ubuntu5) 4.6.3\r\nCopyright (C) 2011 Free Software Foundation, Inc.\r\nThis is free software; see the source for copying conditions. There is NO\r\nwarranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.\r\n\r\n\x1b[34m\x1b[1mLLVM version\x1b[0m\r\nclang version 3.4 (tags/RELEASE_34/final)\r\nTarget: x86_64-unknown-linux-gnu\r\nThread model: posix\r\n\x1b[34m\x1b[1mPre-installed Ruby versions\x1b[0m\r\nruby-1.9.3-p551\r\n\x1b[34m\x1b[1mPre-installed Node.js versions\x1b[0m\r\nv0.10.36\r\n\x1b[34m\x1b[1mPre-installed Go versions\x1b[0m\r\n1.4.1\r\n\x1b[34m\x1b[1mRedis version\x1b[0m\r\nredis-server 2.8.19\r\n\x1b[34m\x1b[1mriak version\x1b[0m\r\n2.0.2\r\n\x1b[34m\x1b[1mMongoDB version\x1b[0m\r\nMongoDB 2.4.12\r\n\x1b[34m\x1b[1mCouchDB version\x1b[0m\r\ncouchdb 1.6.1\r\n\x1b[34m\x1b[1mNeo4j version\x1b[0m\r\n1.9.4\r\n\x1b[34m\x1b[1mRabbitMQ Version\x1b[0m\r\n3.4.3\r\n\x1b[34m\x1b[1mElasticSearch version\x1b[0m\r\n1.4.0\r\n\x1b[34m\x1b[1mInstalled Sphinx versions\x1b[0m\r\n2.0.10\r\n2.1.9\r\n2.2.6\r\n\x1b[34m\x1b[1mDefault Sphinx version\x1b[0m\r\n2.2.6\r\n\x1b[34m\x1b[1mInstalled Firefox version\x1b[0m\r\nfirefox 31.0esr\r\n\x1b[34m\x1b[1mPhantomJS version\x1b[0m\r\n1.9.8\r\n\x1b[34m\x1b[1mant -version\x1b[0m\r\nApache Ant(TM) version 1.8.2 compiled on December 3 2011\r\n\x1b[34m\x1b[1mmvn -version\x1b[0m\r\nApache Maven 3.2.5 (12a6b3acb947671f09b81f49094c53f426d8cea1; 2014-12-14T17:29:23+00:00)\r\nMaven home: /usr/local/maven\r\nJava version: 1.7.0_76, vendor: Oracle Corporation\r\nJava home: /usr/lib/jvm/java-7-oracle/jre\r\nDefault locale: en_US, platform encoding: ANSI_X3.4-1968\r\nOS name: "linux", version: "3.13.0-29-generic", arch: "amd64", family: "unix"\r\ntravis_fold:end:system_info\r\x1b[0K\r\ntravis_fold:start:fix.CVE-2015-7547\r\x1b[0K$ export DEBIAN_FRONTEND=noninteractive\r\nReading package lists...\r\nBuilding dependency tree...\r\nReading state information...\r\nThe following extra packages will be installed:\r\n libc-bin libc-dev-bin libc6-dev\r\nSuggested packages:\r\n glibc-doc\r\nThe following packages will be upgraded:\r\n libc-bin libc-dev-bin libc6 libc6-dev\r\n4 upgraded, 0 newly installed, 0 to remove and 264 not upgraded.\r\nNeed to get 8,856 kB of archives.\r\nAfter this operation, 13.3 kB of additional disk space will be used.\r\nGet:1 http://us.archive.ubuntu.com/ubuntu/ precise-updates/main libc6-dev amd64 2.15-0ubuntu10.18 [2,948 kB]\r\nGet:2 http://us.archive.ubuntu.com/ubuntu/ precise-updates/main libc-dev-bin amd64 2.15-0ubuntu10.18 [84.5 kB]\r\nGet:3 http://us.archive.ubuntu.com/ubuntu/ precise-updates/main libc-bin amd64 2.15-0ubuntu10.18 [1,178 kB]\r\nGet:4 http://us.archive.ubuntu.com/ubuntu/ precise-updates/main libc6 amd64 2.15-0ubuntu10.18 [4,646 kB]\r\nFetched 8,856 kB in 0s (33.9 MB/s)\r\nPreconfiguring packages ...\r\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 72431 files and directories currently installed.)\r\nPreparing to replace libc6-dev 2.15-0ubuntu10.10 (using .../libc6-dev_2.15-0ubuntu10.18_amd64.deb) ...\r\nUnpacking replacement libc6-dev ...\r\nPreparing to replace libc-dev-bin 2.15-0ubuntu10.10 (using .../libc-dev-bin_2.15-0ubuntu10.18_amd64.deb) ...\r\nUnpacking replacement libc-dev-bin ...\r\nPreparing to replace libc-bin 2.15-0ubuntu10.10 (using .../libc-bin_2.15-0ubuntu10.18_amd64.deb) ...\r\nUnpacking replacement libc-bin ...\r\nProcessing triggers for man-db ...\r\nSetting up libc-bin (2.15-0ubuntu10.18) ...\r\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 72430 files and directories currently installed.)\r\nPreparing to replace libc6 2.15-0ubuntu10.10 (using .../libc6_2.15-0ubuntu10.18_amd64.deb) ...\r\nUnpacking replacement libc6 ...\r\nSetting up libc6 (2.15-0ubuntu10.18) ...\r\nSetting up libc-dev-bin (2.15-0ubuntu10.18) ...\r\nSetting up libc6-dev (2.15-0ubuntu10.18) ...\r\nProcessing triggers for libc-bin ...\r\nldconfig deferred processing now taking place\r\ntravis_fold:end:fix.CVE-2015-7547\r\x1b[0Ktravis_fold:start:update_libssl1.0.0\r\x1b[0K$ sudo apt-get install libssl1.0.0\r\n\rReading package lists... 0%\r\rReading package lists... 0%\r\rReading package lists... 1%\r\rReading package lists... 10%\r\rReading package lists... 10%\r\rReading package lists... 11%\r\rReading package lists... 11%\r\rReading package lists... 46%\r\rReading package lists... 46%\r\rReading package lists... 47%\r\rReading package lists... 47%\r\rReading package lists... 47%\r\rReading package lists... 58%\r\rReading package lists... 58%\r\rReading package lists... 58%\r\rReading package lists... 58%\r\rReading package lists... 94%\r\rReading package lists... 94%\r\rReading package lists... 95%\r\rReading package lists... 95%\r\rReading package lists... 95%\r\rReading package lists... 95%\r\rReading package lists... 95%\r\rReading package lists... 95%\r\rReading package lists... 95%\r\rReading package lists... 95%\r\rReading package lists... 95%\r\rReading package lists... 95%\r\rReading package lists... 95%\r\rReading package lists... 95%\r\rReading package lists... 95%\r\rReading package lists... 95%\r\rReading package lists... 95%\r\rReading package lists... 95%\r\rReading package lists... 95%\r\rReading package lists... 95%\r\rReading package lists... 95%\r\rReading package lists... 95%\r\rReading package lists... 95%\r\rReading package lists... 95%\r\rReading package lists... 95%\r\rReading package lists... 95%\r\rReading package lists... 95%\r\rReading package lists... 95%\r\rReading package lists... 95%\r\rReading package lists... 95%\r\rReading package lists... 95%\r\rReading package lists... 95%\r\rReading package lists... 95%\r\rReading package lists... 95%\r\rReading package lists... 95%\r\rReading package lists... 95%\r\rReading package lists... 95%\r\rReading package lists... 95%\r\rReading package lists... 95%\r\rReading package lists... 95%\r\rReading package lists... 95%\r\rReading package lists... 95%\r\rReading package lists... 95%\r\rReading package lists... 95%\r\rReading package lists... 95%\r\rReading package lists... 95%\r\rReading package lists... 95%\r\rReading package lists... 95%\r\rReading package lists... 95%\r\rReading package lists... 95%\r\rReading package lists... 95%\r\rReading package lists... 95%\r\rReading package lists... 95%\r\rReading package lists... 95%\r\rReading package lists... 95%\r\rReading package lists... 95%\r\rReading package lists... 95%\r\rReading package lists... 95%\r\rReading package lists... 95%\r\rReading package lists... 95%\r\rReading package lists... 95%\r\rReading package lists... 96%\r\rReading package lists... 96%\r\rReading package lists... 96%\r\rReading package lists... 96%\r\rReading package lists... 97%\r\rReading package lists... 97%\r\rReading package lists... 98%\r\rReading package lists... 98%\r\rReading package lists... 98%\r\rReading package lists... 98%\r\rReading package lists... 98%\r\rReading package lists... 98%\r\rReading package lists... 98%\r\rReading package lists... 98%\r\rReading package lists... 98%\r\rReading package lists... 98%\r\rReading package lists... 98%\r\rReading package lists... 98%\r\rReading package lists... 98%\r\rReading package lists... 98%\r\rReading package lists... 98%\r\rReading package lists... 98%\r\rReading package lists... 98%\r\rReading package lists... 98%\r\rReading package lists... Done\r\r\n\rBuilding dependency tree... 0%\r\rBuilding dependency tree... 0%\r\rBuilding dependency tree... 50%\r\rBuilding dependency tree... 50%\r\rBuilding dependency tree \r\r\n\rReading state information... 0%\r\rReading state information... 0%\r\rReading state information... Done\r\r\nThe following extra packages will be installed:\r\n libssl-dev\r\nThe following packages will be upgraded:\r\n libssl-dev libssl1.0.0\r\n2 upgraded, 0 newly installed, 0 to remove and 262 not upgraded.\r\nNeed to get 2,634 kB of archives.\r\nAfter this operation, 29.7 kB of additional disk space will be used.\r\n\r\r0% [Working]\r \rGet:1 http://us.archive.ubuntu.com/ubuntu/ precise-updates/main libssl-dev amd64 1.0.1-4ubuntu5.39 [1,580 kB]\r\n\r \r1% [1 libssl-dev 14.1 kB/1,580 kB 1%]\r \r60% [Waiting for headers]\r \rGet:2 http://us.archive.ubuntu.com/ubuntu/ precise-updates/main libssl1.0.0 amd64 1.0.1-4ubuntu5.39 [1,054 kB]\r\n\r \r62% [2 libssl1.0.0 57.7 kB/1,054 kB 5%]\r \r100% [Working]\r \rFetched 2,634 kB in 0s (18.5 MB/s)\r\nPreconfiguring packages ...\r\n(Reading database ... \r(Reading database ... 5%\r(Reading database ... 10%\r(Reading database ... 15%\r(Reading database ... 20%\r(Reading database ... 25%\r(Reading database ... 30%\r(Reading database ... 35%\r(Reading database ... 40%\r(Reading database ... 45%\r(Reading database ... 50%\r(Reading database ... 55%\r(Reading database ... 60%\r(Reading database ... 65%\r(Reading database ... 70%\r(Reading database ... 75%\r(Reading database ... 80%\r(Reading database ... 85%\r(Reading database ... 90%\r(Reading database ... 95%\r(Reading database ... 100%\r(Reading database ... 72430 files and directories currently installed.)\r\nPreparing to replace libssl-dev 1.0.1-4ubuntu5.21 (using .../libssl-dev_1.0.1-4ubuntu5.39_amd64.deb) ...\r\nUnpacking replacement libssl-dev ...\r\nPreparing to replace libssl1.0.0 1.0.1-4ubuntu5.21 (using .../libssl1.0.0_1.0.1-4ubuntu5.39_amd64.deb) ...\r\nUnpacking replacement libssl1.0.0 ...\r\nSetting up libssl1.0.0 (1.0.1-4ubuntu5.39) ...\r\nSetting up libssl-dev (1.0.1-4ubuntu5.39) ...\r\nProcessing triggers for libc-bin ...\r\nldconfig deferred processing now taking place\r\ntravis_fold:end:update_libssl1.0.0\r\x1b[0Ktravis_fold:start:git.checkout\r\x1b[0Ktravis_time:start:0f9e965a\r\x1b[0K$ git clone --depth=50 --branch=master https://github.com/madaari/Test-code-base.git madaari/Test-code-base\r\nCloning into \'madaari/Test-code-base\'...\r\nremote: Counting objects: 18, done.\x1b[K\r\nremote: Compressing objects: 7% (1/14) \x1b[K\rremote: Compressing objects: 14% (2/14) \x1b[K\rremote: Compressing objects: 21% (3/14) \x1b[K\rremote: Compressing objects: 28% (4/14) \x1b[K\rremote: Compressing objects: 35% (5/14) \x1b[K\rremote: Compressing objects: 42% (6/14) \x1b[K\rremote: Compressing objects: 50% (7/14) \x1b[K\rremote: Compressing objects: 57% (8/14) \x1b[K\rremote: Compressing objects: 64% (9/14) \x1b[K\rremote: Compressing objects: 71% (10/14) \x1b[K\rremote: Compressing objects: 78% (11/14) \x1b[K\rremote: Compressing objects: 85% (12/14) \x1b[K\rremote: Compressing objects: 92% (13/14) \x1b[K\rremote: Compressing objects: 100% (14/14) \x1b[K\rremote: Compressing objects: 100% (14/14), done.\x1b[K\r\nremote: Total 18 (delta 3), reused 13 (delta 1), pack-reused 0\x1b[K\r\nUnpacking objects: 5% (1/18) \rUnpacking objects: 11% (2/18) \rUnpacking objects: 16% (3/18) \rUnpacking objects: 22% (4/18) \rUnpacking objects: 27% (5/18) \rUnpacking objects: 33% (6/18) \rUnpacking objects: 38% (7/18) \rUnpacking objects: 44% (8/18) \rUnpacking objects: 50% (9/18) \rUnpacking objects: 55% (10/18) \rUnpacking objects: 61% (11/18) \rUnpacking objects: 66% (12/18) \rUnpacking objects: 72% (13/18) \rUnpacking objects: 77% (14/18) \rUnpacking objects: 83% (15/18) \rUnpacking objects: 88% (16/18) \rUnpacking objects: 94% (17/18) \rUnpacking objects: 100% (18/18) \rUnpacking objects: 100% (18/18), done.\r\nChecking connectivity... done.\r\n\r\ntravis_time:end:0f9e965a:start=1495532851000813406,finish=1495532851355901522,duration=355088116\r\x1b[0K$ cd madaari/Test-code-base\r\n$ git checkout -qf 8e821bea30ad7d0e4a549d3feb332876d0e06832\r\ntravis_fold:end:git.checkout\r\x1b[0Ktravis_time:start:0f196aa2\r\x1b[0K$ source ~/virtualenv/python2.6/bin/activate\r\n\r\ntravis_time:end:0f196aa2:start=1495532857014802755,finish=1495532857019803067,duration=5000312\r\x1b[0K$ python --version\r\nPython 2.6.9\r\n$ pip --version\r\npip 6.0.7 from /home/travis/virtualenv/python2.6.9/lib/python2.6/site-packages (python 2.6)\r\nCould not locate requirements.txt. Override the install: key in your .travis.yml to install dependencies.\r\ntravis_time:start:14936a08\r\x1b[0K$ python hellopy.py\r\n File "hellopy.py", line 1\r\n print("Hello travis)\r\n ^\r\nSyntaxError: EOL while scanning string literal\r\n\r\ntravis_time:end:14936a08:start=1495532857461961052,finish=1495532857475011028,duration=13049976\r\x1b[0K\r\n\x1b[31;1mThe command "python hellopy.py" exited with 1.\x1b[0m\r\n\r\nDone. Your build exited with 1.\r\n'
Error can be obtained by manipulating last few lines i.e:
$ python hellopy.py\r\n File "hellopy.py", line 1\r\n print("Hello travis)\r\n ^\r\nSyntaxError: EOL while scanning string literal\r\n\r\ntravis_time:end:14936a08:start=1495532857461961052,finish=1495532857475011028,duration=13049976\r\x1b[0K\r\n\x1b[31;1mThe command "python hellopy.py" exited with 1.\x1b[0m\r\n\r\nDone. Your build exited with 1.\r\n'
Error:
SyntaxError: EOL while scanning string literal\r
Statement which gave error:
print("Hello travis)\r\n
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment