Download all sentry events for a project. Useful for data processing

download_sentry_data.py

"""Download sentry data.
usage:
python download_sentry_data.py <org>/<project> <api_key>
"""
import requests
import csv
import sys


if __name__ == '__main__':
    with open('data.csv', 'w', encoding='utf-8') as csvfile:
        fieldnames = ['culprit', 'count', 'lastSeen', 'logger']
        writer = csv.DictWriter(csvfile, fieldnames=fieldnames, extrasaction='ignore')
        writer.writeheader()
        url = 'https://app.getsentry.com/api/0/projects/{0}/issues/'.format(sys.argv[1])
        while True:
            response = requests.get(
                url,
                headers={'Authorization': 'Bearer {TOKEN}'.format(TOKEN=sys.argv[2])}
                )
            data = response.json()
            for event in data:
                # tags = {item['key']: item['value'] for item in event['tags']}
                writer.writerow(dict(event))
            link = response.headers.get('Link')
            print("Last event date: {0}".format(data[-1]['lastSeen']))
            if link and '"next"' in link:
                print("Getting next page...")
                url = link.split()[4][1:-2]
            else:
                break

Hi! Thank you so much. Though I'm not a developer, your code could be very helpful for my purposes. Trying to use it. And the process of getting data from Sentry seems to be going very well in the beginning, but in the end I get something like this:

Getting next page...
Last event date: 2019-08-20T14:23:16.284Z
Getting next page...
Traceback (most recent call last):
  File "C:\Personal\download_sentry_data.py", line 26, in <module>
    print("Last event date: {0}".format(data[-1]['lastSeen']))
IndexError: list index out of range

Could you please help me to figure out what the problem is? To my shame, I'm not skilled enough for reading and understanding the Python syntax. :(

Hi @viacheslav-gurin How are you calling the script? What do you see in your data.csv file? It's empty?

@bubenkoff, @vladox, hi!
I'm calling the script via Anaconda prompt:

python C:\Personal\download_sentry_data.py spectrum-data/react [our API key]

where

• spectrum-data is the actual organization name in Sentry
• react is the actual project name in Sentry
• [our API key] is the key I've created in Sentry (API scopes are: event:admin, event:read, member:read, org:read, project:read, project:releases, team:read).

And I don't see data.csv at all.

Also tried to call the script from my laptop at home like this:

python C:\Users\Lenovo\Desktop\Sentry\download_sentry_data.py spectrum-data/react [our API key]

and received the following:

Traceback (most recent call last):
  File "C:\Users\Lenovo\Desktop\Sentry\download_sentry_data.py", line 24, in <module>
    writer.writerow(dict(event))
ValueError: dictionary update sequence element #0 has length 1; 2 is required

BTW the Python version installed both on my work PC and on my laptop is 3.6.5.

It's a pagination issue: the link has results="false".

See https://docs.sentry.io/api/pagination/

ValueError: dictionary update sequence element #0 has length 1; 2 is required

I added a few print statements, turns out I was getting an authorisation error.

https://sentry.io/settings/account/api/auth-tokens/
My auth token did not have the correct scope.

Hi @vladox, thank you so much for useful script.
I improved paging and added date option and more.
https://gist.github.com/siiramone/0d2ed5fe9a78b4f34f70d60d6a9f80b2

Thank you, this helped.

This snippet doesn't deal with pagination. I found that for large number of events (~15000) it only returns (~1500).

Does not work anymore