Skip to content

Instantly share code, notes, and snippets.

@Cactiw
Created December 17, 2022 21:47
Show Gist options
  • Select an option

  • Save Cactiw/c2397561966e2f343b0563a6c5b7f17f to your computer and use it in GitHub Desktop.

Select an option

Save Cactiw/c2397561966e2f343b0563a6c5b7f17f to your computer and use it in GitHub Desktop.
This is quick fix for sentry generated backup.json file.
import json
FILENAME = "backup.json"
FILENAME_FIXED = "backup_fixed.json"
def restore():
with open(FILENAME, "r") as f:
lines = f.readlines()
lines = lines[lines.index('[\n'):]
with open(FILENAME_FIXED, "w") as f:
f.writelines(lines)
with open(FILENAME_FIXED, "r") as f:
data = json.load(f)
for d in data:
value = d.get("fields", {}).get("value")
if value is not None and not isinstance(value, str):
d["fields"]["value"] = str(value)
with open(FILENAME_FIXED, "w") as f:
f.write(json.dumps(data, indent=4, ensure_ascii=False))
return data
if __name__ == "__main__":
restore()
@Lucaber
Copy link
Copy Markdown

Lucaber commented Jan 4, 2023

I also had to remove every sentry.useremail block from the backup file.

Error while importing to an empty database:

django.db.utils.IntegrityError: UniqueViolation('duplicate key value violates unique constraint "sentry_useremail_user_id_email_ade975f1_uniq"\nDETAIL:  Key (user_id, email)=(3, test@example.com) already exists.\n')
SQL: UPDATE "sentry_useremail" SET "user_id" = %s, "email" = %s, "validation_hash" = %s, "date_hash_added" = %s, "is_verified" = %s WHERE "sentry_useremail"."id" = %s

Email addresses are probably already added by the sentry.user backups...

@matthewbyrne
Copy link
Copy Markdown

This script caused us issues with boolean values being converted to uppercase string "False". The import didn't like them.

I've adapted the script to account for this:

import json

FILENAME = "backup.json"
FILENAME_FIXED = "backup_fixed_2.json"


def restore():
    with open(FILENAME, "r") as f:
        lines = f.readlines()
        lines = lines[lines.index('[\n'):]
    with open(FILENAME_FIXED, "w") as f:
        f.writelines(lines)
    with open(FILENAME_FIXED, "r") as f:
        data = json.load(f)
        for d in data:
            value = d.get("fields", {}).get("value")
            if isinstance(value, bool):
                d["fields"]["value"] = str(value).lower()
            elif value is not None and not isinstance(value, str):
                d["fields"]["value"] = str(value)
    with open(FILENAME_FIXED, "w") as f:
        f.write(json.dumps(data, indent=4, ensure_ascii=False))
    return data


if __name__ == "__main__":
    restore()

@Lucaber
Copy link
Copy Markdown

Lucaber commented Jan 31, 2023

This script is broken and imports booleans as (encoded) strings into the database.
A True should be encoded as "gAKILg==" but gets inserted as "gAJYBAAAAFRydWVxAC4=" into the database.
After hours of debugging i finally managed to fix my installation with multiple manual db queries.
Sadly i dont have every query in my history, but some are here:

update sentry_option set value = 'gAKILg==' where value = 'gAJYBAAAAFRydWVxAC4=';
update sentry_organizationoptions set value = 'gAKILg==' where value = 'gAJYBAAAAFRydWVxAC4=';
update sentry_projectoptions set value = 'gAKILg==' where value = 'gAJYBAAAAFRydWVxAC4=';

@matthewbyrne
Copy link
Copy Markdown

@Lucaber , my adapted script above should be of help if you have your original exported backup.

@Lucaber
Copy link
Copy Markdown

Lucaber commented Feb 1, 2023

If i remember correctly, numbers were also saved incorrectly as base64 encoded strings with quotes. 8 -> "8" | base64.
This resulted in broken "sentry:option-epoch" options.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment