Last active
March 21, 2018 12:59
-
-
Save alistairncoles/27c086457a1251e9f7f4b85f57a3b553 to your computer and use it in GitHub Desktop.
hack object to remove x-delete-at
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
#!/usr/bin/python | |
import os | |
import sys | |
from swift.common.utils import Timestamp | |
from swift.obj import diskfile | |
# | |
# make this file executable. | |
# | |
# hack_object_metadata.py <object data dir> | |
# | |
# e.g.: | |
# hack_object_metadata.py /mnt/sdb1/2/node/sdb2/objects/425/508/6a5875aa6f189856ad72817fa8788508 | |
# | |
def remove_x_delete_at(data_dir): | |
for filename in os.listdir(data_dir): | |
path = os.path.join(data_dir, filename) | |
if not filename.endswith(('.data', '.meta')): | |
print('Ignoring %s' % path) | |
continue | |
with open(path, 'wb') as fd: | |
metadata = diskfile.read_metadata(fd) | |
popped = [] | |
for k in metadata.keys(): | |
if k.lower() == 'x-delete-at': | |
popped.append(metadata.pop(k)) | |
diskfile.write_metadata(fd, metadata) | |
if popped: | |
isoformats = [Timestamp(p).isoformat for p in popped] | |
print('Removed x-delete-at value(s) in %s:' % path) | |
for p, iso in zip(popped, isoformats): | |
print(' %s (%s)' % (p, iso)) | |
else: | |
print('No x-delete-at values found in %s' % path) | |
if __name__ == '__main__': | |
remove_x_delete_at(sys.argv[1]) |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
This is not a great idea in production...
This script would need to be applied to every replica of an object for the expiry-reversal to be durable. That is hard to achieve when replicas may be on handoffs or old primaries after a rebalance. The object replicator would NOT replicate the change to a single replica. Unchanged replicas would still 404 and the proxy would eventually (I think) find an 'unexpired' replica - but if the unexpired replica were lost then the original expired replicas would be replicated and the object would become expired again.
If the expirer ran it would delete any replicas not fixed by this script. Perhaps the fixed replicas would then be replicated to replace the removed ones...except they would conflict with the tombstones at same timestamp, so perhaps not. The object would have inconsistent replicas and not be durable.
Perhaps a .meta file could be synthesised? I'm not sure that is possible via the diskfile interface for an expired object.
Conclusion: I would not recommend doing this in a production system.