Boto, s3 and remote metadata
Using boto to store files on s3 might be a little bit confusing when it comes to store metadata for files. Normally you would expect that key.set_metadata
saves data remotely on s3. In fact set_metadata
stores your keys locally in python dict.
>>> bucket.new_key('testing-file')
>>> key.set_contents_from_string('testing content')
>>> key.set_metadata('hello', 'from metadata')
>>> key.get_metadata('hello')
>>> 'from metadata'
In example above you can get metadata form key instance, but what if you get your file from s3 again? Metadata will be a empty dict.
>>> key.get_metadata('hello') is None
>>> True
set_remote_metadata
is the method you probably are looking for.
>>> key.set_remote_metadata?
Type: instancemethod
String Form:<bound ey:="" ="" key.set_remote_metadata="" method="" of="" testing-file="">>
File: /lib/python2.7/site-packages/boto/s3/key.py
Definition: key.set_remote_metadata(self, metadata_plus, metadata_minus, preserve_acl, headers=None)
Docstring: no docstring=""
Interface to that method is more then weird but it does the job.
>>> key = bucket.get_key('testing-file')
>>> key.set_remote_metadata({'hello': 'this is remote metadata'}, {}, True)
>>> remote_key = bucket.get_key('testing-file')
>>> remote_key.metadata
>>> {'hello': u'this is remote metadata'}
In fact boto stores files metadata in headers by adding x-amx-meta prefix, you can check that by downloading file directly.
>>> requests.get('https://you_bucket.s3.amazonaws.com/testing-file').headers['x-amz-meta-hello']
>>> this is remote metadata'
If you want to update file headers without meta prefix, you can do it easily:
>>> key.set_remote_metadata({'Content-Type': 'custom/type'}, {}, True)
>>> requests.get('https://{}.s3.amazonaws.com/testing-file').headers['content-type']
>>> 'custom/type'
Read other posts