Generating. storing and keeping inventory of hundreds of terabyte large database backups is a challenge by itself which we do on daily basis. It’s even a bigger challenge to store them encrypted which is the requirement sometimes. The backup data is being pushed through a socket generated by percona-xtrabackup. On the receiving end this is stored on SAN devices. Because of the size of the backups it’s unfeasible to reread the binary and encrypt them after the full backup data arrived. We had to come up with a solution which can do the whole thing in the stream time while minimising the amount of unnecessary disk reads and writes…
Digital envelope
The idea of digital envelope is not a new concept. Actually it’s very simple if you understand it. It generates a random symmetric key which is being used to do stream cypher and encrypt the data with a symmetric key block cypher (like ARC4). During this operation your envelope is “open”. When it’s finished the public key of the asymmetric key pair you encode your passphrase and store it with the encrypted data.
Only in the possession of the private key can the envelope be opened again. The decryption process is straightforwards from there. You retrieve the passphrase by decrypting the encrypted passphrase and use that to decipher the data with the chosen block cypher (ARC4 in the example)
Implementation
I published the basic implementation on my github: https://github.com/charlesnagy/python-digitalenvelope
Example usage (writing/encryption):
|
1 2 3 4 5 6 7 8 |
from DigitalEnvelope import BaseEnvelope _data = open('any_arbitrary_filename', 'wb') with BaseEnvelope(data=_data) as _e: _e.write('some data you want to encrypt') # Do this _outside of the with context and save it somewhere with the encrypted file _encrypted_passphrase = _e.passphrase |
Example usage (read/decrypt):
|
1 2 3 4 |
from DigitalEnvelope import BaseEnvelope with BaseEnvelope(passphrase=self.backup.encrypted_passphrase, data=_input).open(DBAsettings.PRIVATE_KEY_FILE) as _e: print _e.read() |
Recent comments