Support longer pkcs1-oaep secrets
We have run into a case where we need to store a secret longer than 3760 bits. We may eventually support a hybrid encryption scheme, but for now, let's also support the alt.zuul.secrets protocol where we split the secret into 3760 bit chunks and recombine it. The encrypt_secret utility is updated to output a copy-pastable YAML data structure to simplify dealing with long secrets. Change-Id: Ied372572e5aa29fddfb7043bf07df4cd3e39566c
This commit is contained in:
parent
b8cb3da451
commit
9118c01ecf
|
@ -20,24 +20,41 @@ project and ``<source>`` is the name of that project's connection in
|
||||||
the main Zuul configuration file.
|
the main Zuul configuration file.
|
||||||
|
|
||||||
Zuul currently supports one encryption scheme, PKCS#1 with OAEP, which
|
Zuul currently supports one encryption scheme, PKCS#1 with OAEP, which
|
||||||
can not store secrets longer than the key length, 4096 bits. The
|
can not store secrets longer than the 3760 bits (derived from the key
|
||||||
padding used by this scheme ensures that someone examining the
|
length of 4096 bits minus 336 bits of overhead). The padding used by
|
||||||
encrypted data can not determine the length of the plaintext version
|
this scheme ensures that someone examining the encrypted data can not
|
||||||
of the data, except to know that it is not longer than 4096 bits.
|
determine the length of the plaintext version of the data, except to
|
||||||
|
know that it is not longer than 3760 bits (or some multiple thereof).
|
||||||
|
|
||||||
In the config files themselves, Zuul uses an extensible method of
|
In the config files themselves, Zuul uses an extensible method of
|
||||||
specifying the encryption scheme used for a secret so that other
|
specifying the encryption scheme used for a secret so that other
|
||||||
schemes may be added later. To specify a secret, use the
|
schemes may be added later. To specify a secret, use the
|
||||||
``!encrypted/pkcs1-oaep`` YAML tag along with the base64 encoded
|
``!encrypted/pkcs1-oaep`` YAML tag along with the base64 encoded
|
||||||
value. For example::
|
value. For example:
|
||||||
|
|
||||||
|
.. code-block:: yaml
|
||||||
|
|
||||||
- secret:
|
- secret:
|
||||||
name: test_secret
|
name: test_secret
|
||||||
data:
|
data:
|
||||||
password: !encrypted/pkcs1-oaep |
|
password: !encrypted/pkcs1-oaep |
|
||||||
BFhtdnm8uXx7kn79RFL/zJywmzLkT1GY78P3bOtp4WghUFWobkifSu7ZpaV4NeO0s71YUsi1wGZZ
|
BFhtdnm8uXx7kn79RFL/zJywmzLkT1GY78P3bOtp4WghUFWobkifSu7ZpaV4NeO0s71YUsi
|
||||||
...
|
...
|
||||||
|
|
||||||
|
To support secrets longer than 3760 bits, the value after the
|
||||||
|
encryption tag may be a list rather than a scalar. For example:
|
||||||
|
|
||||||
|
.. code-block:: yaml
|
||||||
|
|
||||||
|
- secret:
|
||||||
|
name: long_secret
|
||||||
|
data:
|
||||||
|
password: !encrypted/pkcs1-oaep
|
||||||
|
- er1UXNOD3OqtsRJaP0Wvaqiqx0ZY2zzRt6V9vqIsRaz1R5C4/AEtIad/DERZHwk3Nk+KV
|
||||||
|
...
|
||||||
|
- HdWDS9lCBaBJnhMsm/O9tpzCq+GKRELpRzUwVgU5k822uBwhZemeSrUOLQ8hQ7q/vVHln
|
||||||
|
...
|
||||||
|
|
||||||
Zuul provides a standalone script to make encrypting values easy; it
|
Zuul provides a standalone script to make encrypting values easy; it
|
||||||
can be found at `tools/encrypt_secret.py` in the Zuul source
|
can be found at `tools/encrypt_secret.py` in the Zuul source
|
||||||
directory.
|
directory.
|
||||||
|
|
|
@ -342,16 +342,41 @@ class TestJob(BaseTestCase):
|
||||||
name: pypi-credentials
|
name: pypi-credentials
|
||||||
data:
|
data:
|
||||||
username: test-username
|
username: test-username
|
||||||
|
longpassword: !encrypted/pkcs1-oaep
|
||||||
|
- BFhtdnm8uXx7kn79RFL/zJywmzLkT1GY78P3bOtp4WghUFWobkifSu7ZpaV4NeO0s71Y
|
||||||
|
Usi1wGZZL0LveZjUN0t6OU1VZKSG8R5Ly7urjaSo1pPVIq5Rtt/H7W14Lecd+cUeKb4j
|
||||||
|
oeusC9drN3AA8a4oykcVpt1wVqUnTbMGC9ARMCQP6eopcs1l7tzMseprW4RDNhIuz3CR
|
||||||
|
gd0QBMPl6VDoFgBPB8vxtJw+3m0rqBYZCLZgCXekqlny8s2s92nJMuUABbJOEcDRarzi
|
||||||
|
bDsSXsfJt1y+5n7yOURsC7lovMg4GF/vCl/0YMKjBO5bpv9EM5fToeKYyPGSKQoHOnCY
|
||||||
|
ceb3cAVcv5UawcCic8XjhEhp4K7WPdYf2HVAC/qtxhbpjTxG4U5Q/SoppOJ60WqEkQvb
|
||||||
|
Xs6n5Dvy7xmph6GWmU/bAv3eUK3pdD3xa2Ue1lHWz3U+rsYraI+AKYsMYx3RBlfAmCeC
|
||||||
|
1ve2BXPrqnOo7G8tnUvfdYPbK4Aakk0ds/AVqFHEZN+S6hRBmBjLaRFWZ3QSO1NjbBxW
|
||||||
|
naHKZYT7nkrJm8AMCgZU0ZArFLpaufKCeiK5ECSsDxic4FIsY1OkWT42qEUfL0Wd+150
|
||||||
|
AKGNZpPJnnP3QYY4W/MWcKH/zdO400+zWN52WevbSqZy90tqKDJrBkMl1ydqbuw1E4ZH
|
||||||
|
vIs=
|
||||||
|
- BFhtdnm8uXx7kn79RFL/zJywmzLkT1GY78P3bOtp4WghUFWobkifSu7ZpaV4NeO0s71Y
|
||||||
|
Usi1wGZZL0LveZjUN0t6OU1VZKSG8R5Ly7urjaSo1pPVIq5Rtt/H7W14Lecd+cUeKb4j
|
||||||
|
oeusC9drN3AA8a4oykcVpt1wVqUnTbMGC9ARMCQP6eopcs1l7tzMseprW4RDNhIuz3CR
|
||||||
|
gd0QBMPl6VDoFgBPB8vxtJw+3m0rqBYZCLZgCXekqlny8s2s92nJMuUABbJOEcDRarzi
|
||||||
|
bDsSXsfJt1y+5n7yOURsC7lovMg4GF/vCl/0YMKjBO5bpv9EM5fToeKYyPGSKQoHOnCY
|
||||||
|
ceb3cAVcv5UawcCic8XjhEhp4K7WPdYf2HVAC/qtxhbpjTxG4U5Q/SoppOJ60WqEkQvb
|
||||||
|
Xs6n5Dvy7xmph6GWmU/bAv3eUK3pdD3xa2Ue1lHWz3U+rsYraI+AKYsMYx3RBlfAmCeC
|
||||||
|
1ve2BXPrqnOo7G8tnUvfdYPbK4Aakk0ds/AVqFHEZN+S6hRBmBjLaRFWZ3QSO1NjbBxW
|
||||||
|
naHKZYT7nkrJm8AMCgZU0ZArFLpaufKCeiK5ECSsDxic4FIsY1OkWT42qEUfL0Wd+150
|
||||||
|
AKGNZpPJnnP3QYY4W/MWcKH/zdO400+zWN52WevbSqZy90tqKDJrBkMl1ydqbuw1E4ZH
|
||||||
|
vIs=
|
||||||
password: !encrypted/pkcs1-oaep |
|
password: !encrypted/pkcs1-oaep |
|
||||||
BFhtdnm8uXx7kn79RFL/zJywmzLkT1GY78P3bOtp4WghUFWobkifSu7ZpaV4NeO0s71YUsi1wGZZ
|
BFhtdnm8uXx7kn79RFL/zJywmzLkT1GY78P3bOtp4WghUFWobkifSu7ZpaV4NeO0s71Y
|
||||||
L0LveZjUN0t6OU1VZKSG8R5Ly7urjaSo1pPVIq5Rtt/H7W14Lecd+cUeKb4joeusC9drN3AA8a4o
|
Usi1wGZZL0LveZjUN0t6OU1VZKSG8R5Ly7urjaSo1pPVIq5Rtt/H7W14Lecd+cUeKb4j
|
||||||
ykcVpt1wVqUnTbMGC9ARMCQP6eopcs1l7tzMseprW4RDNhIuz3CRgd0QBMPl6VDoFgBPB8vxtJw+
|
oeusC9drN3AA8a4oykcVpt1wVqUnTbMGC9ARMCQP6eopcs1l7tzMseprW4RDNhIuz3CR
|
||||||
3m0rqBYZCLZgCXekqlny8s2s92nJMuUABbJOEcDRarzibDsSXsfJt1y+5n7yOURsC7lovMg4GF/v
|
gd0QBMPl6VDoFgBPB8vxtJw+3m0rqBYZCLZgCXekqlny8s2s92nJMuUABbJOEcDRarzi
|
||||||
Cl/0YMKjBO5bpv9EM5fToeKYyPGSKQoHOnCYceb3cAVcv5UawcCic8XjhEhp4K7WPdYf2HVAC/qt
|
bDsSXsfJt1y+5n7yOURsC7lovMg4GF/vCl/0YMKjBO5bpv9EM5fToeKYyPGSKQoHOnCY
|
||||||
xhbpjTxG4U5Q/SoppOJ60WqEkQvbXs6n5Dvy7xmph6GWmU/bAv3eUK3pdD3xa2Ue1lHWz3U+rsYr
|
ceb3cAVcv5UawcCic8XjhEhp4K7WPdYf2HVAC/qtxhbpjTxG4U5Q/SoppOJ60WqEkQvb
|
||||||
aI+AKYsMYx3RBlfAmCeC1ve2BXPrqnOo7G8tnUvfdYPbK4Aakk0ds/AVqFHEZN+S6hRBmBjLaRFW
|
Xs6n5Dvy7xmph6GWmU/bAv3eUK3pdD3xa2Ue1lHWz3U+rsYraI+AKYsMYx3RBlfAmCeC
|
||||||
Z3QSO1NjbBxWnaHKZYT7nkrJm8AMCgZU0ZArFLpaufKCeiK5ECSsDxic4FIsY1OkWT42qEUfL0Wd
|
1ve2BXPrqnOo7G8tnUvfdYPbK4Aakk0ds/AVqFHEZN+S6hRBmBjLaRFWZ3QSO1NjbBxW
|
||||||
+150AKGNZpPJnnP3QYY4W/MWcKH/zdO400+zWN52WevbSqZy90tqKDJrBkMl1ydqbuw1E4ZHvIs=
|
naHKZYT7nkrJm8AMCgZU0ZArFLpaufKCeiK5ECSsDxic4FIsY1OkWT42qEUfL0Wd+150
|
||||||
|
AKGNZpPJnnP3QYY4W/MWcKH/zdO400+zWN52WevbSqZy90tqKDJrBkMl1ydqbuw1E4ZH
|
||||||
|
vIs=
|
||||||
''')[0]['secret']
|
''')[0]['secret']
|
||||||
|
|
||||||
conf['_source_context'] = self.context
|
conf['_source_context'] = self.context
|
||||||
|
@ -441,6 +466,12 @@ class TestJob(BaseTestCase):
|
||||||
self.assertEqual(in_repo_job_with_inherit.auth.secrets[0].name,
|
self.assertEqual(in_repo_job_with_inherit.auth.secrets[0].name,
|
||||||
'pypi-credentials')
|
'pypi-credentials')
|
||||||
self.assertIsNone(in_repo_job_with_inherit_false.auth)
|
self.assertIsNone(in_repo_job_with_inherit_false.auth)
|
||||||
|
self.assertEqual(in_repo_job_with_inherit.auth.secrets[0].
|
||||||
|
secret_data['longpassword'],
|
||||||
|
'test-passwordtest-password')
|
||||||
|
self.assertEqual(in_repo_job_with_inherit.auth.secrets[0].
|
||||||
|
secret_data['password'],
|
||||||
|
'test-password')
|
||||||
|
|
||||||
def test_job_inheritance_job_tree(self):
|
def test_job_inheritance_job_tree(self):
|
||||||
tenant = model.Tenant('tenant')
|
tenant = model.Tenant('tenant')
|
||||||
|
|
|
@ -14,10 +14,13 @@
|
||||||
|
|
||||||
import argparse
|
import argparse
|
||||||
import base64
|
import base64
|
||||||
|
import math
|
||||||
import os
|
import os
|
||||||
|
import re
|
||||||
import subprocess
|
import subprocess
|
||||||
import sys
|
import sys
|
||||||
import tempfile
|
import tempfile
|
||||||
|
import textwrap
|
||||||
|
|
||||||
# we to import Request and urlopen differently for python 2 and 3
|
# we to import Request and urlopen differently for python 2 and 3
|
||||||
try:
|
try:
|
||||||
|
@ -68,28 +71,70 @@ def main():
|
||||||
else:
|
else:
|
||||||
plaintext = sys.stdin.read()
|
plaintext = sys.stdin.read()
|
||||||
|
|
||||||
|
plaintext = plaintext.encode("utf-8")
|
||||||
|
|
||||||
pubkey_file = tempfile.NamedTemporaryFile(delete=False)
|
pubkey_file = tempfile.NamedTemporaryFile(delete=False)
|
||||||
try:
|
try:
|
||||||
pubkey_file.write(pubkey.read())
|
pubkey_file.write(pubkey.read())
|
||||||
pubkey_file.close()
|
pubkey_file.close()
|
||||||
|
|
||||||
p = subprocess.Popen(['openssl', 'rsautl', '-encrypt',
|
p = subprocess.Popen(['openssl', 'rsa', '-text',
|
||||||
'-oaep', '-pubin', '-inkey',
|
'-pubin', '-in',
|
||||||
pubkey_file.name],
|
pubkey_file.name],
|
||||||
stdin=subprocess.PIPE,
|
|
||||||
stdout=subprocess.PIPE)
|
stdout=subprocess.PIPE)
|
||||||
(stdout, stderr) = p.communicate(plaintext.encode("utf-8"))
|
(stdout, stderr) = p.communicate()
|
||||||
if p.returncode != 0:
|
if p.returncode != 0:
|
||||||
raise Exception("Return code %s from openssl" % p.returncode)
|
raise Exception("Return code %s from openssl" % p.returncode)
|
||||||
ciphertext = base64.b64encode(stdout)
|
output = stdout.decode('utf-8')
|
||||||
|
m = re.match(r'^Public-Key: \((\d+) bit\)$', output, re.MULTILINE)
|
||||||
|
nbits = int(m.group(1))
|
||||||
|
nbytes = int(nbits / 8)
|
||||||
|
max_bytes = nbytes - 42 # PKCS1-OAEP overhead
|
||||||
|
chunks = int(math.ceil(float(len(plaintext)) / max_bytes))
|
||||||
|
|
||||||
|
ciphertext_chunks = []
|
||||||
|
|
||||||
|
print("Public key length: {} bits ({} bytes)".format(nbits, nbytes))
|
||||||
|
print("Max plaintext length per chunk: {} bytes".format(max_bytes))
|
||||||
|
print("Input plaintext length: {} bytes".format(len(plaintext)))
|
||||||
|
print("Number of chunks: {}".format(chunks))
|
||||||
|
|
||||||
|
for count in range(chunks):
|
||||||
|
chunk = plaintext[int(count * max_bytes):
|
||||||
|
int((count + 1) * max_bytes)]
|
||||||
|
p = subprocess.Popen(['openssl', 'rsautl', '-encrypt',
|
||||||
|
'-oaep', '-pubin', '-inkey',
|
||||||
|
pubkey_file.name],
|
||||||
|
stdin=subprocess.PIPE,
|
||||||
|
stdout=subprocess.PIPE)
|
||||||
|
(stdout, stderr) = p.communicate(chunk)
|
||||||
|
if p.returncode != 0:
|
||||||
|
raise Exception("Return code %s from openssl" % p.returncode)
|
||||||
|
ciphertext_chunks.append(base64.b64encode(stdout).decode('utf-8'))
|
||||||
|
|
||||||
finally:
|
finally:
|
||||||
os.unlink(pubkey_file.name)
|
os.unlink(pubkey_file.name)
|
||||||
|
|
||||||
|
output = textwrap.dedent(
|
||||||
|
'''
|
||||||
|
- secret:
|
||||||
|
name: <name>
|
||||||
|
data:
|
||||||
|
<fieldname>: !encrypted/pkcs1-oaep
|
||||||
|
''')
|
||||||
|
|
||||||
|
twrap = textwrap.TextWrapper(width=79,
|
||||||
|
initial_indent=' ' * 8,
|
||||||
|
subsequent_indent=' ' * 10)
|
||||||
|
for chunk in ciphertext_chunks:
|
||||||
|
chunk = twrap.fill('- ' + chunk)
|
||||||
|
output += chunk + '\n'
|
||||||
|
|
||||||
if args.outfile:
|
if args.outfile:
|
||||||
with open(args.outfile, "wb") as f:
|
with open(args.outfile, "w") as f:
|
||||||
f.write(ciphertext)
|
f.write(output)
|
||||||
else:
|
else:
|
||||||
print(ciphertext.decode("utf-8"))
|
print(output)
|
||||||
|
|
||||||
|
|
||||||
if __name__ == '__main__':
|
if __name__ == '__main__':
|
||||||
|
|
|
@ -223,7 +223,11 @@ class EncryptedPKCS1_OAEP(yaml.YAMLObject):
|
||||||
yaml_loader = yaml.SafeLoader
|
yaml_loader = yaml.SafeLoader
|
||||||
|
|
||||||
def __init__(self, ciphertext):
|
def __init__(self, ciphertext):
|
||||||
self.ciphertext = base64.b64decode(ciphertext)
|
if isinstance(ciphertext, list):
|
||||||
|
self.ciphertext = [base64.b64decode(x.value)
|
||||||
|
for x in ciphertext]
|
||||||
|
else:
|
||||||
|
self.ciphertext = base64.b64decode(ciphertext)
|
||||||
|
|
||||||
def __ne__(self, other):
|
def __ne__(self, other):
|
||||||
return not self.__eq__(other)
|
return not self.__eq__(other)
|
||||||
|
@ -238,8 +242,14 @@ class EncryptedPKCS1_OAEP(yaml.YAMLObject):
|
||||||
return cls(node.value)
|
return cls(node.value)
|
||||||
|
|
||||||
def decrypt(self, private_key):
|
def decrypt(self, private_key):
|
||||||
return encryption.decrypt_pkcs1_oaep(self.ciphertext,
|
if isinstance(self.ciphertext, list):
|
||||||
private_key).decode('utf8')
|
return ''.join([
|
||||||
|
encryption.decrypt_pkcs1_oaep(chunk, private_key).
|
||||||
|
decode('utf8')
|
||||||
|
for chunk in self.ciphertext])
|
||||||
|
else:
|
||||||
|
return encryption.decrypt_pkcs1_oaep(self.ciphertext,
|
||||||
|
private_key).decode('utf8')
|
||||||
|
|
||||||
|
|
||||||
class NodeSetParser(object):
|
class NodeSetParser(object):
|
||||||
|
|
Loading…
Reference in New Issue