Cache decryption results

The scheduler decrypts secrets during layout generation and job
freezing. In case we have many secrets in the system the time spent on
repeated secrets decryptions is getting relevant for zuul's
performance from the users point of view. However as those secrets are
updated infrequently we can improve this dramatically by using
lru_cache on the decryption function.

This will speed up both layout generation and job freezing as here
many secrets can be decrypted. Further it speeds up reconfigurations
as there many layout generations and job freezings can occur during
re-enqueueing of the queue items.

I've prototyped a test case that enqueues one job with 8 inheritance
layers and 9 secrets (we do have many jobs with that
characteristic). In this example the caching reduced the job freezing
time from 194ms without caching [1] to 8ms with caching [2].

[1] Without caching:
2018-07-15 14:19:02,797 zuul.layout  DEBUG  Collecting jobs test for <Change 0x7fa11875bf98 1,1>
(...)
2018-07-15 14:19:02,805 zuul.layout  DEBUG  Collected jobs test for <Change 0x7fa11875bf98 1,1>
2018-07-15 14:19:02,991 zuul.layout  DEBUG  Froze job test for <Change 0x7fa11875bf98 1,1>

[2] With caching:
2018-07-15 14:16:43,833 zuul.layout  DEBUG  Collecting jobs test for <Change 0x7fe1700bfa90 1,1>
(...)
2018-07-15 14:16:43,840 zuul.layout  DEBUG  Collected jobs test for <Change 0x7fe1700bfa90 1,1>
2018-07-15 14:16:43,841 zuul.layout  DEBUG  Froze job test for <Change 0x7fe1700bfa90 1,1>

Change-Id: I1bf451e5a6e0adfdb849738f9987436ba7fb59a3
This commit is contained in:
Tobias Henkel 2018-07-15 15:54:28 +02:00
parent b354547c52
commit af90d5f5be
No known key found for this signature in database
GPG Key ID: 03750DEC158E5FA2
1 changed files with 3 additions and 0 deletions

View File

@ -17,6 +17,7 @@ from cryptography.hazmat.primitives.asymmetric import rsa
from cryptography.hazmat.primitives import serialization
from cryptography.hazmat.primitives.asymmetric import padding
from cryptography.hazmat.primitives import hashes
from functools import lru_cache
# https://cryptography.io/en/latest/hazmat/primitives/asymmetric/rsa/#generation
@ -95,6 +96,8 @@ def deserialize_rsa_keypair(data):
# https://cryptography.io/en/latest/hazmat/primitives/asymmetric/rsa/#decryption
# lru_cache performs best if maxsize is a power of two
@lru_cache(maxsize=1024)
def decrypt_pkcs1_oaep(ciphertext, private_key):
"""Decrypt PKCS#1 (RSAES-OAEP) encoded ciphertext