Zuul CLI: allow access via REST

Users can set the [webclient] section in their zuul.conf file so that the CLI
relies on REST calls rather than RPC. The CLI accepts a new --auth-token
argument allowing remote users to use privileged REST endpoints.

Change-Id: I5a07fccfd787246c4c494db592b575fbdf90ddb1
This commit is contained in:
mhuin 2019-02-12 12:48:31 +01:00 committed by Matthieu Huin
parent e370e97aeb
commit 9d86c00111
8 changed files with 298 additions and 65 deletions

View File

@ -1,14 +1,12 @@
:title: Zuul Client
.. _zuul-client:
Zuul Client
===========
Zuul includes a simple command line client that may be used by
administrators to affect Zuul's behavior while running. It must be
run on a host that has access to the Gearman server (e.g., locally on
the Zuul host).
Zuul includes a simple command line client that may be used to affect Zuul's
behavior while running. It must be run on a host that has access to the Gearman
server (e.g., locally on the Zuul host), or on a host with access to Zuul's web
server.
Configuration
-------------
@ -16,6 +14,15 @@ Configuration
The client uses the same zuul.conf file as the server, and will look
for it in the same locations if not specified on the command line.
If both sections are present, the ``gearman`` section takes precedence over the
``webclient`` section, meaning the client will execute commands using the Gearman
server over the REST API.
It is also possible to run the client without a configuration file, by using the
``--zuul-url`` option to specify the base URL of the Zuul web server.
.. note:: Not all commands are available through the REST API.
Usage
-----
The general options that apply to all subcommands are:
@ -109,6 +116,9 @@ for these more advanced operations.
Promote
^^^^^^^
.. note:: This command is only available through a Gearman connection.
.. program-output:: zuul promote --help
Example::
@ -119,6 +129,9 @@ Note that the format of changes id is <number>,<patchset>.
Show
^^^^
.. note:: This command is only available through a Gearman connection.
.. program-output:: zuul show --help
Example::
@ -127,6 +140,9 @@ Example::
tenant-conf-check
^^^^^^^^^^^^^^^^^
.. note:: This command is only available through a Gearman connection.
.. program-output:: zuul tenant-conf-check --help
Example::
@ -138,6 +154,11 @@ case of errors detected.
create-auth-token
^^^^^^^^^^^^^^^^^
.. note:: This command is only available if an authenticator is configured in
``zuul.conf``. Furthermore the authenticator's configuration must
include a signing secret.
.. program-output:: zuul create-auth-token --help
Example::

View File

@ -944,6 +944,45 @@ Operation
To start the web server, run ``zuul-web``. To stop it, kill the
PID which was saved in the pidfile specified in the configuration.
Web Client
----------
Zuul's command line client may be configured to make calls to Zuul's web
server. The client will then look for a ``zuul.conf`` file with a ``webclient``
section to set up the connection over HTTP.
Configuration
~~~~~~~~~~~~~
.. attr:: webclient
.. attr:: url
The root URL of Zuul's web server.
.. attr:: verify_ssl
:default: true
Enforce SSL verification when sending requests over to Zuul's web server.
This should only be disabled when working with test servers.
Configuration
~~~~~~~~~~~~~
In addition to the common configuration sections, the following
sections of ``zuul.conf`` are used by the web server:
.. attr:: web
.. attr:: listen_address
:default: 127.0.0.1
IP address or domain name on which to listen.
.. attr:: log_config
Path to log config file for the web server process.
Finger Gateway
--------------

View File

@ -0,0 +1 @@
.. include:: ../admin/client.rst

View File

@ -17,6 +17,7 @@ configure it to meet your needs.
jobs
encryption
web
client-user
badges
howtos
vulnerabilities

View File

@ -40,6 +40,10 @@ port=9000
static_cache_expiry=0
status_url=https://zuul.example.com/status
[webclient]
url=https://zuul.example.com
verify_ssl=true
[auth zuul_operator]
driver=HS256
allow_authz_override=true

View File

@ -1,9 +1,10 @@
---
features:
- |
Add an endpoint protection mechanism to zuul-web's REST API, based on the JWT
standard. A user can access protected endpoints with a valid bearer token.
The actions associated to these endpoints are tenant-scoped via a token claim.
Zuul supports token signatures using the HS256 or RS256 algorithms. External
JWKS are also supported.
Current protected endpoints are "autohold", "enqueue" and "dequeue".
Allow users to perform tenant-scoped, privileged actions either through
zuul-web's REST API or zuul's client, based on the JWT standard. The users
need a valid bearer token to perform such actions; the scope is set via a
token claim.
Zuul supports token signing and validation using the HS256 or RS256 algorithms.
External JWKS are also supported for token validation only.
Current tenant-scoped actions are "autohold", "enqueue" and "dequeue".

View File

@ -24,22 +24,158 @@ import re
import sys
import time
import textwrap
import requests
import urllib.parse
import zuul.rpcclient
import zuul.cmd
from zuul.lib.config import get_default
# todo This should probably live somewhere else
class ZuulRESTClient(object):
"""Basic client for Zuul's REST API"""
def __init__(self, url, verify=False, auth_token=None):
self.url = url
self.auth_token = auth_token
self.base_url = urllib.parse.urljoin(self.url, '/api/')
self.verify = verify
def _check_status(self, req):
try:
req.raise_for_status()
except Exception as e:
if req.status_code == 401:
print('Unauthorized - your token might be invalid or expired.')
elif req.status_code == 403:
print('Insufficient privileges to perform the action.')
else:
print('Unknown error: "%e"' % e)
def autohold(self, tenant, project, job, change, ref,
reason, count, node_hold_expiration):
if not self.auth_token:
raise Exception('Auth Token required')
args = {"reason": reason,
"count": count,
"job": job,
"change": change,
"ref": ref,
"node_hold_expiration": node_hold_expiration}
url = urllib.parse.urljoin(
self.base_url,
'tenant/%s/project/%s/autohold' % (tenant, project))
req = requests.post(
url, json=args, verify=self.verify,
headers={'Authorization': 'Bearer %s' % self.auth_token})
self._check_status(req)
return req.json()
def autohold_list(self, tenant):
if not tenant:
raise Exception('"--tenant" argument required')
url = urllib.parse.urljoin(
self.base_url,
'tenant/%s/autohold' % tenant)
req = requests.get(url, verify=self.verify)
self._check_status(req)
resp = req.json()
# reformat the answer to match RPC format
ret = {}
for d in resp:
key = ','.join([d['tenant'],
d['project'],
d['job'],
d['ref_filter']])
ret[key] = (d['count'], d['reason'], d['node_hold_expiration'])
return ret
def enqueue(self, tenant, pipeline, project, trigger, change):
if not self.auth_token:
raise Exception('Auth Token required')
args = {"trigger": trigger,
"change": change,
"pipeline": pipeline}
url = urllib.parse.urljoin(
self.base_url,
'tenant/%s/project/%s/enqueue' % (tenant, project))
req = requests.post(
url, json=args, verify=self.verify,
headers={'Authorization': 'Bearer %s' % self.auth_token})
self._check_status(req)
return req.json()
def enqueue_ref(self, tenant, pipeline, project,
trigger, ref, oldrev, newrev):
if not self.auth_token:
raise Exception('Auth Token required')
args = {"trigger": trigger,
"ref": ref,
"oldrev": oldrev,
"newrev": newrev,
"pipeline": pipeline}
url = urllib.parse.urljoin(
self.base_url,
'tenant/%s/project/%s/enqueue' % (tenant, project))
req = requests.post(
url, json=args, verify=self.verify,
headers={'Authorization': 'Bearer %s' % self.auth_token})
self._check_status(req)
return req.json()
def dequeue(self, tenant, pipeline, project, change=None, ref=None):
if not self.auth_token:
raise Exception('Auth Token required')
args = {"pipeline": pipeline}
if change and not ref:
args['change'] = change
elif ref and not change:
args['ref'] = ref
else:
raise Exception('need change OR ref')
url = urllib.parse.urljoin(
self.base_url,
'tenant/%s/project/%s/dequeue' % (tenant, project))
req = requests.post(
url, json=args, verify=self.verify,
headers={'Authorization': 'Bearer %s' % self.auth_token})
self._check_status(req)
return req.json()
def promote(self, *args, **kwargs):
raise NotImplementedError(
'This action is unsupported by the REST API')
def get_running_jobs(self, *args, **kwargs):
raise NotImplementedError(
'This action is unsupported by the REST API')
class Client(zuul.cmd.ZuulApp):
app_name = 'zuul'
app_description = 'Zuul RPC client.'
app_description = 'Zuul client.'
log = logging.getLogger("zuul.Client")
def createParser(self):
parser = super(Client, self).createParser()
parser.add_argument('-v', dest='verbose', action='store_true',
help='verbose output')
parser.add_argument('--auth-token', dest='auth_token',
required=False,
default=None,
help='Authentication Token, needed if using the'
'REST API')
parser.add_argument('--zuul-url', dest='zuul_url',
required=False,
default=None,
help='Zuul API URL, needed if using the '
'REST API without a configuration file')
parser.add_argument('--insecure', dest='insecure_ssl',
required=False,
action='store_false',
help='Do not verify SSL connection to Zuul, '
'when using the REST API (Defaults to False)')
subparsers = parser.add_subparsers(title='commands',
description='valid commands',
@ -73,6 +209,8 @@ class Client(zuul.cmd.ZuulApp):
cmd_autohold_list = subparsers.add_parser(
'autohold-list', help='list autohold requests')
cmd_autohold_list.add_argument('--tenant', help='tenant name',
required=False)
cmd_autohold_list.set_defaults(func=self.autohold_list)
cmd_enqueue = subparsers.add_parser('enqueue', help='enqueue a change')
@ -233,23 +371,50 @@ class Client(zuul.cmd.ZuulApp):
def main(self):
self.parseArguments()
self.readConfig()
if not self.args.zuul_url:
self.readConfig()
self.setup_logging()
self.server = self.config.get('gearman', 'server')
self.port = get_default(self.config, 'gearman', 'port', 4730)
self.ssl_key = get_default(self.config, 'gearman', 'ssl_key')
self.ssl_cert = get_default(self.config, 'gearman', 'ssl_cert')
self.ssl_ca = get_default(self.config, 'gearman', 'ssl_ca')
if self.args.func():
sys.exit(0)
else:
sys.exit(1)
def get_client(self):
if self.args.zuul_url:
self.log.debug('Zuul URL provided as argument, using REST client')
client = ZuulRESTClient(self.args.zuul_url,
self.args.insecure_ssl,
self.args.auth_token)
return client
conf_sections = self.config.sections()
if 'gearman' in conf_sections:
self.log.debug('gearman section found in config, using RPC client')
server = self.config.get('gearman', 'server')
port = get_default(self.config, 'gearman', 'port', 4730)
ssl_key = get_default(self.config, 'gearman', 'ssl_key')
ssl_cert = get_default(self.config, 'gearman', 'ssl_cert')
ssl_ca = get_default(self.config, 'gearman', 'ssl_ca')
client = zuul.rpcclient.RPCClient(
server, port, ssl_key,
ssl_cert, ssl_ca)
elif 'webclient' in conf_sections:
self.log.debug('web section found in config, using REST client')
server = get_default(self.config, 'webclient', 'url', None)
verify = get_default(self.config, 'webclient', 'verify_ssl',
self.args.insecure_ssl)
client = ZuulRESTClient(server, verify,
self.args.auth_token)
else:
print('Unable to find a way to connect to Zuul, add a "gearman" '
'or "web" section to your configuration file')
sys.exit(1)
if server is None:
print('Missing "server" configuration value')
sys.exit(1)
return client
def autohold(self):
client = zuul.rpcclient.RPCClient(
self.server, self.port, self.ssl_key, self.ssl_cert, self.ssl_ca)
if self.args.change and self.args.ref:
print("Change and ref can't be both used for the same request")
return False
@ -258,20 +423,21 @@ class Client(zuul.cmd.ZuulApp):
return False
node_hold_expiration = self.args.node_hold_expiration
r = client.autohold(tenant=self.args.tenant,
project=self.args.project,
job=self.args.job,
change=self.args.change,
ref=self.args.ref,
reason=self.args.reason,
count=self.args.count,
node_hold_expiration=node_hold_expiration)
client = self.get_client()
r = client.autohold(
tenant=self.args.tenant,
project=self.args.project,
job=self.args.job,
change=self.args.change,
ref=self.args.ref,
reason=self.args.reason,
count=self.args.count,
node_hold_expiration=node_hold_expiration)
return r
def autohold_list(self):
client = zuul.rpcclient.RPCClient(
self.server, self.port, self.ssl_key, self.ssl_cert, self.ssl_ca)
autohold_requests = client.autohold_list()
client = self.get_client()
autohold_requests = client.autohold_list(tenant=self.args.tenant)
if len(autohold_requests.keys()) == 0:
print("No autohold requests found")
@ -295,35 +461,35 @@ class Client(zuul.cmd.ZuulApp):
return True
def enqueue(self):
client = zuul.rpcclient.RPCClient(
self.server, self.port, self.ssl_key, self.ssl_cert, self.ssl_ca)
r = client.enqueue(tenant=self.args.tenant,
pipeline=self.args.pipeline,
project=self.args.project,
trigger=self.args.trigger,
change=self.args.change)
client = self.get_client()
r = client.enqueue(
tenant=self.args.tenant,
pipeline=self.args.pipeline,
project=self.args.project,
trigger=self.args.trigger,
change=self.args.change)
return r
def enqueue_ref(self):
client = zuul.rpcclient.RPCClient(
self.server, self.port, self.ssl_key, self.ssl_cert, self.ssl_ca)
r = client.enqueue_ref(tenant=self.args.tenant,
pipeline=self.args.pipeline,
project=self.args.project,
trigger=self.args.trigger,
ref=self.args.ref,
oldrev=self.args.oldrev,
newrev=self.args.newrev)
client = self.get_client()
r = client.enqueue_ref(
tenant=self.args.tenant,
pipeline=self.args.pipeline,
project=self.args.project,
trigger=self.args.trigger,
ref=self.args.ref,
oldrev=self.args.oldrev,
newrev=self.args.newrev)
return r
def dequeue(self):
client = zuul.rpcclient.RPCClient(
self.server, self.port, self.ssl_key, self.ssl_cert, self.ssl_ca)
r = client.dequeue(tenant=self.args.tenant,
pipeline=self.args.pipeline,
project=self.args.project,
change=self.args.change,
ref=self.args.ref)
client = self.get_client()
r = client.dequeue(
tenant=self.args.tenant,
pipeline=self.args.pipeline,
project=self.args.project,
change=self.args.change,
ref=self.args.ref)
return r
def create_auth_token(self):
@ -373,16 +539,15 @@ class Client(zuul.cmd.ZuulApp):
sys.exit(err_code)
def promote(self):
client = zuul.rpcclient.RPCClient(
self.server, self.port, self.ssl_key, self.ssl_cert, self.ssl_ca)
r = client.promote(tenant=self.args.tenant,
pipeline=self.args.pipeline,
change_ids=self.args.changes)
client = self.get_client()
r = client.promote(
tenant=self.args.tenant,
pipeline=self.args.pipeline,
change_ids=self.args.changes)
return r
def show_running_jobs(self):
client = zuul.rpcclient.RPCClient(
self.server, self.port, self.ssl_key, self.ssl_cert, self.ssl_ca)
client = self.get_client()
running_items = client.get_running_jobs()
if len(running_items) == 0:

View File

@ -62,7 +62,8 @@ class RPCClient(object):
'node_hold_expiration': node_hold_expiration}
return not self.submitJob('zuul:autohold', data).failure
def autohold_list(self):
# todo allow filtering per tenant, like in the REST API
def autohold_list(self, *args, **kwargs):
data = {}
job = self.submitJob('zuul:autohold_list', data)
if job.failure: