openstack/blazar
Code Issues Proposed changes

Add usage enforcement framework

Browse Source 
Implements filter-based architecture for usage enforcement.

One filter is initially included: MaxLeaseDurationFilter allows
operators to define the maximum duration of a lease in seconds. All
filters must be enabled and configured in the [enforcement] group of
blazar.conf.

When ending a lease, the allocations currently associated with the
reservations of the lease are sent to the enforcement layer for
processing. This works fine when deleting a lease, but when a lease
naturally ends, that is, when the current time is greater than the
lease's recorded end time, the allocation lookup returns 0 results
because we have a filter on the time of the lease/reservation by
default, which starts from datetime.now. This is really not what we want
to do when explicitly filtering on a lease/reservation, so this updates
the generic query function to not apply this filtering in this case.

It turns out, it was possible for usage enforcement's on_end hook to run
multiple times if the first reservation teardown failed. In this
circumstance, the end_lease event would be IN_PROGRESS but the
reservations would be in an error state. The code only checks if any
reservation is DELETED before trying to run on_end again.

To fix this, only try to run on_end if we detected that there was an
UNDONE end_lease event; that's the only time in which we'll actually be
responsible for running this logic.

This also updates the delete logic to be a bit more consistent with the
internal way we track states. It is not enough to check dates, because
it could be that Blazar never got around to actually starting the
leases; we should be checking events. This also makes the logic a bit
easier to understand.

Change-Id: Ic106bee4fc3f5c401c4f8d8ecb1c4e735560bcc2
Co-Authored-By: Jason Anderson <jasonanderson@uchicago.edu>
Co-Authored-By: Pierre Riteau <pierre@stackhpc.com>
Implements: blueprint flexible-reservation-usage-enforcement
This commit is contained in:
Jacob Colleran 2020-08-10 11:26:49 -05:00 committed by Pierre Riteau
parent 77cf9028b4
commit d3f77fd0c5
24 changed files with 1586 additions and 284 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

8
blazar/api/v1/utils.py
View File

@ -25,6 +25,7 @@ from werkzeug import datastructures
from blazar.api import context
from blazar.api.v1 import api_version_request as api_version
from blazar.db import exceptions as db_exceptions
from blazar.enforcement import exceptions as enforcement_exceptions
from blazar import exceptions as ex
from blazar.i18n import _
from blazar.manager import exceptions as manager_exceptions
@ -91,9 +92,12 @@ class Rest(flask.Blueprint):
except ex.BlazarException as e:
return bad_request(e)
except messaging.RemoteError as e:
# Get the exception from manager and common exceptions
cls = getattr(manager_exceptions, e.exc_type,
# Get the exception from enforcement, manager and
# common exceptions
cls = getattr(enforcement_exceptions, e.exc_type,
getattr(ex, e.exc_type, None))
cls = cls or getattr(manager_exceptions, e.exc_type,
getattr(ex, e.exc_type, None))
cls = cls or getattr(opst_exceptions, e.exc_type,
getattr(ex, e.exc_type, None))
if cls is not None:

96
blazar/db/sqlalchemy/utils.py
View File

@ -14,6 +14,7 @@
# License for the specific language governing permissions and limitations
# under the License.
from collections import defaultdict
import sys
import sqlalchemy as sa
@ -93,28 +94,87 @@ def get_reservations_by_host_ids(host_ids, start_date, end_date):
return query.all()
def get_reservations_for_allocations(session, start_date, end_date,
lease_id=None, reservation_id=None):
fields = ['id', 'status', 'lease_id', 'start_date',
'end_date', 'lease_name', 'project_id']
reservations_query = (session.query(
models.Reservation.id,
models.Reservation.status,
models.Reservation.lease_id,
models.Lease.start_date,
models.Lease.end_date,
models.Lease.name,
models.Lease.project_id)
.join(models.Lease))
if lease_id:
reservations_query = reservations_query.filter(
models.Reservation.lease_id == lease_id)
if reservation_id:
reservations_query = reservations_query.filter(
models.Reservation.id == reservation_id)
# Only enforce time restrictions if we're not targeting a specific
# lease or reservation.
if not (lease_id or reservation_id):
border0 = models.Lease.end_date >= start_date
border1 = models.Lease.start_date <= end_date
reservations_query = reservations_query.filter(
sa.and_(border0, border1))
return [dict(zip(fields, r)) for r in reservations_query.all()]
def get_reservation_allocations_by_host_ids(host_ids, start_date, end_date,
lease_id=None,
reservation_id=None):
session = get_session()
border0 = start_date <= models.Lease.end_date
border1 = models.Lease.start_date <= end_date
query = (session.query(models.Reservation.id,
models.Reservation.lease_id,
models.ComputeHostAllocation.compute_host_id)
.join(models.Lease,
models.Lease.id == models.Reservation.lease_id)
.join(models.ComputeHostAllocation,
models.ComputeHostAllocation.reservation_id ==
models.Reservation.id)
.filter(models.ComputeHostAllocation.compute_host_id
.in_(host_ids))
.filter(sa.and_(border0, border1)))
if lease_id:
query = query.filter(models.Reservation.lease_id == lease_id)
if reservation_id:
query = query.filter(models.Reservation.id == reservation_id)
return query.all()
reservations = get_reservations_for_allocations(
session, start_date, end_date, lease_id, reservation_id)
allocations_query = (session.query(
models.ComputeHostAllocation.reservation_id,
models.ComputeHostAllocation.compute_host_id)
.filter(models.ComputeHostAllocation.compute_host_id.in_(host_ids))
.filter(models.ComputeHostAllocation.reservation_id.in_(
list(set([x['id'] for x in reservations])))))
allocations = defaultdict(list)
for row in allocations_query.all():
allocations[row[0]].append(row[1])
allocs = []
for r in reservations:
allocs.append((r['id'], r['lease_id'], allocations[r['id']][0]))
return allocs
def get_reservation_allocations_by_fip_ids(fip_ids, start_date, end_date,
lease_id=None, reservation_id=None):
session = get_session()
reservations = get_reservations_for_allocations(
session, start_date, end_date, lease_id, reservation_id)
allocations_query = (session.query(
models.FloatingIPAllocation.reservation_id,
models.FloatingIPAllocation.floatingip_id)
.filter(models.FloatingIPAllocation.floatingip_id.in_(fip_ids))
.filter(models.FloatingIPAllocation.reservation_id.in_(
list(set([x['id'] for x in reservations])))))
allocations = defaultdict(list)
for row in allocations_query.all():
allocations[row[0]].append(row[1])
for r in reservations:
r['floatingip_ids'] = allocations[r['id']]
return reservations
def get_plugin_reservation(resource_type, resource_id):

6
blazar/db/utils.py
View File

@ -120,6 +120,12 @@ def get_reservation_allocations_by_host_ids(host_ids, start_date, end_date,
reservation_id)
def get_reservation_allocations_by_fip_ids(fip_ids, start_date, end_date,
lease_id=None, reservation_id=None):
return IMPL.get_reservation_allocations_by_fip_ids(
fip_ids, start_date, end_date, lease_id, reservation_id)
def get_plugin_reservation(resource_type, resource_id):
return IMPL.get_plugin_reservation(resource_type, resource_id)

18
blazar/enforcement/__init__.py Normal file
View File

@ -0,0 +1,18 @@
# Copyright (c) 2020 University of Chicago.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.
from blazar.enforcement.enforcement import UsageEnforcement
__all__ = ['UsageEnforcement']

99
blazar/enforcement/enforcement.py Normal file
View File

@ -0,0 +1,99 @@
# Copyright (c) 2020 University of Chicago.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.
from blazar.enforcement import filters
from blazar.utils.openstack import base
from oslo_config import cfg
from oslo_log import log as logging
CONF = cfg.CONF
enforcement_opts = [
cfg.ListOpt('enabled_filters',
default=[],
help='List of enabled usage enforcement filters.'),
]
CONF.register_opts(enforcement_opts, group='enforcement')
LOG = logging.getLogger(__name__)
class UsageEnforcement:
def __init__(self):
self.load_filters()
def load_filters(self):
self.enabled_filters = set()
for filter_name in CONF.enforcement.enabled_filters:
_filter = getattr(filters, filter_name)
if filter_name in filters.all_filters:
self.enabled_filters.add(_filter(conf=CONF))
else:
LOG.error("{} not in filters module.".format(filter_name))
self.enabled_filters = list(self.enabled_filters)
def format_context(self, context, lease_values):
ctx = context.to_dict()
region_name = CONF.os_region_name
auth_url = base.url_for(
ctx['service_catalog'], CONF.identity_service,
os_region_name=region_name)
return dict(user_id=lease_values['user_id'],
project_id=lease_values['project_id'],
auth_url=auth_url, region_name=region_name)
def format_lease(self, lease_values, reservations, allocations):
lease = lease_values.copy()
lease['reservations'] = []
for reservation in reservations:
res = reservation.copy()
resource_type = res['resource_type']
res['allocations'] = allocations[resource_type]
lease['reservations'].append(res)
return lease
def check_create(self, context, lease_values, reservations, allocations):
context = self.format_context(context, lease_values)
lease = self.format_lease(lease_values, reservations, allocations)
for _filter in self.enabled_filters:
_filter.check_create(context, lease)
def check_update(self, context, current_lease, new_lease,
current_allocations, new_allocations,
current_reservations, new_reservations):
context = self.format_context(context, current_lease)
current_lease = self.format_lease(current_lease, current_reservations,
current_allocations)
new_lease = self.format_lease(new_lease, new_reservations,
new_allocations)
for _filter in self.enabled_filters:
_filter.check_update(context, current_lease, new_lease)
def on_end(self, context, lease, allocations):
context = self.format_context(context, lease)
lease_values = self.format_lease(lease, lease['reservations'],
allocations)
for _filter in self.enabled_filters:
_filter.on_end(context, lease_values)

23
blazar/enforcement/exceptions.py Normal file
View File

@ -0,0 +1,23 @@
# Copyright (c) 2020 University of Chicago.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.
from blazar import exceptions
from blazar.i18n import _
class MaxLeaseDurationException(exceptions.NotAuthorized):
code = 400
msg_fmt = _('Lease duration of %(lease_duration)s seconds must be less '
'than or equal to the maximum lease duration of '
'%(max_duration)s seconds.')

21
blazar/enforcement/filters/__init__.py Normal file
View File

@ -0,0 +1,21 @@
# Copyright (c) 2013 Bull.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.
from blazar.enforcement.filters.max_lease_duration_filter import (
MaxLeaseDurationFilter)
__all__ = ['MaxLeaseDurationFilter']
all_filters = __all__

45
blazar/enforcement/filters/base_filter.py Normal file
View File

@ -0,0 +1,45 @@
# Copyright (c) 2020 University of Chicago.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import abc
import six
@six.add_metaclass(abc.ABCMeta)
class BaseFilter:
enforcement_opts = []
def __init__(self, conf=None):
self.conf = conf
for opt in self.enforcement_opts:
self.conf.register_opt(opt, 'enforcement')
def __getattr__(self, name):
func = getattr(self.conf.enforcement, name)
return func
@abc.abstractmethod
def check_create(self, context, lease_values):
pass
@abc.abstractmethod
def check_update(self, context, current_lease_values, new_lease_values):
pass
@abc.abstractmethod
def on_end(self, context, lease_values):
pass

80
blazar/enforcement/filters/max_lease_duration_filter.py Normal file
View File

@ -0,0 +1,80 @@
# Copyright (c) 2020 University of Chicago.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.
from blazar.enforcement import exceptions
from blazar.enforcement.filters import base_filter
from oslo_config import cfg
from oslo_log import log as logging
DEFAULT_MAX_LEASE_DURATION = -1
LOG = logging.getLogger(__name__)
class MaxLeaseDurationFilter(base_filter.BaseFilter):
enforcement_opts = [
cfg.IntOpt(
'max_lease_duration',
default=DEFAULT_MAX_LEASE_DURATION,
help='Maximum lease duration in seconds. If this is set to -1, '
'there is not limit.'),
cfg.ListOpt(
'max_lease_duration_exempt_project_ids',
default=[],
help='Allow list of project ids exempt from filter constraints.'),
]
def __init__(self, conf=None):
super(MaxLeaseDurationFilter, self).__init__(conf=conf)
def _exempt(self, context):
return (context['project_id'] in
self.conf.enforcement.max_lease_duration_exempt_project_ids)
def check_for_duration_violation(self, start_date, end_date):
if self.conf.enforcement.max_lease_duration == -1:
return
lease_duration = (end_date - start_date).total_seconds()
if lease_duration > self.conf.enforcement.max_lease_duration:
raise exceptions.MaxLeaseDurationException(
lease_duration=int(lease_duration),
max_duration=self.conf.enforcement.max_lease_duration)
def check_create(self, context, lease_values):
if self._exempt(context):
return
start_date = lease_values['start_date']
end_date = lease_values['end_date']
self.check_for_duration_violation(start_date, end_date)
def check_update(self, context, current_lease_values, new_lease_values):
if self._exempt(context):
return
start_date = new_lease_values['start_date']
end_date = new_lease_values['end_date']
self.check_for_duration_violation(start_date, end_date)
def on_end(self, context, lease_values):
pass

277
blazar/manager/service.py
View File

@ -20,8 +20,10 @@ from oslo_config import cfg
from oslo_log import log as logging
from stevedore import enabled
from blazar import context
from blazar.db import api as db_api
from blazar.db import exceptions as db_ex
from blazar import enforcement
from blazar import exceptions as common_ex
from blazar import manager
from blazar.manager import exceptions
@ -71,6 +73,7 @@ class ManagerService(service_utils.RPCServer):
self.plugins = self._get_plugins()
self.resource_actions = self._setup_actions()
self.monitors = monitor.load_monitors(self.plugins)
self.enforcement = enforcement.UsageEnforcement()
def start(self):
super(ManagerService, self).start()
@ -216,6 +219,45 @@ class ManagerService(service_utils.RPCServer):
date_format=date_format)
return date
def _parse_lease_dates(self, start_date, end_date):
now = datetime.datetime.utcnow()
now = datetime.datetime(now.year,
now.month,
now.day,
now.hour,
now.minute)
if start_date == 'now':
start_date = now
else:
start_date = self._date_from_string(start_date)
if end_date == 'now':
end_date = now
else:
end_date = self._date_from_string(end_date)
return start_date, end_date, now
def _check_for_invalid_date_inputs(self, lease, values, now):
if (lease['start_date'] < now and
values['start_date'] != lease['start_date']):
raise common_ex.InvalidInput(
'Cannot modify the start date of already started leases')
if (lease['start_date'] > now and
values['start_date'] < now):
raise common_ex.InvalidInput(
'Start date must be later than current date')
if lease['end_date'] < now:
raise common_ex.InvalidInput(
'Terminated leases can only be renamed')
if (values['end_date'] < now or
values['end_date'] < values['start_date']):
raise common_ex.InvalidInput(
'End date must be later than current and start date')
def validate_params(self, values, required_params):
if isinstance(required_params, list):
required_params = set(required_params)
@ -250,20 +292,8 @@ class ManagerService(service_utils.RPCServer):
self.validate_params(res, ['resource_type'])
# Create the lease without the reservations
start_date = lease_values['start_date']
end_date = lease_values['end_date']
now = datetime.datetime.utcnow()
now = datetime.datetime(now.year,
now.month,
now.day,
now.hour,
now.minute)
if start_date == 'now':
start_date = now
else:
start_date = self._date_from_string(start_date)
end_date = self._date_from_string(end_date)
start_date, end_date, now = self._parse_lease_dates(
lease_values['start_date'], lease_values['end_date'])
if start_date < now:
raise common_ex.InvalidInput(
@ -281,6 +311,15 @@ class ManagerService(service_utils.RPCServer):
lease_values['start_date'] = start_date
lease_values['end_date'] = end_date
allocations = self._allocation_candidates(
lease_values, reservations)
try:
self.enforcement.check_create(
context.current(), lease_values, reservations, allocations)
except common_ex.NotAuthorized as e:
LOG.error("Enforcement checks failed. %s", str(e))
raise common_ex.NotAuthorized(e)
events.append({'event_type': 'start_lease',
'time': start_date,
'status': status.event.UNDONE})
@ -376,42 +415,13 @@ class ManagerService(service_utils.RPCServer):
datetime.datetime.strftime(lease['end_date'], LEASE_DATE_FORMAT))
before_end_date = values.get('before_end_date', None)
now = datetime.datetime.utcnow()
now = datetime.datetime(now.year,
now.month,
now.day,
now.hour,
now.minute)
if start_date == 'now':
start_date = now
else:
start_date = self._date_from_string(start_date)
if end_date == 'now':
end_date = now
else:
end_date = self._date_from_string(end_date)
start_date, end_date, now = self._parse_lease_dates(start_date,
end_date)
values['start_date'] = start_date
values['end_date'] = end_date
if (lease['start_date'] < now and
values['start_date'] != lease['start_date']):
raise common_ex.InvalidInput(
'Cannot modify the start date of already started leases')
if (lease['start_date'] > now and
values['start_date'] < now):
raise common_ex.InvalidInput(
'Start date must be later than current date')
if lease['end_date'] < now:
raise common_ex.InvalidInput(
'Terminated leases can only be renamed')
if (values['end_date'] < now or
values['end_date'] < values['start_date']):
raise common_ex.InvalidInput(
'End date must be later than current and start date')
self._check_for_invalid_date_inputs(lease, values, now)
with trusts.create_ctx_from_trust(lease['trust_id']):
if before_end_date:
@ -423,12 +433,13 @@ class ManagerService(service_utils.RPCServer):
LOG.error("Invalid before_end_date param. %s", str(e))
raise e
# TODO(frossigneux) rollback if an exception is raised
reservations = values.get('reservations', [])
reservations_db = db_api.reservation_get_all_by_lease_id(lease_id)
existing_reservations = (
db_api.reservation_get_all_by_lease_id(lease_id))
try:
invalid_ids = set([r['id'] for r in reservations]).difference(
[r['id'] for r in reservations_db])
[r['id'] for r in existing_reservations])
except KeyError:
raise exceptions.MissingParameter(param='reservation ID')
@ -437,7 +448,36 @@ class ManagerService(service_utils.RPCServer):
'Please enter valid reservation IDs. Invalid reservation '
'IDs are: %s' % ','.join([str(id) for id in invalid_ids]))
for reservation in (reservations_db):
try:
[
self.plugins[r['resource_type']] for r
in (reservations + existing_reservations)]
except KeyError:
raise exceptions.CantUpdateParameter(param='resource_type')
existing_allocs = self._existing_allocations(existing_reservations)
if reservations:
new_reservations = reservations
new_allocs = self._allocation_candidates(values,
existing_reservations)
else:
# User is not updating reservation parameters, e.g., is only
# adjusting lease start/end dates.
new_reservations = existing_reservations
new_allocs = existing_allocs
try:
self.enforcement.check_update(context.current(), lease, values,
existing_allocs, new_allocs,
existing_reservations,
new_reservations)
except common_ex.NotAuthorized as e:
LOG.error("Enforcement checks failed. %s", str(e))
raise common_ex.NotAuthorized(e)
# TODO(frossigneux) rollback if an exception is raised
for reservation in existing_reservations:
v = {}
v['start_date'] = values['start_date']
v['end_date'] = values['end_date']
@ -448,6 +488,7 @@ class ManagerService(service_utils.RPCServer):
pass
resource_type = v.get('resource_type',
reservation['resource_type'])
if resource_type != reservation['resource_type']:
raise exceptions.CantUpdateParameter(
param='resource_type')
@ -500,36 +541,58 @@ class ManagerService(service_utils.RPCServer):
result_in=(status.lease.ERROR,))
def delete_lease(self, lease_id):
lease = self.get_lease(lease_id)
if (datetime.datetime.utcnow() >= lease['start_date'] and
datetime.datetime.utcnow() <= lease['end_date']):
start_event = db_api.event_get_first_sorted_by_filters(
'lease_id',
'asc',
{
'lease_id': lease_id,
'event_type': 'start_lease'
}
)
if not start_event:
raise common_ex.BlazarException(
'start_lease event for lease %s not found' % lease_id)
end_event = db_api.event_get_first_sorted_by_filters(
'lease_id',
'asc',
{
'lease_id': lease_id,
'event_type': 'end_lease',
'status': status.event.UNDONE
}
)
if not end_event:
raise common_ex.BlazarException(
'end_lease event for lease %s not found' % lease_id)
start_event = db_api.event_get_first_sorted_by_filters(
'lease_id',
'asc',
{
'lease_id': lease_id,
'event_type': 'start_lease',
}
)
if not start_event:
raise common_ex.BlazarException(
'start_lease event for lease %s not found' % lease_id)
end_event = db_api.event_get_first_sorted_by_filters(
'lease_id',
'asc',
{
'lease_id': lease_id,
'event_type': 'end_lease',
}
)
if not end_event:
raise common_ex.BlazarException(
'end_lease event for lease %s not found' % lease_id)
lease_already_started = start_event['status'] != status.event.UNDONE
lease_not_started = not lease_already_started
lease_already_ended = end_event['status'] != status.event.UNDONE
lease_not_ended = not lease_already_ended
end_lease = lease_already_started and lease_not_ended
if end_lease:
db_api.event_update(end_event['id'],
{'status': status.event.IN_PROGRESS})
with trusts.create_ctx_from_trust(lease['trust_id']) as ctx:
for reservation in lease['reservations']:
reservations = lease['reservations']
if lease_not_started or lease_not_ended:
# Only run the on_end enforcement if we're explicitly
# ending the lease for the first time OR if we're terminating
# it before the lease ever started. It's important to run
# on_end in the second case to inform enforcement that the
# lease is no longer in play.
allocations = self._existing_allocations(reservations)
try:
self.enforcement.on_end(ctx, lease, allocations)
except Exception as e:
LOG.error(e)
for reservation in reservations:
if reservation['status'] != status.reservation.DELETED:
plugin = self.plugins[reservation['resource_type']]
try:
@ -555,7 +618,14 @@ class ManagerService(service_utils.RPCServer):
result_in=(status.lease.TERMINATED, status.lease.ERROR))
def end_lease(self, lease_id, event_id):
lease = self.get_lease(lease_id)
with trusts.create_ctx_from_trust(lease['trust_id']):
with trusts.create_ctx_from_trust(lease['trust_id']) as ctx:
allocations = self._existing_allocations(lease['reservations'])
try:
self.enforcement.on_end(ctx, lease, allocations)
except Exception as e:
LOG.error(e)
self._basic_action(lease_id, event_id, 'on_end',
status.reservation.DELETED)
@ -616,6 +686,59 @@ class ManagerService(service_utils.RPCServer):
db_api.reservation_update(reservation['id'],
{'resource_id': resource_id})
def _allocation_candidates(self, lease, reservations):
"""Returns dict by resource type of reservation candidates."""
allocations = {}
for reservation in reservations:
res = reservation.copy()
resource_type = reservation['resource_type']
res['start_date'] = lease['start_date']
res['end_date'] = lease['end_date']
if resource_type not in self.plugins:
raise exceptions.UnsupportedResourceType(
resource_type=resource_type)
plugin = self.plugins.get(resource_type)
if not plugin:
raise common_ex.BlazarException(
'Invalid plugin names are specified: %s' % resource_type)
candidate_ids = plugin.allocation_candidates(res)
allocations[resource_type] = [
plugin.get(cid) for cid in candidate_ids]
return allocations
def _existing_allocations(self, reservations):
allocations = {}
for reservation in reservations:
resource_type = reservation['resource_type']
if resource_type not in self.plugins:
raise exceptions.UnsupportedResourceType(
resource_type=resource_type)
plugin = self.plugins.get(resource_type)
if not plugin:
raise common_ex.BlazarException(
'Invalid plugin names are specified: %s' % resource_type)
resource_ids = [
x['resource_id'] for x in plugin.list_allocations(
dict(reservation_id=reservation['id']))
if x['reservations']]
allocations[resource_type] = [
plugin.get(rid) for rid in resource_ids]
return allocations
def _send_notification(self, lease, ctx, events=[]):
payload = notification_api.format_lease_payload(lease)

3
blazar/opts.py
View File

@ -44,6 +44,9 @@ def list_opts():
('api', blazar.api.v2.controllers.api_opts),
('manager', itertools.chain(blazar.manager.opts,
blazar.manager.service.manager_opts)),
('enforcement', itertools.chain(
blazar.enforcement.filters.max_lease_duration_filter.MaxLeaseDurationFilter.enforcement_opts, # noqa
blazar.enforcement.enforcement.enforcement_opts)),
('notifications', blazar.notification.notifier.notification_opts),
('nova', blazar.utils.openstack.nova.nova_opts),
(blazar.plugins.oshosts.RESOURCE_TYPE,

21
blazar/plugins/base.py
View File

@ -57,11 +57,32 @@ class BasePlugin(object, metaclass=abc.ABCMeta):
'description': self.description,
}
@abc.abstractmethod
def get(self, resource_id):
"""Get resource by id"""
pass
@abc.abstractmethod
def reserve_resource(self, reservation_id, values):
"""Reserve resource."""
pass
@abc.abstractmethod
def list_allocations(self, query, detail=False):
"""List resource allocations."""
pass
@abc.abstractmethod
def query_allocations(self, resource_id_list, lease_id=None,
reservation_id=None):
"""List resource allocations."""
pass
@abc.abstractmethod
def allocation_candidates(self, lease_values):
"""Get candidates for reservation allocation."""
pass
@abc.abstractmethod
def update_reservation(self, reservation_id, values):
"""Update reservation."""

14
blazar/plugins/dummy_vm_plugin.py
View File

@ -22,9 +22,23 @@ class DummyVMPlugin(base.BasePlugin):
title = 'Dummy VM Plugin'
description = 'This plugin does nothing.'
def get(self, resource_id):
return None
def reserve_resource(self, reservation_id, values):
return None
def list_allocations(self, query, detail=False):
"""List resource allocations."""
pass
def query_allocations(self, resource_id_list, lease_id=None,
reservation_id=None):
return None
def allocation_candidates(self, lease_values):
return None
def update_reservation(self, reservation_id, values):
return None

71
blazar/plugins/floatingips/floatingip_plugin.py
View File

@ -35,6 +35,8 @@ from blazar.utils import plugins as plugins_utils
CONF = cfg.CONF
LOG = logging.getLogger(__name__)
QUERY_TYPE_ALLOCATION = 'allocation'
class FloatingIpPlugin(base.BasePlugin):
"""Plugin for floating IP resource."""
@ -42,6 +44,9 @@ class FloatingIpPlugin(base.BasePlugin):
resource_type = plugin.RESOURCE_TYPE
title = 'Floating IP Plugin'
description = 'This plugin creates and assigns floating IPs.'
query_options = {
QUERY_TYPE_ALLOCATION: ['lease_id', 'reservation_id']
}
def check_params(self, values):
if 'network_id' not in values:
@ -262,6 +267,19 @@ class FloatingIpPlugin(base.BasePlugin):
fip = db_api.floatingip_get(alloc['floatingip_id'])
fip_pool.delete_reserved_floatingip(fip['floating_ip_address'])
def allocation_candidates(self, values):
self.check_params(values)
required_fips = values.get('required_floatingips', [])
amount = int(values['amount'])
if len(required_fips) > amount:
raise manager_ex.TooLongFloatingIPs()
return self._matching_fips(values['network_id'], required_fips,
amount, values['start_date'],
values['end_date'])
def _matching_fips(self, network_id, fip_addresses, amount,
start_date, end_date):
filter_array = []
@ -344,6 +362,9 @@ class FloatingIpPlugin(base.BasePlugin):
return floatingip
def get(self, fip_id):
return self.get_floatingip(fip_id)
def get_floatingip(self, fip_id):
fip = db_api.floatingip_get(fip_id)
if fip is None:
@ -370,3 +391,53 @@ class FloatingIpPlugin(base.BasePlugin):
except db_ex.BlazarDBException as e:
raise manager_ex.CantDeleteFloatingIP(floatingip=fip_id,
msg=str(e))
def list_allocations(self, query, detail=False):
fip_id_list = [f['id'] for f in db_api.floatingip_list()]
options = self.get_query_options(query, QUERY_TYPE_ALLOCATION)
options['detail'] = detail
fip_allocations = self.query_allocations(fip_id_list, **options)
return [{"resource_id": fip, "reservations": allocs}
for fip, allocs in fip_allocations.items()]
def query_allocations(self, resource_id_list, detail=None, lease_id=None,
reservation_id=None):
return self.query_fip_allocations(resource_id_list, detail=detail,
lease_id=lease_id,
reservation_id=reservation_id)
def query_fip_allocations(self, fips, detail=None, lease_id=None,
reservation_id=None):
"""Return dict of host and its allocations.
The list element forms
{
'-id': [
{
'lease_id': lease_id,
'id': reservation_id
},
]
}.
"""
start = datetime.datetime.utcnow()
end = datetime.date.max
reservations = db_utils.get_reservation_allocations_by_fip_ids(
fips, start, end, lease_id, reservation_id)
fip_allocations = {fip: [] for fip in fips}
for reservation in reservations:
if not detail:
del reservation['project_id']
del reservation['lease_name']
del reservation['status']
for fip_id in reservation['floatingip_ids']:
if fip_id in fip_allocations.keys():
fip_allocations[fip_id].append({
k: v for k, v in reservation.items()
if k != 'floatingip_ids'})
return fip_allocations

60
blazar/plugins/instances/instance_plugin.py
View File

@ -43,6 +43,7 @@ FLAVOR_EXTRA_SPEC = "aggregate_instance_extra_specs:" + RESERVATION_PREFIX
INSTANCE_DELETION_TIMEOUT = 10 * 60 * 1000 # 10 minutes
NONE_VALUES = ('None', 'none', None)
QUERY_TYPE_ALLOCATION = 'allocation'
class VirtualInstancePlugin(base.BasePlugin, nova.NovaClientWrapper):
@ -50,6 +51,9 @@ class VirtualInstancePlugin(base.BasePlugin, nova.NovaClientWrapper):
resource_type = plugin.RESOURCE_TYPE
title = 'Virtual Instance Plugin'
query_options = {
QUERY_TYPE_ALLOCATION: ['lease_id', 'reservation_id']
}
def __init__(self):
super(VirtualInstancePlugin, self).__init__(
@ -137,6 +141,43 @@ class VirtualInstancePlugin(base.BasePlugin, nova.NovaClientWrapper):
used_disk += disk
return hosts_list
def allocation_candidates(self, reservation):
return self.pickup_hosts(None, reservation)['added']
def list_allocations(self, query):
hosts_id_list = [h['id'] for h in db_api.host_list()]
options = self.get_query_options(query, QUERY_TYPE_ALLOCATION)
hosts_allocations = self.query_allocations(hosts_id_list, **options)
return [{"resource_id": host, "reservations": allocs}
for host, allocs in hosts_allocations.items()]
def query_allocations(self, hosts, lease_id=None, reservation_id=None):
"""Return dict of host and its allocations.
The list element forms
{
'host-id': [
{
'lease_id': lease_id,
'id': reservation_id
},
]
}.
"""
start = datetime.datetime.utcnow()
end = datetime.date.max
# To reduce overhead, this method only executes one query
# to get the allocation information
rsv_lease_host = db_utils.get_reservation_allocations_by_host_ids(
hosts, start, end, lease_id, reservation_id)
hosts_allocs = collections.defaultdict(list)
for rsv, lease, host in rsv_lease_host:
hosts_allocs[host].append({'lease_id': lease, 'id': rsv})
return hosts_allocs
def query_available_hosts(self, cpus=None, memory=None, disk=None,
resource_properties=None,
start_date=None, end_date=None,
@ -757,3 +798,22 @@ class VirtualInstancePlugin(base.BasePlugin, nova.NovaClientWrapper):
additional=True)
LOG.warn('Resource changed for reservation %s (lease: %s).',
reservation['id'], lease['name'])
def _get_extra_capabilities(self, host_id):
extra_capabilities = {}
raw_extra_capabilities = (
db_api.host_extra_capability_get_all_per_host(host_id))
for capability in raw_extra_capabilities:
key = capability['capability_name']
extra_capabilities[key] = capability['capability_value']
return extra_capabilities
def get(self, host_id):
host = db_api.host_get(host_id)
extra_capabilities = self._get_extra_capabilities(host_id)
if host is not None and extra_capabilities:
res = host.copy()
res.update(extra_capabilities)
return res
else:
return host

31
blazar/plugins/oshosts/host_plugin.py
View File

@ -100,17 +100,11 @@ class PhysicalHostPlugin(base.BasePlugin, nova.NovaClientWrapper):
def reserve_resource(self, reservation_id, values):
"""Create reservation."""
self._check_params(values)
host_ids = self.allocation_candidates(values)
host_ids = self._matching_hosts(
values['hypervisor_properties'],
values['resource_properties'],
values['count_range'],
values['start_date'],
values['end_date'],
)
if not host_ids:
raise manager_ex.NotEnoughHostsAvailable()
pool = nova.ReservationPool()
pool_name = reservation_id
az_name = "%s%s" % (CONF[self.resource_type].blazar_az_prefix,
@ -314,6 +308,9 @@ class PhysicalHostPlugin(base.BasePlugin, nova.NovaClientWrapper):
extra_capabilities[key] = capability['capability_value']
return extra_capabilities
def get(self, host_id):
return self.get_computehost(host_id)
def get_computehost(self, host_id):
host = db_api.host_get(host_id)
extra_capabilities = self._get_extra_capabilities(host_id)
@ -510,21 +507,19 @@ class PhysicalHostPlugin(base.BasePlugin, nova.NovaClientWrapper):
hosts_id_list = [h['id'] for h in db_api.host_list()]
options = self.get_query_options(query, QUERY_TYPE_ALLOCATION)
hosts_allocations = self.query_host_allocations(hosts_id_list,
**options)
hosts_allocations = self.query_allocations(hosts_id_list, **options)
return [{"resource_id": host, "reservations": allocs}
for host, allocs in hosts_allocations.items()]
def get_allocations(self, host_id, query):
options = self.get_query_options(query, QUERY_TYPE_ALLOCATION)
host_allocations = self.query_host_allocations([host_id], **options)
host_allocations = self.query_allocations([host_id], **options)
if host_id not in host_allocations:
host_allocations = {host_id: []}
allocs = host_allocations[host_id]
return {"resource_id": host_id, "reservations": allocs}
def query_host_allocations(self, hosts, lease_id=None,
reservation_id=None):
def query_allocations(self, hosts, lease_id=None, reservation_id=None):
"""Return dict of host and its allocations.
The list element forms
@ -550,6 +545,16 @@ class PhysicalHostPlugin(base.BasePlugin, nova.NovaClientWrapper):
hosts_allocs[host].append({'lease_id': lease, 'id': rsv})
return hosts_allocs
def allocation_candidates(self, values):
self._check_params(values)
return self._matching_hosts(
values['hypervisor_properties'],
values['resource_properties'],
values['count_range'],
values['start_date'],
values['end_date'])
def _matching_hosts(self, hypervisor_properties, resource_properties,
count_range, start_date, end_date):
"""Return the matching hosts (preferably not allocated)

0
blazar/tests/enforcement/__init__.py Normal file
View File

0
blazar/tests/enforcement/filters/__init__.py Normal file
View File

250
blazar/tests/enforcement/filters/test_max_lease_duration_filter.py Executable file
View File

@ -0,0 +1,250 @@
# Copyright (c) 2021 StackHPC.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import datetime
from unittest import mock
import ddt
from blazar import context
from blazar import enforcement
from blazar.enforcement import exceptions
from blazar.enforcement import filters
from blazar import tests
from oslo_config import cfg
def get_fake_host(host_id):
return {
'id': host_id,
'hypervisor_hostname': 'hypvsr1',
'service_name': 'compute1',
'vcpus': 4,
'cpu_info': 'foo',
'hypervisor_type': 'xen',
'hypervisor_version': 1,
'memory_mb': 8192,
'local_gb': 10,
}
def get_fake_lease(**kwargs):
fake_lease = {
'id': '1',
'name': 'lease_test',
'start_date': datetime.datetime(2014, 1, 1, 1, 23),
'end_date': datetime.datetime(2014, 1, 1, 2, 23),
'user_id': '111',
'project_id': '222',
'trust_id': '35b17138b3644e6aa1318f3099c5be68',
'reservations': [{'resource_id': '1234',
'resource_type': 'virtual:instance'}],
'events': [],
'before_end_date': datetime.datetime(2014, 1, 1, 1, 53),
'action': None,
'status': None,
'status_reason': None}
if kwargs:
fake_lease.update(kwargs)
return fake_lease
@ddt.ddt
class MaxLeaseDurationTestCase(tests.TestCase):
def setUp(self):
super(MaxLeaseDurationTestCase, self).setUp()
self.cfg = cfg
self.region = 'RegionOne'
filters.all_filters = ['MaxLeaseDurationFilter']
self.enforcement = enforcement.UsageEnforcement()
cfg.CONF.set_override(
'enabled_filters', filters.all_filters, group='enforcement')
cfg.CONF.set_override('os_region_name', self.region)
self.enforcement.load_filters()
cfg.CONF.set_override('max_lease_duration', 3600, group='enforcement')
self.fake_service_catalog = [
dict(
type='identity', endpoints=[
dict(
interface='public', region=self.region,
url='https://fakeauth.com')
]
)
]
self.ctx = context.BlazarContext(
user_id='111', project_id='222',
service_catalog=self.fake_service_catalog)
self.set_context(self.ctx)
self.fake_host_id = '1'
self.fake_host = {
'id': self.fake_host_id,
'hypervisor_hostname': 'hypvsr1',
'service_name': 'compute1',
'vcpus': 4,
'cpu_info': 'foo',
'hypervisor_type': 'xen',
'hypervisor_version': 1,
'memory_mb': 8192,
'local_gb': 10,
}
self.addCleanup(self.cfg.CONF.clear_override, 'enabled_filters',
group='enforcement')
self.addCleanup(self.cfg.CONF.clear_override, 'max_lease_duration',
group='enforcement')
self.addCleanup(self.cfg.CONF.clear_override,
'max_lease_duration_exempt_project_ids',
group='enforcement')
self.addCleanup(self.cfg.CONF.clear_override, 'os_region_name')
def tearDown(self):
super(MaxLeaseDurationTestCase, self).tearDown()
def test_check_create_allowed_with_max_lease_duration(self):
allocation_candidates = {'virtual:instance': [get_fake_host('1')]}
lease_values = get_fake_lease()
reservations = list(lease_values['reservations'])
del lease_values['reservations']
ctx = context.current()
self.enforcement.check_create(ctx, lease_values, reservations,
allocation_candidates)
def test_check_create_denied_beyond_max_lease_duration(self):
allocation_candidates = {'virtual:instance': [get_fake_host('1')]}
lease_values = get_fake_lease(
end_date=datetime.datetime(2014, 1, 1, 2, 24))
reservations = list(lease_values['reservations'])
del lease_values['reservations']
ctx = context.current()
self.assertRaises(exceptions.MaxLeaseDurationException,
self.enforcement.check_create, ctx, lease_values,
reservations, allocation_candidates)
def test_check_update_allowed(self):
current_allocations = {'virtual:instance': [get_fake_host('1')]}
lease = get_fake_lease(end_date=datetime.datetime(2014, 1, 1, 2, 22))
reservations = list(lease['reservations'])
new_lease_values = get_fake_lease(
end_date=datetime.datetime(2014, 1, 1, 2, 23))
new_reservations = list(new_lease_values['reservations'])
allocation_candidates = {'virtual:instance': [get_fake_host('2')]}
del new_lease_values['reservations']
ctx = context.current()
with mock.patch.object(datetime, 'datetime',
mock.Mock(wraps=datetime.datetime)) as patched:
patched.utcnow.return_value = datetime.datetime(2014, 1, 1, 1, 1)
self.enforcement.check_update(
ctx, lease, new_lease_values, current_allocations,
allocation_candidates, reservations, new_reservations)
def test_check_update_denied(self):
current_allocations = {'virtual:instance': [get_fake_host('1')]}
lease = get_fake_lease(end_date=datetime.datetime(2014, 1, 1, 2, 22))
reservations = list(lease['reservations'])
new_lease_values = get_fake_lease(
end_date=datetime.datetime(2014, 1, 1, 2, 24))
new_reservations = list(new_lease_values['reservations'])
allocation_candidates = {'virtual:instance': [get_fake_host('2')]}
del new_lease_values['reservations']
ctx = context.current()
with mock.patch.object(datetime, 'datetime',
mock.Mock(wraps=datetime.datetime)) as patched:
patched.utcnow.return_value = datetime.datetime(2014, 1, 1, 1, 1)
self.assertRaises(exceptions.MaxLeaseDurationException,
self.enforcement.check_update, ctx, lease,
new_lease_values, current_allocations,
allocation_candidates, reservations,
new_reservations)
def test_check_update_active_lease_allowed(self):
current_allocations = {'virtual:instance': [get_fake_host('1')]}
lease = get_fake_lease(end_date=datetime.datetime(2014, 1, 1, 1, 53))
reservations = list(lease['reservations'])
new_lease_values = get_fake_lease()
new_reservations = list(new_lease_values['reservations'])
allocation_candidates = {'virtual:instance': [get_fake_host('2')]}
del new_lease_values['reservations']
ctx = context.current()
with mock.patch.object(datetime, 'datetime',
mock.Mock(wraps=datetime.datetime)) as patched:
patched.utcnow.return_value = datetime.datetime(2014, 1, 1, 1, 50)
self.enforcement.check_update(
ctx, lease, new_lease_values, current_allocations,
allocation_candidates, reservations, new_reservations)
def test_check_create_exempt(self):
cfg.CONF.set_override('max_lease_duration_exempt_project_ids', ['222'],
group='enforcement')
allocation_candidates = {'virtual:instance': [get_fake_host('1')]}
lease_values = get_fake_lease(
end_date=datetime.datetime(2014, 1, 1, 2, 24))
reservations = list(lease_values['reservations'])
del lease_values['reservations']
ctx = context.current()
self.enforcement.check_create(ctx, lease_values, reservations,
allocation_candidates)
def test_check_update_exempt(self):
cfg.CONF.set_override('max_lease_duration_exempt_project_ids', ['222'],
group='enforcement')
current_allocations = {'virtual:instance': [get_fake_host('1')]}
lease = get_fake_lease(end_date=datetime.datetime(2014, 1, 1, 2, 22))
reservations = list(lease['reservations'])
new_lease_values = get_fake_lease(
end_date=datetime.datetime(2014, 1, 1, 2, 24))
new_reservations = list(new_lease_values['reservations'])
allocation_candidates = {'virtual:instance': [get_fake_host('2')]}
del new_lease_values['reservations']
ctx = context.current()
with mock.patch.object(datetime, 'datetime',
mock.Mock(wraps=datetime.datetime)) as patched:
patched.utcnow.return_value = datetime.datetime(2014, 1, 1, 1, 1)
self.enforcement.check_update(
ctx, lease, new_lease_values, current_allocations,
allocation_candidates, reservations, new_reservations)
def test_on_end(self):
allocations = {'virtual:instance': [get_fake_host('1')]}
lease = get_fake_lease()
ctx = context.current()
self.enforcement.on_end(ctx, lease, allocations)

306
blazar/tests/enforcement/test_enforcement.py Executable file
View File

@ -0,0 +1,306 @@
# Copyright (c) 2020 University of Chicago.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import datetime
import ddt
from blazar import context
from blazar import enforcement
from blazar.enforcement import filters
from blazar import exceptions
from blazar.manager import service
from blazar import tests
from oslo_config import cfg
def get_fake_host(host_id):
return {
'id': host_id,
'hypervisor_hostname': 'hypvsr1',
'service_name': 'compute1',
'vcpus': 4,
'cpu_info': 'foo',
'hypervisor_type': 'xen',
'hypervisor_version': 1,
'memory_mb': 8192,
'local_gb': 10,
}
def get_fake_lease(**kwargs):
fake_lease = {
'id': '1',
'name': 'lease_test',
'start_date': datetime.datetime.utcnow().strftime(
service.LEASE_DATE_FORMAT),
'end_date': (
datetime.datetime.utcnow() + datetime.timedelta(days=1)).strftime(
service.LEASE_DATE_FORMAT),
'user_id': '111',
'project_id': '222',
'reservations': [{'resource_id': '1234',
'resource_type': 'virtual:instance'}],
'events': [],
'before_end_date': '2014-02-01 10:37',
'action': None,
'status': None,
'status_reason': None,
'trust_id': 'exxee111qwwwwe'}
if kwargs:
fake_lease.update(kwargs)
return fake_lease
def get_lease_rsv_allocs():
allocation_candidates = {'virtual:instance': [get_fake_host('1')]}
lease_values = get_fake_lease()
reservations = list(lease_values['reservations'])
del lease_values['reservations']
return lease_values, reservations, allocation_candidates
class FakeFilter(filters.base_filter.BaseFilter):
enforcement_opts = [
cfg.IntOpt('fake_opt', default=1, help='This is a fake config.'),
]
def __init__(self, conf=None):
super(FakeFilter, self).__init__(conf=conf)
def check_create(self, context, lease_values):
pass
def check_update(self, context, current_lease_values, new_lease_values):
pass
def on_end(self, context, lease_values):
pass
@ddt.ddt
class EnforcementTestCase(tests.TestCase):
def setUp(self):
super(EnforcementTestCase, self).setUp()
self.cfg = cfg
self.region = 'RegionOne'
filters.FakeFilter = FakeFilter
filters.all_filters = ['FakeFilter']
self.enforcement = enforcement.UsageEnforcement()
cfg.CONF.set_override(
'enabled_filters', filters.all_filters, group='enforcement')
cfg.CONF.set_override('os_region_name', self.region)
self.enforcement.load_filters()
self.fake_service_catalog = [
dict(
type='identity', endpoints=[
dict(
interface='public', region=self.region,
url='https://fakeauth.com')
]
)
]
self.ctx = context.BlazarContext(
user_id='111', project_id='222',
service_catalog=self.fake_service_catalog)
self.set_context(self.ctx)
self.fake_host_id = '1'
self.fake_host = {
'id': self.fake_host_id,
'hypervisor_hostname': 'hypvsr1',
'service_name': 'compute1',
'vcpus': 4,
'cpu_info': 'foo',
'hypervisor_type': 'xen',
'hypervisor_version': 1,
'memory_mb': 8192,
'local_gb': 10,
}
self.addCleanup(self.cfg.CONF.clear_override, 'enabled_filters',
group='enforcement')
self.addCleanup(self.cfg.CONF.clear_override, 'os_region_name')
def tearDown(self):
super(EnforcementTestCase, self).tearDown()
def get_formatted_lease(self, lease_values, rsv, allocs):
expected_lease = lease_values.copy()
if rsv:
expected_lease['reservations'] = rsv
for res in expected_lease['reservations']:
res['allocations'] = allocs[res['resource_type']]
return expected_lease
def test_load_filters(self):
self.assertEqual(len(self.enforcement.enabled_filters), 1)
fake_filter = self.enforcement.enabled_filters.pop()
self.assertIsInstance(fake_filter, FakeFilter)
self.assertEqual(fake_filter.conf.enforcement.fake_opt, 1)
def test_format_context(self):
formatted_context = self.enforcement.format_context(
context.current(), get_fake_lease())
expected = dict(user_id='111', project_id='222',
region_name=self.region,
auth_url='https://fakeauth.com')
self.assertDictEqual(expected, formatted_context)
def test_format_lease(self):
lease_values, rsv, allocs = get_lease_rsv_allocs()
formatted_lease = self.enforcement.format_lease(lease_values, rsv,
allocs)
expected_lease = self.get_formatted_lease(lease_values, rsv, allocs)
self.assertDictEqual(expected_lease, formatted_lease)
def test_check_create(self):
lease_values, rsv, allocs = get_lease_rsv_allocs()
ctx = context.current()
check_create = self.patch(self.enforcement.enabled_filters[0],
'check_create')
self.enforcement.check_create(ctx, lease_values, rsv, allocs)
formatted_lease = self.enforcement.format_lease(lease_values, rsv,
allocs)
formatted_context = self.enforcement.format_context(ctx, lease_values)
check_create.assert_called_once_with(formatted_context,
formatted_lease)
expected_context = dict(user_id='111', project_id='222',
region_name=self.region,
auth_url='https://fakeauth.com')
expected_lease = self.get_formatted_lease(lease_values, rsv, allocs)
self.assertDictEqual(expected_context, formatted_context)
self.assertDictEqual(expected_lease, formatted_lease)
def test_check_create_with_exception(self):
lease_values, rsv, allocs = get_lease_rsv_allocs()
ctx = context.current()
check_create = self.patch(self.enforcement.enabled_filters[0],
'check_create')
check_create.side_effect = exceptions.BlazarException
self.assertRaises(exceptions.BlazarException,
self.enforcement.check_create,
context=ctx, lease_values=lease_values,
reservations=rsv, allocations=allocs)
def test_check_update(self):
lease, rsv, allocs = get_lease_rsv_allocs()
new_lease_values = get_fake_lease(end_date='2014-02-07 13:37')
new_reservations = list(new_lease_values['reservations'])
allocation_candidates = {'virtual:instance': [get_fake_host('2')]}
del new_lease_values['reservations']
ctx = context.current()
check_update = self.patch(self.enforcement.enabled_filters[0],
'check_update')
self.enforcement.check_update(
ctx, lease, new_lease_values, allocs, allocation_candidates,
rsv, new_reservations)
formatted_context = self.enforcement.format_context(ctx, lease)
formatted_lease = self.enforcement.format_lease(lease, rsv, allocs)
new_formatted_lease = self.enforcement.format_lease(
new_lease_values, new_reservations, allocation_candidates)
expected_context = dict(user_id='111', project_id='222',
region_name=self.region,
auth_url='https://fakeauth.com')
expected_lease = self.get_formatted_lease(lease, rsv, allocs)
expected_new_lease = self.get_formatted_lease(
new_lease_values, new_reservations, allocation_candidates)
check_update.assert_called_once_with(
formatted_context, formatted_lease, new_formatted_lease)
self.assertDictEqual(expected_context, formatted_context)
self.assertDictEqual(expected_lease, formatted_lease)
self.assertDictEqual(expected_new_lease, new_formatted_lease)
def test_check_update_with_exception(self):
lease, rsv, allocs = get_lease_rsv_allocs()
new_lease_values = get_fake_lease(end_date='2014-02-07 13:37')
new_reservations = list(new_lease_values['reservations'])
allocation_candidates = {'virtual:instance': [get_fake_host('2')]}
del new_lease_values['reservations']
ctx = context.current()
check_update = self.patch(self.enforcement.enabled_filters[0],
'check_update')
check_update.side_effect = exceptions.BlazarException
self.assertRaises(
exceptions.BlazarException, self.enforcement.check_update,
context=ctx, current_lease=lease, new_lease=new_lease_values,
current_allocations=allocs, new_allocations=allocation_candidates,
current_reservations=rsv, new_reservations=new_reservations)
def test_on_end(self):
allocations = {'virtual:instance': [get_fake_host('1')]}
lease = get_fake_lease()
ctx = context.current()
on_end = self.patch(self.enforcement.enabled_filters[0], 'on_end')
self.enforcement.on_end(ctx, lease, allocations)
formatted_context = self.enforcement.format_context(ctx, lease)
formatted_lease = self.enforcement.format_lease(
lease, lease['reservations'], allocations)
on_end.assert_called_once_with(formatted_context, formatted_lease)
expected_context = dict(user_id='111', project_id='222',
region_name=self.region,
auth_url='https://fakeauth.com')
expected_lease = self.get_formatted_lease(lease, None, allocations)
self.assertDictEqual(expected_context, formatted_context)
self.assertDictEqual(expected_lease, formatted_lease)

384
blazar/tests/manager/test_service.py
View File

@ -27,6 +27,8 @@ import testtools
from blazar import context
from blazar.db import api as db_api
from blazar.db import exceptions as db_ex
from blazar import enforcement
from blazar.enforcement import exceptions as enforcement_ex
from blazar import exceptions
from blazar.manager import exceptions as manager_ex
from blazar.manager import service
@ -51,9 +53,22 @@ class FakePlugin(base.BasePlugin):
title = 'Fake Plugin'
description = 'This plugin is fake.'
def get(self, resource_id):
return None
def reserve_resource(self, reservation_id, values):
return None
def query_allocations(self, resource_id_list, lease_id=None,
reservation_id=None):
return None
def allocation_candidates(self, lease_values):
return None
def list_allocations(self, query, defail=False):
return None
def update_reservation(self, reservation_id, values):
return None
@ -127,12 +142,16 @@ class ServiceTestCase(tests.TestCase):
'send_lease_notification')
cfg.CONF.set_override('plugins', ['dummy.vm.plugin'], group='manager')
cfg.CONF.set_override(
'enabled_filters', ['MaxLeaseDurationFilter'],
group='enforcement')
with mock.patch('blazar.status.lease.lease_status',
FakeLeaseStatus.lease_status):
importlib.reload(service)
self.service = service
self.manager = self.service.ManagerService()
self.enforcement = self.patch(self.manager, 'enforcement')
self.lease_id = '11-22-33'
self.user_id = '123'
@ -159,12 +178,27 @@ class ServiceTestCase(tests.TestCase):
'start_date': datetime.datetime(2013, 12, 20, 13, 00),
'end_date': datetime.datetime(2013, 12, 20, 15, 00),
'trust_id': 'exxee111qwwwwe'}
self.lease_values = {
'id': self.lease_id,
'user_id': self.user_id,
'project_id': self.project_id,
'name': 'lease-name',
'reservations': [{'id': '111',
'resource_id': '111',
'resource_type': 'virtual:instance',
'status': 'FAKE PROGRESS'}],
'start_date': '2026-11-13 13:13',
'end_date': '2026-12-13 13:13',
'trust_id': 'exxee111qwwwwe'}
self.good_date = datetime.datetime.strptime('2012-12-13 13:13',
'%Y-%m-%d %H:%M')
self.ctx = self.patch(self.context, 'BlazarContext')
self.ctx_current = self.patch(context, 'current')
self.trust_ctx = self.patch(self.trusts, 'create_ctx_from_trust')
self.trust_create = self.patch(self.trusts, 'create_trust')
self.patch(enforcement.UsageEnforcement, 'format_context')
self.lease_get = self.patch(self.db_api, 'lease_get')
self.lease_get.return_value = self.lease
self.lease_list = self.patch(self.db_api, 'lease_list')
@ -387,61 +421,33 @@ class ServiceTestCase(tests.TestCase):
self.lease_list.assert_called_once_with()
def test_create_lease_now(self):
trust_id = 'exxee111qwwwwe'
lease_values = {
'id': self.lease_id,
'name': 'lease-name',
'reservations': [{'id': '111',
'resource_id': '111',
'resource_type': 'virtual:instance',
'status': 'FAKE PROGRESS'}],
'start_date': 'now',
'end_date': '2026-12-13 13:13',
'trust_id': trust_id}
lease_values = self.lease_values
lease = self.manager.create_lease(lease_values)
self.trust_ctx.assert_called_once_with(trust_id)
self.enforcement.check_create.assert_called_once()
self.trust_ctx.assert_called_once_with(lease_values['trust_id'])
self.lease_create.assert_called_once_with(lease_values)
self.assertEqual(lease, self.lease)
expected_context = self.trust_ctx.return_value
self.fake_notifier.assert_called_once_with(
expected_context.__enter__.return_value,
notifier_api.format_lease_payload(lease),
'lease.create')
def test_create_lease_some_time(self):
trust_id = 'exxee111qwwwwe'
lease_values = {
'id': self.lease_id,
'name': 'lease-name',
'reservations': [{'id': '111',
'resource_id': '111',
'resource_type': 'virtual:instance',
'status': 'FAKE PROGRESS'}],
'start_date': '2026-11-13 13:13',
'end_date': '2026-12-13 13:13',
'trust_id': trust_id}
lease_values = self.lease_values.copy()
self.lease['start_date'] = '2026-11-13 13:13'
lease = self.manager.create_lease(lease_values)
self.trust_ctx.assert_called_once_with(trust_id)
self.trust_ctx.assert_called_once_with(lease_values['trust_id'])
self.lease_create.assert_called_once_with(lease_values)
self.assertEqual(lease, self.lease)
def test_create_lease_validate_created_events(self):
lease_values = {
'id': self.lease_id,
'name': 'lease-name',
'reservations': [{'id': '111',
'resource_id': '111',
'resource_type': 'virtual:instance',
'status': 'FAKE PROGRESS'}],
'start_date': '2026-11-13 13:13',
'end_date': '2026-12-13 13:13',
'trust_id': 'exxee111qwwwwe'}
lease_values = self.lease_values.copy()
self.lease['start_date'] = '2026-11-13 13:13:00'
self.lease['end_date'] = '2026-12-13 13:13:00'
self.lease['events'][0]['time'] = '2026-11-13 13:13:00'
@ -476,16 +482,7 @@ class ServiceTestCase(tests.TestCase):
self.assertEqual('UNDONE', event['status'])
def test_create_lease_before_end_event_is_before_lease_start(self):
lease_values = {
'id': self.lease_id,
'name': 'lease-name',
'reservations': [{'id': '111',
'resource_id': '111',
'resource_type': 'virtual:instance',
'status': 'FAKE PROGRESS'}],
'start_date': '2026-11-13 13:13',
'end_date': '2026-11-13 14:13',
'trust_id': 'exxee111qwwwwe'}
lease_values = self.lease_values.copy()
self.lease['start_date'] = '2026-11-13 13:13:00'
self.lease['end_date'] = '2026-12-13 13:13:00'
self.lease['events'][0]['time'] = '2026-11-13 13:13:00'
@ -520,15 +517,8 @@ class ServiceTestCase(tests.TestCase):
self.assertEqual('UNDONE', event['status'])
def test_create_lease_before_end_event_before_start_without_lease_id(self):
lease_values = {
'name': 'lease-name',
'reservations': [{'id': '111',
'resource_id': '111',
'resource_type': 'virtual:instance',
'status': 'FAKE PROGRESS'}],
'start_date': '2026-11-13 13:13',
'end_date': '2026-11-13 14:13',
'trust_id': 'exxee111qwwwwe'}
lease_values = self.lease_values.copy()
self.lease['start_date'] = '2026-11-13 13:13'
self.cfg.CONF.set_override('minutes_before_end_lease', 120,
@ -541,53 +531,27 @@ class ServiceTestCase(tests.TestCase):
self.assertEqual(3, len(lease['events']))
def test_create_lease_before_end_param_is_before_lease_start(self):
before_end_date = '2026-11-11 13:13'
start_date = '2026-11-13 13:13'
lease_values = {
'id': self.lease_id,
'name': 'lease-name',
'reservations': [{'id': '111',
'resource_id': '111',
'resource_type': 'virtual:instance',
'status': 'FAKE PROGRESS'}],
'start_date': start_date,
'end_date': '2026-11-14 13:13',
'trust_id': 'exxee111qwwwwe',
'before_end_date': before_end_date}
lease_values = self.lease_values.copy()
lease_values['before_end_date'] = '2026-11-11 13:13'
lease_values['start_date'] = '2026-11-13 13:13'
self.lease['start_date'] = '2026-11-13 13:13'
self.assertRaises(
exceptions.NotAuthorized, self.manager.create_lease, lease_values)
def test_create_lease_before_end_param_is_past_lease_ending(self):
before_end_date = '2026-11-15 13:13'
lease_values = {
'id': self.lease_id,
'name': 'lease-name',
'reservations': [{'id': '111',
'resource_id': '111',
'resource_type': 'virtual:instance',
'status': 'FAKE PROGRESS'}],
'start_date': '2026-11-13 13:13',
'end_date': '2026-11-14 13:13',
'trust_id': 'exxee111qwwwwe',
'before_end_date': before_end_date}
lease_values = self.lease_values.copy()
lease_values['start_date'] = '2026-11-13 13:13'
lease_values['end_date'] = '2026-11-14 13:13'
lease_values['before_end_date'] = '2026-11-15 13:13'
self.lease['start_date'] = '2026-11-13 13:13'
self.assertRaises(
exceptions.NotAuthorized, self.manager.create_lease, lease_values)
def test_create_lease_no_before_end_event(self):
lease_values = {
'id': self.lease_id,
'name': 'lease-name',
'reservations': [{'id': '111',
'resource_id': '111',
'resource_type': 'virtual:instance',
'status': 'FAKE PROGRESS'}],
'start_date': '2026-11-13 13:13',
'end_date': '2026-11-14 13:13',
'trust_id': 'exxee111qwwwwe'}
lease_values = self.lease_values.copy()
self.lease['start_date'] = '2026-11-13 13:13:00'
self.lease['end_date'] = '2026-11-14 13:13:00'
self.lease['events'][0]['time'] = '2026-11-13 13:13:00'
@ -616,18 +580,9 @@ class ServiceTestCase(tests.TestCase):
self.assertEqual('UNDONE', event['status'])
def test_create_lease_with_before_end_date_param(self):
before_end_date = '2026-11-14 10:13'
lease_values = {
'id': self.lease_id,
'name': 'lease-name',
'reservations': [{'id': '111',
'resource_id': '111',
'resource_type': 'virtual:instance',
'status': 'FAKE PROGRESS'}],
'start_date': '2026-11-13 13:13',
'end_date': '2026-11-14 13:13',
'trust_id': 'exxee111qwwwwe',
'before_end_date': before_end_date}
lease_values = self.lease_values.copy()
lease_values['before_end_date'] = '2026-11-14 10:13'
self.lease['start_date'] = '2026-11-13 13:13:00'
self.lease['end_date'] = '2026-11-14 13:13:00'
self.lease['events'][0]['time'] = '2026-11-13 13:13:00'
@ -656,75 +611,55 @@ class ServiceTestCase(tests.TestCase):
event = lease['events'][2]
self.assertEqual('before_end_lease', event['event_type'])
expected_before_end_time = datetime.datetime.strptime(
before_end_date, service.LEASE_DATE_FORMAT)
lease_values['before_end_date'], service.LEASE_DATE_FORMAT)
self.assertEqual(str(expected_before_end_time), event['time'])
self.assertEqual('UNDONE', event['status'])
def test_create_lease_wrong_date(self):
lease_values = {'name': 'lease-name',
'start_date': '2025-13-35 13:13',
'end_date': '2025-12-31 13:13',
'trust_id': 'exxee111qwwwwe'}
lease_values = self.lease_values.copy()
lease_values['start_date'] = '2025-13-35 13:13'
lease_values['end_date'] = '2025-12-31 13:13'
self.assertRaises(
manager_ex.InvalidDate, self.manager.create_lease, lease_values)
def test_create_lease_wrong_format_before_end_date(self):
before_end_date = '2026-14 10:13'
lease_values = {
'name': 'lease-name',
'start_date': '2026-11-13 13:13',
'end_date': '2026-11-14 13:13',
'before_end_date': before_end_date,
'trust_id': 'exxee111qwwwwe'}
lease_values = self.lease_values.copy()
lease_values['before_end_date'] = '2026-14 10:13'
self.assertRaises(
manager_ex.InvalidDate, self.manager.create_lease, lease_values)
def test_create_lease_start_date_in_past(self):
lease_values = {
'name': 'lease-name',
'start_date':
datetime.datetime.strftime(
datetime.datetime.utcnow() - datetime.timedelta(days=1),
service.LEASE_DATE_FORMAT),
'end_date': '2025-12-31 13:13',
'trust_id': 'exxee111qwwwwe'}
lease_values = self.lease_values.copy()
lease_values['start_date'] = datetime.datetime.strftime(
datetime.datetime.utcnow() - datetime.timedelta(days=1),
service.LEASE_DATE_FORMAT)
self.assertRaises(
exceptions.InvalidInput, self.manager.create_lease, lease_values)
def test_create_lease_end_before_start(self):
lease_values = {
'name': 'lease-name',
'start_date': '2026-11-13 13:13',
'end_date': '2026-11-13 12:13',
'trust_id': 'exxee111qwwwwe'}
lease_values = self.lease_values.copy()
lease_values['start_date'] = '2026-11-13 13:13'
lease_values['end_date'] = '2026-11-13 12:13'
self.assertRaises(
exceptions.InvalidInput, self.manager.create_lease, lease_values)
def test_create_lease_unsupported_resource_type(self):
lease_values = {
'id': self.lease_id,
'name': 'lease-name',
'reservations': [{'id': '111',
'resource_id': '111',
'resource_type': 'unsupported:type',
'status': 'FAKE PROGRESS'}],
'start_date': '2026-11-13 13:13',
'end_date': '2026-12-13 13:13',
'trust_id': 'exxee111qwwwwe'}
lease_values = self.lease_values.copy()
lease_values['reservations'] = [{'id': '111',
'resource_id': '111',
'resource_type': 'unsupported:type',
'status': 'FAKE PROGRESS'}]
self.assertRaises(manager_ex.UnsupportedResourceType,
self.manager.create_lease, lease_values)
def test_create_lease_duplicated_name(self):
lease_values = {
'name': 'duplicated_name',
'start_date': '2026-11-13 13:13',
'end_date': '2026-12-13 13:13',
'trust_id': 'exxee111qwwwwe'}
lease_values = self.lease_values.copy()
lease_values['name'] = 'duplicated_name'
self.patch(self.db_api,
'lease_create').side_effect = db_ex.BlazarDBDuplicateEntry
@ -732,10 +667,8 @@ class ServiceTestCase(tests.TestCase):
self.manager.create_lease, lease_values)
def test_create_lease_without_trust_id(self):
lease_values = {
'name': 'name',
'start_date': '2026-11-13 13:13',
'end_date': '2026-12-13 13:13'}
lease_values = self.lease_values.copy()
del lease_values['trust_id']
self.assertRaises(manager_ex.MissingTrustId,
self.manager.create_lease, lease_values)
@ -766,6 +699,18 @@ class ServiceTestCase(tests.TestCase):
self.assertRaises(manager_ex.MissingParameter,
self.manager.create_lease, value)
def test_create_lease_with_filter_exception(self):
lease_values = self.lease_values.copy()
self.enforcement.check_create.side_effect = (
enforcement_ex.MaxLeaseDurationException(lease_duration=200,
max_duration=100))
self.assertRaises(exceptions.NotAuthorized,
self.manager.create_lease,
lease_values=lease_values)
self.lease_create.assert_not_called()
def test_update_lease_completed_lease_rename(self):
lease_values = {'name': 'renamed'}
target = datetime.datetime(2015, 1, 1)
@ -846,7 +791,8 @@ class ServiceTestCase(tests.TestCase):
{
'id': '593e7028-c0d1-4d76-8642-2ffd890b324c',
'min': 3,
'max': 3
'max': 3,
'resource_type': 'virtual:instance'
}
]
}
@ -874,7 +820,8 @@ class ServiceTestCase(tests.TestCase):
'end_date': datetime.datetime(2013, 12, 20, 15, 00),
'id': '593e7028-c0d1-4d76-8642-2ffd890b324c',
'min': 3,
'max': 3
'max': 3,
'resource_type': 'virtual:instance'
}
)
calls = [mock.call('2eeb784a-2d84-4a89-a201-9d42d61eecb1',
@ -1374,32 +1321,93 @@ class ServiceTestCase(tests.TestCase):
'2eeb784a-2d84-4a89-a201-9d42d61eecb1',
{'time': datetime.datetime(2013, 12, 20, 13, 0)})
def test_delete_lease_before_starting_date(self):
fake_get_lease = self.patch(self.manager, 'get_lease')
fake_get_lease.return_value = self.lease
def test_update_lease_with_filter_exception(self):
self.enforcement.check_update.side_effect = (
enforcement_ex.MaxLeaseDurationException(lease_duration=200,
max_duration=100))
target = datetime.datetime(2013, 12, 20, 12, 00)
def fake_event_get(sort_key, sort_dir, filters):
if filters['event_type'] == 'start_lease':
return {'id': '2eeb784a-2d84-4a89-a201-9d42d61eecb1'}
elif filters['event_type'] == 'end_lease':
return {'id': '7085381b-45e0-4e5d-b24a-f965f5e6e5d7'}
elif filters['event_type'] == 'before_end_lease':
delta = datetime.timedelta(hours=1)
return {'id': '452bf850-e223-4035-9d13-eb0b0197228f',
'time': self.lease['end_date'] - delta,
'status': 'UNDONE'}
lease_values = {
'reservations': [
{
'id': '593e7028-c0d1-4d76-8642-2ffd890b324c',
'min': 3,
'max': 3,
'resource_type': 'virtual:instance'
}
]
}
reservation_get_all = (
self.patch(self.db_api, 'reservation_get_all_by_lease_id'))
reservation_get_all.return_value = [
{
'id': '593e7028-c0d1-4d76-8642-2ffd890b324c',
'resource_type': 'virtual:instance',
}
]
event_get = self.patch(db_api, 'event_get_first_sorted_by_filters')
event_get.side_effect = fake_event_get
target = datetime.datetime(2013, 12, 15)
with mock.patch.object(datetime,
'datetime',
mock.Mock(wraps=datetime.datetime)) as patched:
patched.utcnow.return_value = target
self.manager.delete_lease(self.lease_id)
self.assertRaises(exceptions.NotAuthorized,
self.manager.update_lease,
lease_id=self.lease_id, values=lease_values)
self.lease_update.assert_not_called()
def test_delete_lease_before_start(self):
def fake_event_get(sort_key, sort_dir, filters):
if filters['event_type'] == 'start_lease':
return {'id': 'fake', 'status': 'UNDONE'}
elif filters['event_type'] == 'end_lease':
return {'id': 'fake', 'status': 'UNDONE'}
else:
return None
fake_get_lease = self.patch(self.manager, 'get_lease')
fake_get_lease.return_value = self.lease
event_get = self.patch(db_api, 'event_get_first_sorted_by_filters')
event_get.side_effect = fake_event_get
enforcement_on_end = self.patch(self.enforcement, 'on_end')
self.manager.delete_lease(self.lease_id)
self.trust_ctx.assert_called_once_with(self.lease['trust_id'])
self.lease_destroy.assert_called_once_with(self.lease_id)
self.fake_plugin.on_end.assert_called_with('111')
enforcement_on_end.assert_called_once()
def test_delete_lease_after_ending(self):
def fake_event_get(sort_key, sort_dir, filters):
if filters['event_type'] == 'start_lease':
return {'id': 'fake', 'status': 'DONE'}
elif filters['event_type'] == 'end_lease':
return {'id': 'fake', 'status': 'DONE'}
else:
return None
def test_delete_lease_after_ending_date(self):
self.lease['reservations'][0]['status'] = 'deleted'
fake_get_lease = self.patch(self.manager, 'get_lease')
fake_get_lease.return_value = self.lease
event_get = self.patch(db_api, 'event_get_first_sorted_by_filters')
event_get.side_effect = fake_event_get
enforcement_on_end = self.patch(self.enforcement, 'on_end')
target = datetime.datetime(2013, 12, 20, 16, 00)
with mock.patch.object(datetime,
'datetime',
mock.Mock(wraps=datetime.datetime)) as patched:
patched.utcnow.return_value = target
self.manager.delete_lease(self.lease_id)
self.manager.delete_lease(self.lease_id)
expected_context = self.trust_ctx.return_value
self.lease_destroy.assert_called_once_with(self.lease_id)
@ -1408,8 +1416,9 @@ class ServiceTestCase(tests.TestCase):
self.notifier_api.format_lease_payload(self.lease),
'lease.delete')
self.fake_plugin.on_end.assert_not_called()
enforcement_on_end.assert_not_called()
def test_delete_lease_after_starting_date(self):
def test_delete_lease_after_start(self):
def fake_event_get(sort_key, sort_dir, filters):
if filters['event_type'] == 'start_lease':
return {'id': 'fake', 'status': 'DONE'}
@ -1417,23 +1426,22 @@ class ServiceTestCase(tests.TestCase):
return {'id': 'fake', 'status': 'UNDONE'}
else:
return None
event_get = self.patch(db_api, 'event_get_first_sorted_by_filters')
event_get.side_effect = fake_event_get
fake_get_lease = self.patch(self.manager, 'get_lease')
fake_get_lease.return_value = self.lease
target = datetime.datetime(2013, 12, 20, 13, 30)
with mock.patch.object(datetime,
'datetime',
mock.Mock(wraps=datetime.datetime)) as patched:
patched.utcnow.return_value = target
self.manager.delete_lease(self.lease_id)
enforcement_on_end = self.patch(self.enforcement, 'on_end')
self.manager.delete_lease(self.lease_id)
self.event_update.assert_called_once_with('fake',
{'status': 'IN_PROGRESS'})
self.fake_plugin.on_end.assert_called_with('111')
self.lease_destroy.assert_called_once_with(self.lease_id)
enforcement_on_end.assert_called_once()
def test_delete_lease_after_starting_date_with_error_status(self):
def test_delete_lease_after_start_with_error_status(self):
def fake_event_get(sort_key, sort_dir, filters):
if filters['event_type'] == 'start_lease':
return {'id': 'fake', 'status': 'ERROR'}
@ -1441,21 +1449,47 @@ class ServiceTestCase(tests.TestCase):
return {'id': 'fake', 'status': 'UNDONE'}
else:
return None
event_get = self.patch(db_api, 'event_get_first_sorted_by_filters')
event_get.side_effect = fake_event_get
fake_get_lease = self.patch(self.manager, 'get_lease')
fake_get_lease.return_value = self.lease
target = datetime.datetime(2013, 12, 20, 13, 30)
with mock.patch.object(datetime,
'datetime',
mock.Mock(wraps=datetime.datetime)) as patched:
patched.utcnow.return_value = target
self.manager.delete_lease(self.lease_id)
enforcement_on_end = self.patch(self.enforcement, 'on_end')
self.manager.delete_lease(self.lease_id)
self.event_update.assert_called_once_with('fake',
{'status': 'IN_PROGRESS'})
self.fake_plugin.on_end.assert_called_with('111')
self.lease_destroy.assert_called_once_with(self.lease_id)
enforcement_on_end.assert_called_once()
def test_delete_lease_with_filter_exception(self):
self.enforcement.on_end.side_effect = (
exceptions.BlazarException)
def fake_event_get(sort_key, sort_dir, filters):
if filters['event_type'] == 'start_lease':
return {'id': 'fake', 'status': 'DONE'}
elif filters['event_type'] == 'end_lease':
return {'id': 'fake', 'status': 'UNDONE'}
else:
return None
event_get = self.patch(db_api, 'event_get_first_sorted_by_filters')
event_get.side_effect = fake_event_get
fake_get_lease = self.patch(self.manager, 'get_lease')
fake_get_lease.return_value = self.lease
enforcement_on_end = self.patch(self.enforcement, 'on_end')
self.manager.delete_lease(self.lease_id)
self.event_update.assert_called_once_with('fake',
{'status': 'IN_PROGRESS'})
self.fake_plugin.on_end.assert_called_with('111')
self.lease_destroy.assert_called_once_with(self.lease_id)
enforcement_on_end.assert_called_once()
def test_start_lease(self):
basic_action = self.patch(self.manager, '_basic_action')
@ -1468,12 +1502,14 @@ class ServiceTestCase(tests.TestCase):
def test_end_lease(self):
basic_action = self.patch(self.manager, '_basic_action')
enforcement_on_end = self.patch(self.enforcement, 'on_end')
self.manager.end_lease(self.lease_id, '1')
self.trust_ctx.assert_called_once_with(self.lease['trust_id'])
basic_action.assert_called_once_with(self.lease_id, '1', 'on_end',
'deleted')
enforcement_on_end.assert_called_once()
def test_before_end_lease(self):
basic_action = self.patch(self.manager, '_basic_action')

1
doc/source/admin/index.rst
View File

@ -10,3 +10,4 @@ Administrator Guide
../cli/index
../restapi/index
blazar-status
usage-enforcement

50
doc/source/admin/usage-enforcement.rst Normal file
View File

@ -0,0 +1,50 @@
=================
Usage Enforcement
=================
Synopsis
========
Usage enforcement and lease constraints can be implemented by operators via
custom usage enforcement filters.
Description
===========
Usage enforcement filters are called on ``lease_create``, ``lease_update`` and
``on_end`` operations. The filters check whether or not lease values or
allocation criteria pass admin defined thresholds. There is currently one
filter provided out-of-the-box. The ``MaxLeaseDurationFilter`` restricts the
duration of leases.
Options
=======
All filters are a subclass of the BaseFilter class located in
``blazar/enforcement/filter/base_filter.py``. Custom filters must implement
methods for ``check_create``, ``check_update``, and ``on_end``. The
``MaxLeaseDurationFilter`` is a good example to follow. Filters are enabled in
``blazar.conf`` under the ``[enforcement]`` group. For example, enabling the
``MaxLeaseDurationFilter`` to limit lease durations to only one day would work
as follows:
.. sourcecode:: console
[enforcement]
enabled_filters = MaxLeaseDurationFilter
max_lease_duration = 86400
..
MaxLeaseDurationFilter
----------------------
This filter simply examines the lease ``start_time`` and ``end_time``
attributes and rejects the lease if its duration exceeds a threshold. It
supports two configuration options:
* ``max_lease_duration``
* ``max_lease_duration_exempt_project_ids``
See the :doc:`../configuration/blazar-conf` page for a description of these
options.

6
releasenotes/notes/usage-enforcement-f997ce618f542104.yaml Normal file
View File

@ -0,0 +1,6 @@
---
features:
- |
A filter-based usage enforcement framework is introduced in this release.
Enforcement filters allow operators to define lease constraints. The first
filter introduced in this release restricts maximum lease duration.