New releases of oslo.config support a 'mutable' parameter to Opts.
This is only respected when the new method mutate_config_files is
called instead of reload_config_files. Heat delegates making this call
to oslo.service, so how do we switch?
Icec3e664f3fe72614e373b2938e8dee53cf8bc5e allows services to tell
oslo.service they want mutate_config_files to be called by passing a
parameter, which this patch does.
This allows Heat to benefit from
I1e7a69de169cc85f4c09954b2f46ce2da7106d90, where the 'debug' option
(owned by oslo.log) is made mutable. We should be able to turn debug
logging on and off by changing the config and sending SIGHUP.
Part of bp:mutable-config
Add a subcommand to heat-manage to migrate resource and events
properties data from the legacy db locations to the new. I.e., migrate
properties data out of the legacy columns in the resource and event
tables into the recently added resource_properties_data table. No
attempt at de-duplication between resources and events is made for the
migration: a new row is created in resource_properties_data for every
row that has legacy properties data in the resource or event tables.
This changes newly added heat-manage command for migrating
non-convergence stacks to convergence, to use '_' like other
commands. Also updates doc with the command.
The db.api module provides a useless indirection to the only
implementation we ever had, sqlalchemy. Let's use that directly instead
of the wrapper.
Avoid large sql "in" clauses by operating on smaller batches of stacks
at a time.
To avoid transaction overhead and contention on the resource table,
the first deletions occur outside of a transaction (are
autocommitted). This is OK because the purge is re-rentrant -- we
won't lose any stack_id's to delete if something goes wrong before the
conn.begin() block. That is, we will not orphan any rows if the purge
is run multiple times where an error occurs.
This patch implements a new heat-all command that can be used
to launch a single process version of the configured heat services.
The end user can control which services are launched by setting
'enabled_services' in the heat config file:
enabled_services = api,engine
One use case for this launcher would be to launch a single process heat
using rpc_backend = fake, connection=sqlite://heat.db, to have access
to a minimal all in one Heat API/Engine for TripleO undercloud
deployments via Heat templates.
Run `heat-manage migrate-convergence-1 <stack_id>` to migrate
legacy stack to convergence engine.
Heat engine is used for doing migration i.e. migration can't
be done offline.
Add project-id argument to heat-manage purge_deleted command in order
to be able to hard delete DB entries for a specific project.
Implements: bp heat-manage-purge-deleted-tenant
The raw_input() raises NameError: name 'raw_input' is not defined in python3.
This patch fixes it by replacing raw_input with input to make PY3 compatible.
In case the "heat-manage update-params" utility hangs in the middle of
a run, it is useful to know the exact raw_template or resource that
was being processed. Verbose logging of each raw_template or resource
that is being processed will aid in debugging and checking for corrupt
data (if the heat-manage process has to be killed).
Adds a new heat-manage reset_stack_status to recover from specific
crashes that leaves resources in progress. It removes resource hooks and
stack locks as well.
The default values needed for heat's implementation of cors
middleware have been moved from paste.ini into the configuration
hooks provided by oslo.config. Furthermore, these values have been
added to the default configuration parsing. This ensures
that if a value remains unset in heat.conf, it will be set
to use sane defaults, and that an operator modifying the
configuration file will be presented with a default set of
necessary sane headers.
Downstream test environments are frequently having failing stacks with
error messages like:
MessagingTimeout: resources: Timed out waiting for a reply to
message ID ...
These environments generally have 1 or 2 cores, so only spawn one or two
engine workers. This deadlocks with stacks that have many nested stacks
due to engine->engine RPC calls.
Even our own functional tests don't work reliably with less than 4
workers, and the workaround has been to set that explicitly in
This change sets the default minimum number of workers to 4, but still
matches workers to cores for larger servers.
This change also moves the default evaluation to heat.cmd.engine so that
generated configuration doesn't get a inappropriate default value.
During the troubleshooting of issues, admin needs to
decrypt the values stored in the resource_data table.
This patch addes below command to address this need.
heat-manage resource_data_list <resource_id>
Previously we used "nargs='?'" parameter for 'crypt_operation' option.
It leads to situation, when we execute this command without any result
Now when user will try to execute this command he will get message, that
should specify 'crypt_operation' option.
Note, that we used same parameters for other heat-manage commands.
However, it can be used, because if user will not provide these options,
then commands will be executed with default values (sometimes it's None).
All these default values are expected in utility functions or db
Also comments were added for clarification commands and their arguments.
db_sync command has two positional arguments, but in code we use only
one. Also original db_sync command for db API get only one parameter
As a follow-up to Ibfd54e7b1bae700e4fb49a32e5bf7c00d1f104d9,
moving all the scripts from bin/ to setuptools entry points.
The devstack change is here: Iffb6d09bfef593d854b38e68200ae6039c4727e7
partial blueprint upgrade-tests
Adding support to encrypt/decrypt parameters through heat-manage
Implements: blueprint encrypt-hidden-parameters
Co-Authored-By: Jason Dunsmore <firstname.lastname@example.org>
The oslo-incubator log modlule has been removed, so port to the oslo_log
library. Note this uses the new (non namespaced, e.g oslo.log) import
convention, we'll need to align other imports in a future commit.
Some import reordering was required due to pedantic H30 checks, and
the services have all been converted to initialize the oslo_log library
as this is done differently to the log.py in incubator.
Adds required REST API, Db model and engine service
changes for reporting the heat engine service status.
Implements: blueprint heat-manage-service-list
Common db code was updated in oslo. The most important thing is that
engine instances don't stored anymore in oslo.db - ce69e7f.
This patch moves methods `get_engine` and `get_session` to module
Latest commit in oslo related to db module:
Follow oslo.config style guide for help strings better to create
consistent help strings:
* Capitalize first word of each help string
* Finish help strings with "."
* Improve wording
* Add missing space between strings
heat-manage db_sync is currently broken because
heat is trying to use two backends with the Oslo DB
api which is configurable via heat.conf where only
a single DB api can be specified.
Currently this defaults to:
#The backend to use for db (string value)
To fix things we:
1) merge heat/db/migration.py into heat/db/api.py.
2) Add db_sync and db_version calls to heat/db/sqlalchemy/api.py
which call into the functions in heat/db/sqlalchemy/migration.py.
3) Remove the old heat/db/migration.py module (no longer needed).
In the process we also move the INIT_VERSION constant into
heat/db/sqlalchemy/api.py where it is actually used
anyway. (it probably shouldn't have been in the higher level
migration module to begin with)
This should fix SmokeStack which stopped working last Wednesday
Oslo version 96d1f887dda21b43ba4376187f31953dee6f5273
This commit just migrates Heat to new db related code from Oslo
Partially implements blueprint oslo-db-support
Add a purge_deleted command to heat-manage. The command takes an
'age' argument, and removes all database records that have been soft
deleted for more than 'age' days. Default to 90.
Fixes bug 1184923
because prog was missing the CONF object was looking for
the config file in /etc/heat/heat.conf this should have