Commit Graph

47 Commits (cb80df4f4ab0e8de16d53c7fddff7c44dedd3f94)

Author SHA1 Message Date
ricolin 3bb0b8b4e4 Enable mutable config in Heat
New releases of oslo.config support a 'mutable' parameter to Opts.
This is only respected when the new method mutate_config_files is
called instead of reload_config_files. Heat delegates making this call
to oslo.service, so how do we switch?
Icec3e664f3fe72614e373b2938e8dee53cf8bc5e allows services to tell
oslo.service they want mutate_config_files to be called by passing a
parameter, which this patch does.

This allows Heat to benefit from
I1e7a69de169cc85f4c09954b2f46ce2da7106d90, where the 'debug' option
(owned by oslo.log) is made mutable. We should be able to turn debug
logging on and off by changing the config and sending SIGHUP.

Part of bp:mutable-config

Change-Id: I885f5e088440d3d8b6fb3506506598d7667e7083
6 years ago
liyi 8f10215ffd Remove log translations
Log messages are no longer being translated. This removes all use of
the _LE, _LI, and _LW translation markers to simplify logging and to
avoid confusion with new contributions.


Change-Id: Ieec8028305099422e1b0f8fc84bc90c9ca6c694f
6 years ago
Crag Wolfe 98505288b2 Add heat-manage subcommand to migrate legacy prop. data
Add a subcommand to heat-manage to migrate resource and events
properties data from the legacy db locations to the new. I.e., migrate
properties data out of the legacy columns in the resource and event
tables into the recently added resource_properties_data table. No
attempt at de-duplication between resources and events is made for the
migration: a new row is created in resource_properties_data for every
row that has legacy properties data in the resource or event tables.

Change-Id: I364d509c357539d1929eb2e40704e60049469ea2
6 years ago
rabi 0a66b7750f Sanitize newly added heat-manage command
This changes newly added heat-manage command for migrating
non-convergence stacks to convergence, to use '_' like other
commands. Also updates doc with the command.

Change-Id: I2ea5beb1c71cf51dceee7df6af6a6827d920c179
7 years ago
Thomas Herve 84067dba88 Remove db.api wrapper
The db.api module provides a useless indirection to the only
implementation we ever had, sqlalchemy. Let's use that directly instead
of the wrapper.

Change-Id: I80353cfed801b95571523515fd3228eae45c96ae
7 years ago
Crag Wolfe 882a640f18 Refactor purge_deleted, operate on batches of stacks
Avoid large sql "in" clauses by operating on smaller batches of stacks
at a time.

To avoid transaction overhead and contention on the resource table,
the first deletions occur outside of a transaction (are
autocommitted). This is OK because the purge is re-rentrant -- we
won't lose any stack_id's to delete if something goes wrong before the
conn.begin() block. That is, we will not orphan any rows if the purge
is run multiple times where an error occurs.

Change-Id: I9edf0558ed54820842193560e323df6501411d1d
7 years ago
Dan Prince 1486289a75 heat-all: all-in-one heat launcher
This patch implements a new heat-all command that can be used
to launch a single process version of the configured heat services.
The end user can control which services are launched by setting
'enabled_services' in the heat config file:

  enabled_services = api,engine

One use case for this launcher would be to launch a single process heat
using rpc_backend = fake, connection=sqlite://heat.db, to have access
to a minimal all in one Heat API/Engine for TripleO undercloud
deployments via Heat templates.

Change-Id: Ic35bdc455fe1397ac575522af0c8fca96dd95676
7 years ago
shizhihui f34a7bb0d8 Trivial:Standardize the output format of 'help' in some files
In some files,it is possible that we import '_'from heat.commom.i18n.
For more information:


Change-Id: I545b9e51111a9c9c542e3089bef1490a3ba3a40b
7 years ago
Oleksii Chuprykov 68944d2230 Migrate stacks from legacy to convergence engine
Run `heat-manage migrate-convergence-1 <stack_id>` to migrate
legacy stack to convergence engine.

Heat engine is used for doing migration i.e. migration can't
be done offline.

Change-Id: Ie7c2498b37937438f16d154b154b3a6ecbf9ff74
Implements-bp: convergence-migrate-stack
7 years ago
Ala Rezmerita 61836dbf42 Purge deleted stacks for specific project
Add project-id argument to heat-manage purge_deleted command in order
to be able to hard delete DB entries for a specific project.

Change-Id: Ifffe5657a40ce97db9d059ff1516b8e1eb801132
Implements: bp heat-manage-purge-deleted-tenant
7 years ago
yuyafei add4dc3b08 Replace raw_input with input to make PY3 compatible
The raw_input() raises NameError: name 'raw_input' is not defined in python3.
This patch fixes it by replacing raw_input with input to make PY3 compatible.

Change-Id: I50aad3161893250dab1d60af4569bb65d7edef5a
Closes-Bug: #1595827
7 years ago
Jenkins 869249a495 Merge "Add --verbose-update-params option to heat-manage" 7 years ago
Jason Dunsmore 59084fa24e Add --verbose-update-params option to heat-manage
In case the "heat-manage update-params" utility hangs in the middle of
a run, it is useful to know the exact raw_template or resource that
was being processed. Verbose logging of each raw_template or resource
that is being processed will aid in debugging and checking for corrupt
data (if the heat-manage process has to be killed).

Change-Id: Ic552bdaf234c7b9a6a76c345aab816a171b404ab
7 years ago
Thomas Herve 7977f9f2f3 Add command to reset one stack status
Adds a new heat-manage reset_stack_status to recover from specific
crashes that leaves resources in progress. It removes resource hooks and
stack locks as well.

Closes-Bug: #1561214
Change-Id: I70fa5857c959bc5f1424d562ff8b7740331b5328
7 years ago
Michael Krotscheck 59ccb2f751 Moved CORS middleware configuration into oslo-config-generator
The default values needed for heat's implementation of cors
middleware have been moved from paste.ini into the configuration
hooks provided by oslo.config. Furthermore, these values have been
added to the default configuration parsing. This ensures
that if a value remains unset in heat.conf, it will be set
to use sane defaults, and that an operator modifying the
configuration file will be presented with a default set of
necessary sane headers.

Change-Id: Ie3791007b33788829417ce508a3c719ae626bbce
Closes-Bug: 1551836
7 years ago
Steve Baker adb2121795 Make minimum default num_engine_workers>=4
Downstream test environments are frequently having failing stacks with
error messages like:

  MessagingTimeout: resources[0]: Timed out waiting for a reply to
  message ID ...

These environments generally have 1 or 2 cores, so only spawn one or two
engine workers. This deadlocks with stacks that have many nested stacks
due to engine->engine RPC calls.

Even our own functional tests don't work reliably with less than 4
workers, and the workaround has been to set that explicitly in

This change sets the default minimum number of workers to 4, but still
matches workers to cores for larger servers.

This change also moves the default evaluation to heat.cmd.engine so that
generated configuration doesn't get a inappropriate default value.

Change-Id: Iae6b3956bad414406d901bb2213c9ec230ff4304
Closes-Bug: #1526045
8 years ago
Jenkins 4b53e073a2 Merge "Stop ignore skipped crypt_operation option" 8 years ago
Jenkins fb54b8d602 Merge "Remove unused argument 'current_version'" 8 years ago
Kanagaraj Manickam 6ef46af6b4 heat-manage resource_data_list
During the troubleshooting of issues, admin needs to
decrypt the values stored in the resource_data table.
This patch addes below command to address this need.

heat-manage resource_data_list <resource_id>

Closes-bug: #1524295

Change-Id: I5bb814c0873f58b3968f312c96bcabcd3024085d
8 years ago
Sergey Kraynev d915471db1 Stop ignore skipped crypt_operation option
Previously we used "nargs='?'" parameter for 'crypt_operation' option.
It leads to situation, when we execute this command without any result
or error.
Now when user will try to execute this command he will get message, that
should specify 'crypt_operation' option.

Note, that we used same parameters for other heat-manage commands.
However, it can be used, because if user will not provide these options,
then commands will be executed with default values (sometimes it's None).
All these default values are expected in utility functions or db

Also comments were added for clarification commands and their arguments.

Change-Id: I561bc7fc83e6d845c9b59d85c59650e640001190
8 years ago
Sergey Kraynev a77739b46b Remove unused argument 'current_version'
db_sync command has two positional arguments, but in code we use only
one. Also original db_sync command for db API get only one parameter

Change-Id: I65732b16fbdc093ad2c7a3a521c3b75ae7183585
8 years ago
Peter Razumovsky 4e0510ffe6 Fix [H405] pep rule in heat/cmd
Implements bp docstring-improvements

Change-Id: I6600d0ac4825c97da486b50f9695d0075b3f16ce
8 years ago
Sirushti Murugesan 0649787f1d Fix file permissions
Remove the executable bit

Change-Id: I81220d3222e6a6e27f074489f607158581851a6b
8 years ago
Ethan Lynn d6b618be55 Add command to clean dead engine from service list
Add command 'service clean' to remove dead engines
from service list.

Closes-Bug: #1495374
Change-Id: I644eee717f8e76ab9d5d7d6847442740ea201e6b
8 years ago
Sirushti Murugesan 063f80ffdf Add console_scripts entry points for all heat services
As a follow-up to Ibfd54e7b1bae700e4fb49a32e5bf7c00d1f104d9,
moving all the scripts from bin/ to setuptools entry points.

The devstack change is here: Iffb6d09bfef593d854b38e68200ae6039c4727e7

partial blueprint upgrade-tests

Change-Id: Ic8f5b8dc85098de752bbf673c4b15c06fdc4162a
8 years ago
Jason Dunsmore bafd884419 Add doc about db being offline to encryption utility
The utility wasn't designed to be run while live stacks are being

Change-Id: I89e5152ad32ea22a38da7ae410c47e1e7a8551a9
8 years ago
Vijendar Komalla 654032459c Support for encrypt/decrypt parameters in heat-manage
Adding support to encrypt/decrypt parameters through heat-manage

Change-Id: I2cd7b8837156ebe892d591f48ad2b1c6ecca5f9d
Implements: blueprint encrypt-hidden-parameters
Co-Authored-By: Jason Dunsmore <>
8 years ago
Sergey Kraynev e5f952e4b5 Use Service objects instead of db_api
Fix missed place for migration from db_api to Service object.

Change-Id: I1d733d0a1bd13ab860860e4497771c46e7d2118b
8 years ago
Jens Rosenboom a99fe6a760 Make import of oslo libraries namespaceless

Closes-Bug: 1423174
Change-Id: I48680ce6e7ce91005c147ab4388203946171d433
8 years ago
Steven Hardy 70112c103a Move logging to use oslo_log library
The oslo-incubator log modlule has been removed, so port to the oslo_log
library.  Note this uses the new (non namespaced, e.g oslo.log) import
convention, we'll need to align other imports in a future commit.

Some import reordering was required due to pedantic H30[57] checks, and
the services have all been converted to initialize the oslo_log library
as this is done differently to the in incubator.

Change-Id: Ib5a97123fe1b287bc531e42d7887c13ba6205628
8 years ago
Kanagaraj Manickam 65134b8d99 heat-manage service list
Adds required REST API, Db model and engine service
changes for reporting the heat engine service status.

Change-Id: I3ef29c1efd2015d62eec1033ed3a8c9f42e7a6e2
Implements: blueprint heat-manage-service-list
8 years ago
Ethan Lynn 11c1b62052 Remove deprecated function i18n.install('heat')
Since i18n.install() is deprecated, remove it from heat codes and
import i18n._() to where it needed.

blueprint oslo-i18n
Change-Id: Icefada18b5a33112b425cd90d31d3a6a5f06188a
9 years ago
Andrey Kurilin cfcbc994d3 Sync latest oslo.db code
Common db code was updated in oslo. The most important thing is that
engine instances don't stored anymore in oslo.db - ce69e7f.
This patch moves methods `get_engine` and `get_session` to module

Latest commit in oslo related to db module:

Change-Id: Iaa2e9ba26e824c678c698914170e3dffbf1c5c95
9 years ago
Jason Dunsmore af464c9afc Make the first line of every file consistent.
Change-Id: I2e1a809cfca8e88693551d58d33e747f54ee5eb1
9 years ago
Jason Dunsmore 97135cbfc0 Remove vim header from files.
Change-Id: Ia6131f8fcac6902c80c80aa805c8cd7a2a41f280
Closes-Bug: #1229324
9 years ago
Andreas Jaeger fb538071f6 Improve help strings
Follow oslo.config style guide for help strings better to create
consistent help strings:
* Capitalize first word of each help string
* Finish help strings with "."
* Improve wording
* Add missing space between strings

Change-Id: Ia86734d44fdf4eb3711fef800fe6bdfd1a1b59d4
9 years ago
Jenkins daa8d6c216 Merge "Add granularity option to purge_deleted" 10 years ago
Dan Prince afd761d4e7 Move db_sync into the sqlalchemy DB api.
heat-manage db_sync is currently broken because
heat is trying to use two backends with the Oslo DB
api which is configurable via heat.conf where only
a single DB api can be specified.

Currently this defaults to:

 #The backend to use for db (string value)

To fix things we:

1) merge heat/db/ into heat/db/

2) Add db_sync and db_version calls to heat/db/sqlalchemy/
  which call into the functions in heat/db/sqlalchemy/

3) Remove the old heat/db/ module (no longer needed).

In the process we also move the INIT_VERSION constant into
heat/db/sqlalchemy/ where it is actually used
anyway. (it probably shouldn't have been in the higher level
migration module to begin with)

This should fix SmokeStack which stopped working last Wednesday
with a05dc4b9cd

Closes-bug #1241865

Change-Id: I50d489531056016c762bd3be047e3a129cbee4f8
10 years ago
Liang Chen b61cd62db6 Add granularity option to purge_deleted
Allow users to choose how many days, hours, minutes, or seconds to preserve
deleted data.

Closes-bug: #1237195

Change-Id: I9227f5bf53c5b099a7a691a49c4ff2f5f0662098
10 years ago
Jenkins 3da6d4d8a9 Merge "Migrate to Oslo DB 96d1f887dda Part 3" 10 years ago
Alexander Gordeev 4582a1a0dd Migrate to Oslo DB 96d1f887dda Part 3
Oslo version 96d1f887dda21b43ba4376187f31953dee6f5273

This commit just migrates Heat to new db related code from Oslo

Partially implements blueprint oslo-db-support

Change-Id: Ib0de40a70857fb3e029a2282f06746269a628c71
10 years ago
Clint Byrum b215a117c0 Add help for purge_deleted command age argument
At a bare minimum explaining this argument helps understand what
purge_deleted is going to do.

Change-Id: If73ba272e8ea0742f8be93994ec2fac68d4ed3fd
10 years ago
Liang Chen 30b8a186a8 Provide a way to clean up soft deleted data
Add a purge_deleted command to heat-manage. The command takes an
'age' argument, and removes all database records that have been soft
deleted for more than 'age' days. Default to 90.

blueprint event-persistence

Change-Id: I8a62108a907b4e709c91a93a1f7db6c702cec6b5
10 years ago
guohliu eb0dafb44b Fix command name display issue in heat-manage usage
Fixed bug #1194840

Change-Id: Ibe43d0a025f0aed6bec7e414f7e80d0a0f652308
10 years ago
Dirk Mueller e6f6890ac0 Use print_function compatible syntax
This changes the code to use a python 3.x compatible
print function syntax (or import from __future__ where

Change-Id: Ia1f19d0ac082853d25c7c9b754b440469c0526eb
10 years ago
Derek Higgins 3f72ad87f4 Use heat-engine.conf for heat-manage config
Fixes bug 1184923

because prog was missing the CONF object was looking for
the config file in /etc/heat/heat.conf  this should have
been /etc/heat/heat-engine.conf

Change-Id: Ie23b9998df368274d69180e0d619b042dd9dc053
10 years ago
Angus Salkeld 126bf9f28c Add a basic heat-manage
The heat/cmd/ structure is copied from glance to give
us some consistency.

Removing unused cliutils ...

blueprint heat-manage
Change-Id: If3b69eb4c3672740515fd308ec868af1cd3c9396
10 years ago