History

Dan Smith 30d9bf9a6d Async task support We have a ton of stuff in devstack that is very linear, specifically the ten-ish minutes we spend loading osc to run a single API command against something. We also generate configs, sync databases, and other things that use one core of our worker and make our runtime longer than it really needs to be. The idea in this patch is to make it super simple to run some things in the background and then wait for them to finish before proceeding to something that will require them to be done. This avoids the interleaving you would expect by redirecting the async tasks to a log file, and then cat'ing that log file synchronously during the wait operation. The per-task log file remains so it's easier to examine it in isolation. Multiple people have reported between 22-30% improvement in the time it takes to stack with this. More can be done, but what is here already makes a significant difference. Change-Id: I270a910b531641b023c13f75dfedca057a1f1031	2021-02-09 15:57:04 -08:00
..
80-tempest.sh	Async task support	2021-02-09 15:57:04 -08:00
README.md	doc: document override_defaults phase	2015-11-18 12:43:54 +05:30

Dan Smith 30d9bf9a6d Async task support

We have a *ton* of stuff in devstack that is very linear, specifically
the ten-ish minutes we spend loading osc to run a single API command
against something. We also generate configs, sync databases, and other
things that use one core of our worker and make our runtime longer
than it really needs to be.

The idea in this patch is to make it super simple to run some things
in the background and then wait for them to finish before proceeding
to something that will require them to be done. This avoids the
interleaving you would expect by redirecting the async tasks to a log
file, and then cat'ing that log file synchronously during the wait
operation. The per-task log file remains so it's easier to examine
it in isolation.

Multiple people have reported between 22-30% improvement in the
time it takes to stack with this. More can be done, but what is here
already makes a significant difference.

Change-Id: I270a910b531641b023c13f75dfedca057a1f1031

2021-02-09 15:57:04 -08:00

80-tempest.sh

Async task support

2021-02-09 15:57:04 -08:00

README.md

doc: document override_defaults phase

2015-11-18 12:43:54 +05:30

README.md

Extras Hooks

The extras.d directory contains project dispatch scripts that are called at specific times by stack.sh, unstack.sh and clean.sh. These hooks are used to install, configure and start additional projects during a DevStack run without any modifications to the base DevStack scripts.

When stack.sh reaches one of the hook points it sources the scripts in extras.d that end with .sh. To control the order that the scripts are sourced their names start with a two digit sequence number. DevStack reserves the sequence numbers 00 through 09 and 90 through 99 for its own use.

The scripts are sourced at the beginning of each script that calls them. The entire stack.sh variable space is available. The scripts are sourced with one or more arguments, the first of which defines the hook phase:

override_defaults | source | stack | unstack | clean

override_defaults: always called first in any of the scripts, used to
    override defaults (if need be) that are otherwise set in lib/* scripts

source: called by stack.sh. Used to set the initial defaults in a lib/*
    script or similar

stack: called by stack.sh.  There are four possible values for
    the second arg to distinguish the phase stack.sh is in:

    arg 2:  pre-install | install | post-config | extra

unstack: called by unstack.sh

clean: called by clean.sh.  Remember, clean.sh also calls unstack.sh
    so that work need not be repeated.

The stack phase sub-phases are called from stack.sh in the following places:

pre-install - After all system prerequisites have been installed but before any
    DevStack-specific services are installed (including database and rpc).

install - After all OpenStack services have been installed and configured
    but before any OpenStack services have been started.  Changes to OpenStack
    service configurations should be done here.

post-config - After OpenStack services have been initialized but still before
    they have been started. (This is probably mis-named, think of it as post-init.)

extra - After everything is started.