[Spec] Use Centralized database for cache operations
Change-Id: I0b9d023b1736a5517e0583b13e0cb834d9495ab8
This commit is contained in:
Abhishek Kekane
parent
c6492bd705
commit
a3920b7fb2
210
specs/2024.1/approved/glance/centralized-cache-db.rst
Normal file
210
specs/2024.1/approved/glance/centralized-cache-db.rst
Normal file
@ -0,0 +1,210 @@
|
||||
..
|
||||
This work is licensed under a Creative Commons Attribution 3.0 Unported
|
||||
License.
|
||||
|
||||
http://creativecommons.org/licenses/by/3.0/legalcode
|
||||
|
||||
====================================
|
||||
Use Centralized database for caching
|
||||
====================================
|
||||
|
||||
https://blueprints.launchpad.net/glance/+spec/centralized-cache-db
|
||||
|
||||
Problem description
|
||||
===================
|
||||
|
||||
We use two different database(s) for glance. One is a centralized database
|
||||
managed by oslo.db (default is MySQL) for storing information related to
|
||||
regular operations; the other one is a sqlite database for
|
||||
storing information about caching related operations. Even though the
|
||||
sqlite database is only created if we enable cache middleware we still
|
||||
need to maintain it along with centralized database. Also glance caches
|
||||
are local to each controller node (glance api service), so in case of
|
||||
multiple glance services running we have multiple sqlite database(s)
|
||||
operational.
|
||||
|
||||
Proposed change
|
||||
===============
|
||||
|
||||
We propose to use centralized database (default MySql) for caching related
|
||||
operations and stop using sqlite database from this release onwards. In this
|
||||
case we can get rid of creating sqlite database per glance service and
|
||||
have everything under central database.
|
||||
|
||||
At present users can choose between `sqlite` (Default) and `xattr` for how
|
||||
caching can be controlled using `image_cache_driver` configuration option. We
|
||||
propose to introduce new driver `centralized_db` and required configuration
|
||||
options, make it default and mark `sqlite` driver and related configuration
|
||||
options as deprecated in this cycle and remove it in 'D' development
|
||||
cycle.
|
||||
|
||||
In order to use central database for caching we recommend to have almost
|
||||
similar schema (columns) as sqlite database with additional reference to
|
||||
local node (glance service). New deployments will directly start using this
|
||||
new central database if cache is enabled.
|
||||
|
||||
In case of upgrades/updates we need to deal with migrating existing records
|
||||
from sqlite database to central database. Since we might have multiple
|
||||
glance(s) running this will be difficult to handle using alembic migrations.
|
||||
To avoid this using migration we can perform this operation one time during
|
||||
service startup. If sqlite database is present at service start and
|
||||
`image_cache_driver` is not set to `centralized_db` then we will
|
||||
read records from it and insert those in newly created `cached_images` table
|
||||
in central database. Once all records are migrated we will clear the sqlite
|
||||
database table and keep the sqlite database file as it is (to be deleted by
|
||||
administrator/operator later if required). Important point here is once
|
||||
deployer chooses to use `centralized_db` and we migrate their records out
|
||||
of `sqlite` to centralized database, then we will not migrate them back
|
||||
if deployer wants to revert back to `sqlite` driver.
|
||||
|
||||
As deployment can have multiple glance services running, to record cache
|
||||
details separately for multiple glance services, we can use existing
|
||||
configuration option ``worker_self_reference_url`` which will
|
||||
differentiate cache records for each glance service running in the
|
||||
deployment. ``worker_self_reference_url`` is used to identify nodes in the case
|
||||
of new import workflow earlier, and now the same can be used for this purpose
|
||||
as well. Since ``worker_self_reference_url`` is required for `glance-direct`
|
||||
import method as well, if we found that it was not set by the deployer for
|
||||
any of the glance setup, then at the service startup we will log appropriate
|
||||
error message and prevent the service from starting. Also, since in the case of
|
||||
multiple nodes, ``worker_self_reference_url`` needs to be set to a unique value
|
||||
per node, and if, by mistake, two or more nodes have the same value for it,
|
||||
then we will not migrate the cache data of that node to a centralized database
|
||||
with logging an appropriate warning message.
|
||||
|
||||
Once we migrate existing cache records to central database we need to
|
||||
ensure that existing cache related command line utilities like cache-cleaner,
|
||||
cache-pruner are working as expected. Since these utilities use dedicated
|
||||
configuration file (glance-cache.conf), we need to make provision to
|
||||
introduce required configuration options in this file so that these tools can
|
||||
connect to central database.
|
||||
|
||||
|
||||
Alternatives
|
||||
------------
|
||||
|
||||
None
|
||||
|
||||
Data model impact
|
||||
-----------------
|
||||
|
||||
Create two new tables in mysql, one to record cache related operations and
|
||||
other to record glance api references as shown below.
|
||||
|
||||
Schema for cached_images::
|
||||
|
||||
CREATE TABLE cached_images (
|
||||
id bigint auto increment PRIMARY KEY,
|
||||
image_id varchar(36),
|
||||
last_accessed DateTime,
|
||||
last_modified DateTime,
|
||||
size bigint,
|
||||
hits int,
|
||||
checksum varchar(32),
|
||||
node_reference_id int,
|
||||
UNIQUE KEY `uq_cache_unique_id` (`image_id`,`node_reference_id`)
|
||||
FOREIGN KEY (node_reference_id) REFERENCES cache_node_reference(node_reference_id)
|
||||
|
||||
)
|
||||
|
||||
Schema for cache_node_reference::
|
||||
|
||||
CREATE TABLE cache_node_reference (
|
||||
node_reference_id int PRIMARY KEY auto increment,
|
||||
node_reference_url varchar(255)
|
||||
UNIQUE KEY `uq_node_reference_url` (`node_reference_url`)
|
||||
)
|
||||
|
||||
REST API impact
|
||||
---------------
|
||||
|
||||
None
|
||||
|
||||
Security impact
|
||||
---------------
|
||||
|
||||
None
|
||||
|
||||
Notifications impact
|
||||
--------------------
|
||||
|
||||
None
|
||||
|
||||
Other end user impact
|
||||
---------------------
|
||||
|
||||
None
|
||||
|
||||
Performance Impact
|
||||
------------------
|
||||
|
||||
None
|
||||
|
||||
Other deployer impact
|
||||
---------------------
|
||||
|
||||
None
|
||||
|
||||
Developer impact
|
||||
----------------
|
||||
|
||||
None
|
||||
|
||||
Implementation
|
||||
==============
|
||||
|
||||
Assignee(s)
|
||||
-----------
|
||||
|
||||
Primary assignee:
|
||||
abhishekk
|
||||
|
||||
Other contributors:
|
||||
None
|
||||
|
||||
Work Items
|
||||
----------
|
||||
|
||||
* Deprecate existing sqlite driver and related configuration options
|
||||
|
||||
* Create alembic scripts for two new tables
|
||||
|
||||
* Add model/queries to sqlalchemy for new tables
|
||||
|
||||
* Introduce new centralized_db driver and related configuration options
|
||||
|
||||
* Change cache code to use new mysql database
|
||||
|
||||
* Add logic at service startup to migrate data from sqlite to mysql
|
||||
|
||||
* Modify related commandline tools like cache-cleaner, cache-pruner
|
||||
to use centralized driver
|
||||
|
||||
* Tempest tests to ensure multiple glance nodes uses centralized
|
||||
database.
|
||||
|
||||
* Grenade test(s) to verify upgrading scenario(s)
|
||||
|
||||
Dependencies
|
||||
============
|
||||
|
||||
None
|
||||
|
||||
Testing
|
||||
=======
|
||||
|
||||
* Unit Tests
|
||||
* Functional Tests
|
||||
* Tempest Tests
|
||||
* grenade tests to verify upgrade scenario
|
||||
|
||||
Documentation Impact
|
||||
====================
|
||||
|
||||
Need to document use of ``worker_self_reference_url`` in case of import workflow
|
||||
and cache related operations.
|
||||
|
||||
References
|
||||
==========
|
||||
|
||||
None
|
||||
@ -6,7 +6,13 @@
|
||||
:glob:
|
||||
:maxdepth: 1
|
||||
|
||||
TODO: fill this in once a new approved spec is added.
|
||||
2024.1 approved specs for glance:
|
||||
|
||||
.. toctree::
|
||||
:glob:
|
||||
:maxdepth: 1
|
||||
|
||||
glance/*
|
||||
|
||||
|
||||
|
||||
|
||||
Loading…
Reference in New Issue
Block a user