recordsansible/ara
Code Issues Proposed changes

container-images: Run SQL migrations automatically at boot

Browse Source 
This removes a necessary step from users when running the container
image.

Change-Id: Ia4d796b77d7db3df93d2c3751533d46f1cb6df13
This commit is contained in:
David Moreau Simard 2020-05-05 13:34:59 -04:00
parent a637534021
commit 3fa950f5e6
No known key found for this signature in database
GPG Key ID: 938880DAFC753E80
5 changed files with 57 additions and 24 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

61
contrib/container-images/README.rst
View File

@ -89,14 +89,9 @@ You can validate if the container is running properly with podman:
$ podman ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
bc4b7630c265 localhost/ara-api:latest /usr/bin/gunicorn... 12 seconds ago Up 11 seconds ago 0.0.0.0:8000->8000/tcp api-server
At this point, the API server is running but if it is your first time launching
it, it will not be able to accept requests until you run initial database
migrations for the default sqlite backend:
.. code-block:: bash
$ podman exec -it api-server ara-manage migrate
$ podman logs api-server
[ara] No setting found for SECRET_KEY. Generating a random key...
[ara] Writing default settings to /opt/ara/settings.yaml
[ara] Using settings file: /opt/ara/settings.yaml
Operations to perform:
Apply all migrations: admin, api, auth, contenttypes, db, sessions
@ -124,9 +119,15 @@ migrations for the default sqlite backend:
Applying auth.0011_update_proxy_permissions... OK
Applying db.0001_initial... OK
Applying sessions.0001_initial... OK
[2020-05-05 17:29:22 +0000] [1] [INFO] Starting gunicorn 20.0.4
[2020-05-05 17:29:22 +0000] [1] [INFO] Listening at: http://0.0.0.0:8000 (1)
[2020-05-05 17:29:22 +0000] [1] [INFO] Using worker: sync
[2020-05-05 17:29:22 +0000] [5] [INFO] Booting worker with pid: 5
[2020-05-05 17:29:22 +0000] [6] [INFO] Booting worker with pid: 6
[2020-05-05 17:29:23 +0000] [7] [INFO] Booting worker with pid: 7
[2020-05-05 17:29:23 +0000] [8] [INFO] Booting worker with pid: 8
Once SQL migrations have been run, the API server should be reachable at
http://127.0.0.1:8000 but it'll be empty.
At this point, the API server will be running but it'll be empty.
Data must be sent to it by running an Ansible playbook with the ARA callback
installed and configured to use this API server.
@ -198,3 +199,43 @@ To connect to database backends other than the sqlite default, edit
- `DATABASE_HOST <https://ara.readthedocs.io/en/latest/api-configuration.html#ara-database-host>`_
- `DATABASE_PORT <https://ara.readthedocs.io/en/latest/api-configuration.html#ara-database-port>`_
- `DATABASE_CONN_MAX_AGE <https://ara.readthedocs.io/en/latest/api-configuration.html#ara-database-conn-max-age>`_
Running SQL migrations
~~~~~~~~~~~~~~~~~~~~~~
The container image will automatically take care of running SQL migrations before
starting.
However, if you need to run them manually, either for a new database or after
an upgrade, the command ``ara-manage migrate`` can be run from inside the container:
.. code-block:: bash
$ podman exec -it api-server ara-manage migrate
[ara] Using settings file: /opt/ara/settings.yaml
Operations to perform:
Apply all migrations: admin, api, auth, contenttypes, db, sessions
Running migrations:
Applying contenttypes.0001_initial... OK
Applying auth.0001_initial... OK
Applying admin.0001_initial... OK
Applying admin.0002_logentry_remove_auto_add... OK
Applying admin.0003_logentry_add_action_flag_choices... OK
Applying api.0001_initial... OK
Applying api.0002_remove_host_alias... OK
Applying api.0003_add_missing_result_properties... OK
Applying api.0004_duration_in_database... OK
Applying api.0005_unique_label_names... OK
Applying contenttypes.0002_remove_content_type_name... OK
Applying auth.0002_alter_permission_name_max_length... OK
Applying auth.0003_alter_user_email_max_length... OK
Applying auth.0004_alter_user_username_opts... OK
Applying auth.0005_alter_user_last_login_null... OK
Applying auth.0006_require_contenttypes_0002... OK
Applying auth.0007_alter_validators_add_error_messages... OK
Applying auth.0008_alter_user_username_max_length... OK
Applying auth.0009_alter_user_last_name_max_length... OK
Applying auth.0010_alter_group_name_max_length... OK
Applying auth.0011_update_proxy_permissions... OK
Applying db.0001_initial... OK
Applying sessions.0001_initial... OK

4
contrib/container-images/fedora-distribution.sh
View File

@ -6,9 +6,9 @@ build=$(buildah from fedora:32)
# This lets users swap easily from the sqlite default to mysql or postgresql just by tweaking settings.yaml.
buildah run "${build}" -- /bin/bash -c "dnf update -y && dnf install -y ara ara-server python3-psycopg2 python3-mysql python3-gunicorn && dnf clean all"
# Set up the container to run the API server with gunicorn and expose the port
# Set up the container to execute SQL migrations and run the API server with gunicorn
buildah config --env ARA_BASE_DIR=/opt/ara "${build}"
buildah config --cmd "/usr/bin/gunicorn-3 --workers=4 --access-logfile - --bind 0.0.0.0:8000 ara.server.wsgi" "${build}"
buildah config --cmd "bash -c '/usr/bin/ara-manage migrate && /usr/bin/gunicorn-3 --workers=4 --access-logfile - --bind 0.0.0.0:8000 ara.server.wsgi'" "${build}"
buildah config --port 8000 "${build}"
# Commit this container to an image name

4
contrib/container-images/fedora-pypi.sh
View File

@ -11,9 +11,9 @@ buildah run "${build}" -- /bin/bash -c "dnf update -y && dnf install -y python3-
# Install ara from source with API server extras for dependencies (django & django-rest-framework)
buildah run "${build}" -- /bin/bash -c "pip3 install ara[server]"
# Set up the container to run the API server with gunicorn and expose the port
# Set up the container to execute SQL migrations and run the API server with gunicorn
buildah config --env ARA_BASE_DIR=/opt/ara "${build}"
buildah config --cmd "/usr/bin/gunicorn-3 --workers=4 --access-logfile - --bind 0.0.0.0:8000 ara.server.wsgi" "${build}"
buildah config --cmd "bash -c '/usr/local/bin/ara-manage migrate && /usr/bin/gunicorn-3 --workers=4 --access-logfile - --bind 0.0.0.0:8000 ara.server.wsgi'" "${build}"
buildah config --port 8000 "${build}"
# Commit this container to an image name

4
contrib/container-images/fedora-source.sh
View File

@ -23,9 +23,9 @@ buildah run "${build}" -- /bin/bash -c "dnf update -y && dnf install -y python3-
# Install ara from source with API server extras for dependencies (django & django-rest-framework)
buildah run --volume ${TMPDIR}/ara:/usr/local/src/ara:z "${build}" -- /bin/bash -c "pip3 install /usr/local/src/ara/${sdist}[server]"
# Set up the container to run the API server with gunicorn and expose the port
# Set up the container to execute SQL migrations and run the API server with gunicorn
buildah config --env ARA_BASE_DIR=/opt/ara "${build}"
buildah config --cmd "/usr/bin/gunicorn-3 --workers=4 --access-logfile - --bind 0.0.0.0:8000 ara.server.wsgi" "${build}"
buildah config --cmd "bash -c '/usr/local/bin/ara-manage migrate && /usr/bin/gunicorn-3 --workers=4 --access-logfile - --bind 0.0.0.0:8000 ara.server.wsgi'" "${build}"
buildah config --port 8000 "${build}"
# Commit this container to an image name

8
tests/container_test_tasks.yaml
View File

@ -13,14 +13,6 @@
--volume {{ ara_api_root_dir }}/server:/opt/ara:z -p 8000:8000 \
{{ item.name }}:{{ item.tag }}
- name: Run SQL migrations
command: podman exec -it api-server ara-manage migrate
register: _sql_migrations
# Allow the container to settle from booting up
retries: 3
delay: 5
until: _sql_migrations is succeeded
- block:
- name: Get the API root
uri: