Document tips on debugging translation job scripts
The need for this information is rare but when I develop the translation job scripts I always research the same thing so it is useful at least for me. Change-Id: I83f38fd673456e0d1179c50d9df903e16b01bd5f
This commit is contained in:
Akihiro Motoki
parent
80e0fee56b
commit
78a21fa830
@ -35,6 +35,7 @@ Contents
|
||||
tools
|
||||
i18n_team_meeting
|
||||
infra
|
||||
infra-debug
|
||||
project_setup
|
||||
reviewing-translation-import
|
||||
release_management
|
||||
|
||||
152
doc/source/infra-debug.rst
Normal file
152
doc/source/infra-debug.rst
Normal file
@ -0,0 +1,152 @@
|
||||
=====================
|
||||
Debugging job scripts
|
||||
=====================
|
||||
|
||||
While it would be rare, you may would like to debug the translation
|
||||
job scripts. This section describes tips for such cases.
|
||||
``propose_translation_update.sh`` is mainly covered.
|
||||
|
||||
.. note::
|
||||
|
||||
It is not a complete document for debugging.
|
||||
It was just written as a note. Feel free to add more topics.
|
||||
|
||||
Environment
|
||||
-----------
|
||||
|
||||
It is better to use the same environment (e.g., Linux distribution version)
|
||||
used in the translation job. To find the information, check ``nodeset`` of the
|
||||
``propose-translation-update`` job definition in
|
||||
`zuul configuration <https://opendev.org/openstack/project-config/src/branch/master/zuul.d/jobs.yaml>`__
|
||||
in openstack/project-config repository.
|
||||
As of Feb 2021, ``ubuntu-bionic`` is used.
|
||||
|
||||
Preparations
|
||||
------------
|
||||
|
||||
You can prepare dependencies by following
|
||||
`prepare-zanata-client role <https://opendev.org/openstack/openstack-zuul-jobs/src/branch/master/roles/prepare-zanata-client/tasks/main.yaml>`__.
|
||||
|
||||
What you need are:
|
||||
|
||||
* Installing Zanata CLI and its dependencies.
|
||||
Ensure ``zanata-cli`` command is found in your path.
|
||||
* Prepare ``~/.config/zanata.ini``.
|
||||
See :ref:`Zanata CLI <zanata-cli>` for more detail.
|
||||
* Copy the translation scripts to your working directory from
|
||||
`openstack-zuul-jobs/roles/prepare-zanata-client/files <https://opendev.org/openstack/openstack-zuul-jobs/src/branch/master/roles/prepare-zanata-client/files>`__.
|
||||
In addition, you need to copy ``common.sh`` from
|
||||
`project-config/roles/copy-proposal-common-scripts/files <https://opendev.org/openstack/project-config/src/branch/master/roles/copy-proposal-common-scripts/files>`__
|
||||
to the same working directory.
|
||||
|
||||
Commenting out CI-specific codes
|
||||
--------------------------------
|
||||
|
||||
The job scripts contain codes specific to OpenStack CI.
|
||||
For example, there is a code to communicate with OpenStack gerrit
|
||||
and the account is hardcoded.
|
||||
It looks convenient to comment out such code blocks
|
||||
to debug the scripts in a local env. It should be useful
|
||||
unless you are debuging a CI-specific issue.
|
||||
|
||||
Code blocks which look better to be commented out are:
|
||||
|
||||
* common_translation_update.sh
|
||||
|
||||
* ``trap "finish" EXIT`` (It depends on testrepository and usually fails in
|
||||
local envs)
|
||||
|
||||
* propose_translation_update.sh
|
||||
|
||||
* ``setup_review`` and a logic to check an existing change
|
||||
(It is unnecessary as we do not propose a real change to gerrit.
|
||||
The account used is hardcoded and it always fails in local runs.)
|
||||
* ``send_patch`` (It is unnecessary as we do not want to propose
|
||||
a real change to gerrit.)
|
||||
* (optional)
|
||||
The main logic (from ``case "$PROJECT" in`` to ``filter_commits``)
|
||||
(we will debug the main logic, so perhaps we would like to run the script
|
||||
piece by piece.)
|
||||
|
||||
The diff would be like http://paste.openstack.org/show/802260/.
|
||||
|
||||
Copying upper-constraints.txt
|
||||
-----------------------------
|
||||
|
||||
The job scripts assume that upper-constraints.txt exists in the top directory of
|
||||
the target project repository.
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
$ cd $PROJECT_DIR
|
||||
$ wget https://releases.openstack.org/constraints/upper/master
|
||||
|
||||
Note that you need to adjust the URL of the upper-constraints file
|
||||
when you work on a stable branch.
|
||||
|
||||
Creating ~/.venv virtualenv
|
||||
---------------------------
|
||||
|
||||
The job scripts assume that required python modules are installed in
|
||||
``~/.venv`` virtualenv. In the zuul job, this virtualenv is prepared via
|
||||
``ensure-sphinx`` and ``ensure-babel`` roles in zuul/zuul-jobs.
|
||||
You need to do the same.
|
||||
|
||||
Note that doc/requirements.txt in most projects are almost samer so perhaps you
|
||||
can reuse this virtualenv for most projects. If it does not work, consider
|
||||
recreating the virtualenv.
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
$ python3 -m venv ~/.venv
|
||||
$ . ~/.venv/bin/activate
|
||||
# $PROJECT_DIR is a target project repository like horizon, ironic-ui ....
|
||||
(.venv) $ cd $PROJECT_DIR
|
||||
# Install sphinx related modules ensure-sphinx installs
|
||||
(.venv) $ pip install -r doc/requirements.txt -c ../requirements/upper-constraints.txt
|
||||
# Install modules ensure-babel installs
|
||||
(.venv) $ pip install Babel lxml pbr requests -c ../requirements/upper-constraints.txt
|
||||
(.venv) $ deactivate
|
||||
|
||||
Loading scripts
|
||||
---------------
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
# It is better to start a new shell when debugging the script
|
||||
$ bash
|
||||
> cd $PROJECT_REPO
|
||||
# The arguments are: {{ zuul.project.short_name }} {{ zuul.branch }} {{ zuul.job }} $HORIZON_REPO
|
||||
> . $WORKDIR/propose_translation_update.sh ironic-ui master propose-translation-update ../horizon
|
||||
>
|
||||
|
||||
Simulating translation changes
|
||||
------------------------------
|
||||
|
||||
If you would like to simulate translation changes, you can download translations
|
||||
in advance (using zanata-cli) and modify them as you want.
|
||||
In such case, you can modify the script as follows:
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
--- a/roles/prepare-zanata-client/files/common_translation_update.sh
|
||||
+++ b/roles/prepare-zanata-client/files/common_translation_update.sh
|
||||
@@ -734,7 +732,8 @@ function pull_from_zanata {
|
||||
# Since Zanata does not currently have an option to not download new
|
||||
# files, we download everything, and then remove new files that are not
|
||||
# translated enough.
|
||||
- zanata-cli -B -e pull
|
||||
+ #zanata-cli -B -e pull
|
||||
+ cp -r $DOWNLOAD_TRANSLATIONS/$project/* .
|
||||
|
||||
# We skip directories starting with '.' because they never contain
|
||||
# translations for the project (in particular, '.tox'). Likewise
|
||||
|
||||
You can download translations as below.
|
||||
``zanata.xml`` will be created once you run the propose_translation_update.sh
|
||||
(or you can prepare it by following :ref:`zanata-cli` "Project configuration").
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
$ cd $DOWNLOAD_TRANSLATIONS/$project
|
||||
$ zanata-cli -B -e pull --project-config $PROJECT_DIR/zanata.xml
|
||||
@ -95,3 +95,8 @@ repository.
|
||||
|
||||
Note that the scripts in the tasks use `zanata-cli <http://docs.zanata.org/en/release/client/>`__
|
||||
to pull and push translation content.
|
||||
|
||||
Debugging job scripts
|
||||
---------------------
|
||||
|
||||
See :doc:`infra-debug`.
|
||||
|
||||
Loading…
Reference in New Issue
Block a user