openstack/tripleo-repos
Code Issues Proposed changes

Retire Tripleo: remove repo content

Browse Source 
TripleO project is retiring
- https://review.opendev.org/c/openstack/governance/+/905145

this commit remove the content of this project repo

Change-Id: I0b5c328f22d893f7d4e73935b31281f0f9826cd4
This commit is contained in:
Ghanshyam Mann 2024-02-24 11:33:44 -08:00
parent c73cb0f636
commit fd05df0293
74 changed files with 8 additions and 6559 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

25
.gitignore vendored
View File

@ -1,25 +0,0 @@
*.py[cod]
# Unit test / coverage reports
.coverage
cover
.tox
.testrepository
.stestr/
# Packages
.eggs
*.egg*
dist
build
eggs
parts
sdist
develop-eggs
.installed.cfg
lib
lib64
# built ansible collection
*.tar.gz

3
.stestr.conf
View File

@ -1,3 +0,0 @@
[DEFAULT]
test_path=./tests/unit
top_dir=./

17
CONTRIBUTING.rst
View File

@ -1,17 +0,0 @@
If you would like to contribute to the development of OpenStack, you must
follow the steps in this page:
http://docs.openstack.org/infra/manual/developers.html
If you already have a good understanding of how the system works and your
OpenStack accounts are set up, you can skip to the development workflow
section of this documentation to learn how changes to OpenStack should be
submitted for review via the Gerrit tool:
http://docs.openstack.org/infra/manual/developers.html#development-workflow
Pull requests submitted through GitHub will be ignored.
Bugs should be filed on Launchpad, not GitHub:
https://bugs.launchpad.net/tripleo

175
LICENSE
View File

@ -1,175 +0,0 @@
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.

3
MANIFEST.in
View File

@ -1,3 +0,0 @@
global-exclude *.py[cod]
global-exclude __pycache__
recursive-exclude plugins *

1
README.md
View File

@ -1 +0,0 @@
# tripleo.repos collection

76
README.rst
View File

@ -1,70 +1,10 @@
tripleo-repos
=============
This project is no longer maintained.
A tool for managing tripleo repos from places like RDO Trunk and Ceph.
The contents of this repository are still available in the Git
source code management system. To see the contents of this
repository before it reached its end of life, please check out the
previous commit with "git checkout HEAD^1".
See: https://blogs.rdoproject.org/2016/04/newbie-in-rdo-2-rdo-trunk-from-a-bird-s-eye-view/
Also ensures yum-plugin-priorities is installed since the RDO Trunk repos
require that to work sanely.
.. note:: The tool will remove any delorean* repos at the target location
to avoid conflicts with older repos. This means you must specify
all of the repos you want to enable in one tripleo-repos call.
Examples
--------
Install TripleO CI testing repos for UBI-8 by the distro specific path::
tripleo-repos -d ubi8 tripleo-ci-testing --output-path /etc/distro.repos.d
Install current master RDO Trunk repo and the deps repo::
tripleo-repos current
Install current-tripleo RDO Trunk repo and the deps repo::
tripleo-repos current-tripleo
Install the current-tripleo-dev repo. This will also pull current and deps,
and will adjust the priorities of each repo appropriately::
tripleo-repos current-tripleo-dev
Install the mitaka RDO Trunk repo and deps::
tripleo-repos -b mitaka current
Write repos to a different path::
tripleo-repos -o ~/test-repos current
Install the current-tripleo, deps, and ceph repos. NOTE: The Ceph repo is
installed from a package and thus does not respect -o::
tripleo-repos current-tripleo ceph
TripleO
```````
To use this for TripleO development, replace the tripleo.sh --repo-setup
step with the following::
git clone https://github.com/cybertron/tripleo-repos
cd tripleo-repos
sudo ./setup.py install
sudo tripleo-repos current-tripleo-dev ceph
Now you're ready to install the undercloud::
tripleo.sh --undercloud
And to build images::
export OVERCLOUD_IMAGES_DIB_YUM_REPO_CONF="$(ls /etc/yum.repos.d/delorean* /etc/yum.repos.d/CentOS-Ceph-*)"
tripleo.sh --overcloud-images
.. note:: This is a tool for bootstrapping the repo setup for TripleO,
so it should not have any runtime OpenStack dependencies
or we end up in a chicken-and-egg pickle, and let's be honest - no one wants a
chicken and egg pickle.
For any further questions, please email
openstack-discuss@lists.openstack.org or join #openstack-dev on
OFTC.

4
ansible.cfg
View File

@ -1,4 +0,0 @@
# Used for testing to avoid using one from outside repository
[defaults]
# Avoid noise which can make debugging test failures harder
deprecation_warnings=False

3
doc/requirements.txt
View File

@ -1,3 +0,0 @@
sphinx>=1.8.0,!=2.1.0 # BSD
openstackdocstheme>=2.0.0 # Apache-2.0

80
doc/source/conf.py
View File

@ -1,80 +0,0 @@
# -*- coding: utf-8 -*-
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import os
import sys
sys.path.insert(0, os.path.abspath('../..'))
# -- General configuration ----------------------------------------------------
# Add any Sphinx extension module names here, as strings. They can be
# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom ones.
extensions = [
'sphinx.ext.autodoc',
#'sphinx.ext.intersphinx',
'openstackdocstheme'
]
# autodoc generation is a bit aggressive and a nuisance when doing heavy
# text edit cycles.
# execute "export SPHINX_DEBUG=1" in your terminal to disable
# The suffix of source filenames.
source_suffix = '.rst'
# The master toctree document.
master_doc = 'index'
# General information about the project.
project = u'tripleo-repos'
copyright = u'2017, OpenStack Foundation'
# If true, '()' will be appended to :func: etc. cross-reference text.
add_function_parentheses = True
# If true, the current module name will be prepended to all description
# unit titles (such as .. function::).
add_module_names = True
# The name of the Pygments (syntax highlighting) style to use.
pygments_style = 'sphinx'
html_theme = 'openstackdocs'
repository_name = 'openstack/tripleo-repos'
bug_project = 'tripleo'
# -- Options for HTML output --------------------------------------------------
# The theme to use for HTML and HTML Help pages. Major themes that come with
# Sphinx are currently 'default' and 'sphinxdoc'.
# html_theme_path = ["."]
# html_theme = '_theme'
# html_static_path = ['static']
# Output file base name for HTML help builder.
htmlhelp_basename = '%sdoc' % project
# Grouping the document tree into LaTeX files. List of tuples
# (source start file, target name, title, author, documentclass
# [howto/manual]).
latex_documents = [
('index',
'%s.tex' % project,
u'%s Documentation' % project,
u'OpenStack Foundation', 'manual'),
]
# Example configuration for intersphinx: refer to the Python standard library.
#intersphinx_mapping = {'http://docs.python.org/': None}

4
doc/source/contributing.rst
View File

@ -1,4 +0,0 @@
============
Contributing
============
.. include:: ../../CONTRIBUTING.rst

25
doc/source/index.rst
View File

@ -1,25 +0,0 @@
.. tripleo-repos documentation master file, created by
sphinx-quickstart on Tue Jul 9 22:26:36 2013.
You can adapt this file completely to your liking, but it should at least
contain the root `toctree` directive.
Welcome to tripleo-repos's documentation!
=========================================
Contents:
.. toctree::
:maxdepth: 2
readme
installation
usage
contributing
Indices and tables
==================
* :ref:`genindex`
* :ref:`modindex`
* :ref:`search`

12
doc/source/installation.rst
View File

@ -1,12 +0,0 @@
============
Installation
============
At the command line::
$ pip install tripleo-repos
Or, if you have virtualenvwrapper installed::
$ mkvirtualenv tripleo-repos
$ pip install tripleo-repos

1
doc/source/readme.rst
View File

@ -1 +0,0 @@
.. include:: ../../README.rst

3
doc/source/usage.rst
View File

@ -1,3 +0,0 @@
=====
Usage
=====

86
docs/get_hash.md
View File

@ -1,86 +0,0 @@
# tripleo.repos.get_hash
## What is tripleo.repos.get_hash
This utility is meant for use by TripleO deployments, particularly in zuul
continuous integration jobs. Given an [RDO named tag](https://docs.openstack.org/tripleo-docs/latest/ci/stages-overview.html#rdo-dlrn-promotion-criteria),
such as 'current-tripleo' or 'tripleo-ci-testing' it will return the hash
information, including the commit, distro and full hashes where available.
It includes a simple command line interface. If you clone the source you can
try it out of the box without installation invoking it as a module:
```
tripleo-get-hash # by default centos8, master, current-tripleo.
tripleo-get-hash --component tripleo --release victoria --os-version centos8
tripleo-get-hash --release master --os-version centos7
tripleo-get-hash --release train # by default centos8
tripleo-get-hash --os-version rhel8 --release osp16-2 --dlrn-url http://osp-trunk.hosted.upshift.rdu2.redhat.com
tripleo-get-hash --help
```
## Quick start
#### Install using setup.py
It is recommended to perform a user/local installation using python setup.py
to avoid the use of sudo. However you may need to set your PYTHONPATH depending
on where the python code is installed on your system.
```
python setup.py install --user
tripleo-get-hash --help
```
The tripleo-get-hash utility uses a yaml configuration file named 'config.yaml'.
If you install this utility using --user as above, the configuration file
is placed in $HOME/.local/etc/tripleo_get_hash/config.yaml (on fedora).
If this cannot be found then the config is used directly from the source directory.
When you invoke tripleo-get-hash it will tell you which config is in use:
```
$ tripleo-get-hash
2021-10-15 16:22:23,724 - tripleo-get-hash - INFO - Using config file at /home/username/.local/etc/tripleo_get_hash/config.yaml
```
#### Install using pip
You can also install using python pip - you can see the
[tripleo-get-hash module here](https://pypi.org/project/tripleo-repos/)
```
pip install tripleo-repos --user
```
After installation you can invoke tripleo-get-hash --help to see the various
options:
```
tripleo-get-hash --help
```
By default this queries the delorean server at "https://trunk.rdoproject.org",
with this URL specified in config.yaml. To use a different delorean server you
can either update config.yaml or use the --dlrn-url parameter to the cli. If
instead you are instantiating TripleOHashInfo objects in code, you can create
the objects passing an existing 'config' dictionary. Note this has to contain
all of constants.CONFIG_KEYS to avoid explosions.
## Ansible Module
It is required that you install `tripleo.repos` collection to use the ansible
module.
See the [example playbook](https://opendev.org/openstack/tripleo-repos/src/branch/master/playbooks/example_get_hash.yaml) included here for examples of
usage. You can also test the ansible module is available and working correctly
from using shell:
```
$ ansible localhost -m tripleo.repos.get_hash -a "component=compute release=victoria"
localhost | SUCCESS => {
"changed": false,
"commit_hash": "e954a56fec69637ebd671643d41bb0ecc85a2656",
"distro_hash": "de7baf4889fba4d42ac39c9e912c42e38abb5193",
"dlrn_url": "https://trunk.rdoproject.org/centos8-victoria/component/compute/current-tripleo/commit.yaml",
"error": "",
"extended_hash": "None",
"full_hash": "e954a56fec69637ebd671643d41bb0ecc85a2656_de7baf48",
"success": true
}
```

107
docs/yum_config.md
View File

@ -1,107 +0,0 @@
# tripleo.repos.yum_config
*tripleo-yum-config* utility was designed to simplify the way that TripleO
deployments manage their yum configuration. This tool helps on updating
specific configuration options for different yum configuration files like yum
repos, yum modules and yum global configuration file.
## Quick start
### Using as a python module
It is possible to use *tripleo-yum-config* as a standalone module by cloning
its repository and invoking in command line:
* **repo**
This subcommand lets you enable or disable a repo and sets its configuration options.
The *tripleo-yum-config* module will search for the provided repo name in all *.repo* files at REPO_DIR_PATH.
Optionally, you can provide a path where your repo files live or specify the full path of the repo file.
By default REPO_DIR_PATH is set to */etc/yum.repos.d/*.
Examples:
```
sudo python -m tripleo_yum_config repo --name appstream --enable --set-opts baseurl=http://newbaseurl exclude="package*"
sudo python -m tripleo_yum_config repo --name epel --disable --config-dir-path=/path/to/yum.repos.d
```
The parameter *--down-url* can be used to retrieve a configuration file from a URL and populate the destination
configuration file with all its content. When used together with *--name*, only the requested repo name will be
updated in the process.
Examples:
```
sudo python -m tripleo_yum_config repo --down-url http://remoterepofile.repo --enable --set-opts priority=20 --config-file-path=/path/to/file.repo
sudo python -m tripleo_yum_config repo --name appstream --down-url http://remoterepofile.repo --enable
```
* **module**
This subcommand lets you enable, disable, remove, install or reset a module.
Depending on the selected operation and module, the optional parameters 'stream' or 'profile' will also need to be provided:
1. when enabling a module, the *stream* version will be required if the module has zero or more than one default stream.
2. when installing a module, the *profile* will be required if the enabled stream has no default profile set.
Examples:
```
sudo tripleo-yum-config module remove tomcat
sudo tripleo-yum-config module disable tomcat
sudo tripleo-yum-config module enable nginx --stream mainline
sudo tripleo-yum-config module install nginx --profile common
```
* **global**
This subcommand lets you set any global yum/dnf configuration value under *[main]* section.
If no configuration file is found by the module, a new one is created and populated.
Optionally you can also provide the path to the configuration file.
By default CONFIG_FILE_PATH is set to */etc/yum.conf*
Example:
```
sudo python -m tripleo_yum_config global --set-opts keepcache=1 cachedir="/var/cache/dnf"
```
* **enable-compose-repos**
This subcommand will enable a list os yum repos based on the metadata retrieved from the `compose-url`.
The *tripleo-yum-config* module will create new repo files at REPO_DIR_PATH and enable them.
Optionally, you can provide a path where your repo files live, specify the variants that should be created and which repos need to be disabled afterwards.
By default REPO_DIR_PATH is set to */etc/yum.repos.d/*.
Example:
```
sudo python -m tripleo_yum_config enable-compose-repos --compose-url https://composes.centos.org/latest-CentOS-Stream-8/compose/ --release centos-stream-8 --disable-all-conflicting
```
#### Install using setup.py
Installation using python setup.py requires sudo, because the python source
is installed at /usr/local/lib/python.
```
sudo python setup.py install
```
#### Install using pip
Alternatively you can install tripleo-yum-config with python pip:
```
pip install tripleo-repos --user
```
See PyPI [tripleo-repos](https://pypi.org/project/tripleo-repos/)
project for more details.
## Usage
The utility provides a command line interface with various options. You can
invoke *tripleo-yum-config --help* to see all the available commands.
```
tripleo-yum-config --help
```
## Ansible Module
It is required that you install `tripleo.repos` collection to use the ansible
module.
An ansible module [tripleo.repos.yum_config](https://opendev.org/openstack/tripleo-repos/src/branch/master/modules/module/modules/yum_config.py)
is available for you when you install `tripleo.repos` collection.
An [example playbook](https://opendev.org/openstack/tripleo-repos/src/branch/master/playbooks/example_yum_config.yaml)
is available to assist with module usage.

39
galaxy.yml
View File

@ -1,39 +0,0 @@
name: repos
namespace: tripleo
version: 0.0.5
readme: README.md
authors:
- Red Hat
description: TripleO Repos
build_ignore:
- "*.egg-info"
- .DS_Store
- .eggs
- .gitignore
- .gitreview
- .mypy_cache
- .pytest_cache
- .stestr
- .stestr.conf
- .tox
- .vscode
- MANIFEST.in
- build
- dist
- doc
- report.html
- setup.cfg
- setup.py
- "tests/unit/*.*"
- README.rst
- tox.ini
- tripleo_repos
- zuul.d
# excluded because galaxy server refuses uploads with __main___ inside
- plugins/module_utils/tripleo_repos/get_hash/__main__.py
- plugins/module_utils/tripleo_repos/yum_config/__main__.py
# that is not needed for ansible modules and it would upset sanity (pylint)
- plugins/module_utils/tripleo_repos/main.py
repository: https://opendev.org/openstack/tripleo-repos
license_file: LICENSE

1
meta/runtime.yml
View File

@ -1 +0,0 @@
requires_ansible: ">=2.9.0"

23
molecule/default/assert_ini_key_value.yml
View File

@ -1,23 +0,0 @@
---
- name: Get the content of {{ item.name }} repo
command: "cat {{ item.path }}"
register: file_output
- name: Print {{ item.name }} repo content
debug:
msg:
- "Content of {{ item.name }} repo located at '{{ item.path }}'"
- "{{ file_output.stdout_lines }}"
- set_fact:
tmp_repo_file: /tmp/{{ item.name|lower }}.temp
- name: Retrieve remote repo ini file
fetch:
src: "{{ item.path }}"
dest: "{{ tmp_repo_file }}"
flat: yes
- assert:
that:
- "{{ lookup('ini', '{{ item.key }} section={{ item.section|lower }} file={{ tmp_repo_file }}') }} == {{ item.value }}"

86
molecule/default/converge.yml
View File

@ -1,86 +0,0 @@
---
- name: Converge
hosts: all
tasks:
- name: "Check get_hash"
tripleo.repos.get_hash:
release: master
- name: "Check get_hash with invalid url"
tripleo.repos.get_hash:
release: master
dlrn_url: 'https://httpbin.org/status/404'
register: result
failed_when: result is success
- name: "Test disable system repo"
become: true
tripleo.repos.yum_config:
type: repo
name: "{{ 'rt' if (ansible_distribution_major_version is version(8, '>=')) else 'cr' }}"
enabled: false
tags:
# TODO: fix yum_config to correctly report changed state and uncomment
# the line below which disables molecule idempotence test.
- molecule-idempotence-notest
- name: "Test create new repo file"
become: true
tripleo.repos.yum_config:
type: repo
name: "fakerepo"
# Keep it disabled to not affect any other test
enabled: false
file_path: "/etc/yum.repos.d/fake_repo.repo"
set_options:
baseurl: "http://fakemirror/fakerepo"
priority: "10"
gpgcheck: "0"
exclude: "fakepkg*"
tags:
# TODO: fix yum_config to correctly report changed state and uncomment
# the line below which disables molecule idempotence test.
- molecule-idempotence-notest
- name: "Test yum-config global config"
become: true
tripleo.repos.yum_config:
type: global
file_path: "{{ '/etc/dnf/dnf.conf' if (ansible_distribution_major_version is version(8, '>=')) else '/etc/yum.conf' }}"
set_options:
skip_if_unavailable: "False"
fake_conf: "True"
tags:
# TODO: fix yum_config to correctly report changed state and uncomment
# the line below which disables molecule idempotence test.
- molecule-idempotence-notest
- name: "Test yum_config enable-compose-repos"
become: true
tripleo.repos.yum_config:
type: enable-compose-repos
compose_url: https://composes.centos.org/latest-CentOS-Stream-8/compose/
centos_release: centos-stream-8
variants:
- AppStream
- BaseOS
disable_repos:
- /etc/yum.repos.d/CentOS-Stream-AppStream.repo
- /etc/yum.repos.d/CentOS-Stream-BaseOS.repo
tags:
- molecule-idempotence-notest
# NOTE: operation available only for CentOS >= 8
when: ansible_distribution_major_version is version(8, '>=')
- name: "Test create repo from repo file"
become: true
tripleo.repos.yum_config:
type: repo
enabled: true
file_path: "/etc/yum.repos.d/delorean.repo"
down_url: "https://trunk.rdoproject.org/centos8-master/current-tripleo/delorean.repo"
set_options:
priority: "20"
tags:
# TODO: fix yum_config to correctly report changed state and uncomment
# the line below which disables molecule idempotence test.
- molecule-idempotence-notest

14
molecule/default/molecule.yml
View File

@ -1,14 +0,0 @@
---
dependency:
name: galaxy
driver:
name: docker
platforms:
- name: instance
image: quay.io/centos/centos:stream8
- name: instance_c7
image: quay.io/centos/centos:centos7
provisioner:
name: ansible
verifier:
name: ansible

7
molecule/default/prepare.yml
View File

@ -1,7 +0,0 @@
- hosts: instance_c7
tasks:
- name: Update ca-certificates
become: true
package:
name: ca-certificates
state: latest

47
molecule/default/verify.yml
View File

@ -1,47 +0,0 @@
---
- name: Verify
hosts: all
tasks:
- name: Check if RT or CR repos are disabled
vars:
section_name: "{{ 'rt' if (ansible_distribution_major_version is version(8, '>=')) else 'cr' }}"
repo_path: /etc/yum.repos.d/CentOS-{{ 'Stream-RealTime' if (ansible_distribution_major_version is version(8, '>=')) else 'CR' }}.repo
include_tasks: assert_ini_key_value.yml
with_items:
- name: "{{ section_name|upper }}"
path: "{{ repo_path }}"
section: "{{ section_name }}"
key: enabled
value: "0"
- name: Check if yum/dnf conf file was updated
vars:
conf_file_path: "{{ '/etc/dnf/dnf.conf' if (ansible_distribution_major_version is version(8, '>=')) else '/etc/yum.conf' }}"
include_tasks: assert_ini_key_value.yml
with_items:
- name: global_conf
path: "{{ conf_file_path }}"
section: main
key: skip_if_unavailable
value: "False"
- name: global_conf
path: "{{ conf_file_path }}"
section: main
key: fake_conf
value: "True"
- name: Validate compose repos outputs
include_tasks: verify_compose_repos.yml
with_items:
- "AppStream"
- "BaseOS"
# NOTE: operation available only for CentOS >= 8
when: ansible_distribution_major_version is version(8, '>=')
- name: Check if 'priority' was set to 20 in 'delorean-component-compute'
include_tasks: assert_ini_key_value.yml
with_items:
- name: "delorean-component-compute"
path: "/etc/yum.repos.d/delorean.repo"
section: "delorean-component-compute"
key: priority
value: "20"

26
molecule/default/verify_compose_repos.yml
View File

@ -1,26 +0,0 @@
---
- name: "Search for {{ item }} repos"
find:
paths: "/etc/yum.repos.d"
file_type: file
use_regex: yes
patterns:
- "^CentOS-Stream.*{{ item }}.*.repo$"
excludes:
- "CentOS-Stream-{{ item }}.repo"
register: compose_repos
failed_when: compose_repos.files|length != 1
- name: Validate repo file configuration
include_tasks: assert_ini_key_value.yml
with_items:
- name: "{{ item }}"
path: "{{ compose_repos.files[0].path }}"
section: "{{ item|lower }}"
key: enabled
value: "1"
- name: "{{ item }}"
path: "/etc/yum.repos.d/CentOS-Stream-{{ item }}.repo"
section: "{{ item|lower }}"
key: enabled
value: "0"

40
playbooks/example_get_hash.yaml
View File

@ -1,40 +0,0 @@
---
- name: Example usage for tripleo-get-hash python module
hosts: localhost
tasks:
- name: get component-ci-testing for victoria compute component
tripleo.repos.get_hash:
os_version: centos8 # default: centos8
release: victoria # default: master
component: compute # default: None
tag: component-ci-testing # default: current-tripleo
register: component_ci_testing_victoria_compute
- debug:
msg: "Centos8 component-ci-testing victoria compute component: {{ component_ci_testing_victoria_compute['full_hash'] }}"
- debug:
var: component_ci_testing_victoria_compute
- name: get centos7 tripleo-ci-testing for train
tripleo.repos.get_hash:
os_version: centos7
release: train
tag: tripleo-ci-testing
register: centos7_tripleo_ci_testing_train
- debug:
msg: "Centos7 current-tripleo train: {{ centos7_tripleo_ci_testing_train['full_hash'] }}"
- debug:
var: centos7_tripleo_ci_testing_train
- name: get current-tripleo centos8 for master branch
tripleo.repos.get_hash:
register: centos8_current_tripleo_master
- debug:
msg: "Centos8 current-tripleo master: {{ centos8_current_tripleo_master['full_hash'] }}"
- debug:
var: centos8_current_tripleo_master

33
playbooks/example_yum_config.yaml
View File

@ -1,33 +0,0 @@
---
- name: Example usage for tripleo.repos.yum_config ansible module
hosts: localhost
tasks:
- name: Enable appstream yum repo and set exclude packages
become: true
tripleo.repos.yum_config:
type: repo
name: appstream
enabled: true
set_options:
exclude:
- nodejs*
- mariadb*
- name: Enable and install nginx module
become: true
tripleo.repos.yum_config:
type: module
name: nginx
enabled: true
operation: install
profile: common
stream: mainline
- name: Set yum global options in dnf.conf
become: true
tripleo.repos.yum_config:
type: global
file_path: /etc/dnf/dnf.conf
set_options:
skip_if_unavailable: "False"
keepcache: "0"

0
plugins/module_utils/tripleo_repos/__init__.py
View File

0
plugins/module_utils/tripleo_repos/get_hash/__init__.py
View File

114
plugins/module_utils/tripleo_repos/get_hash/__main__.py
View File

@ -1,114 +0,0 @@
# Copyright 2021 Red Hat, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
#
#
import argparse
import logging
import sys
from tripleo_repos.utils import load_logging
from tripleo_repos.get_hash.tripleo_hash_info import TripleOHashInfo
import tripleo_repos.get_hash.exceptions as exc
def _validate_args(parsed_args):
if parsed_args.os_version == 'centos7' and (
parsed_args.component is not None
):
raise exc.TripleOHashInvalidParameter(
'Cannot specify component for centos 7'
)
def main():
load_logging(module_name="tripleo-get-hash")
config = TripleOHashInfo.load_config()
parser = argparse.ArgumentParser(description='tripleo-get-hash.py')
parser.add_argument(
'--component',
help=('Use this to specify a component '
'This is NOT valid for Centos 7.'),
choices=config['tripleo_ci_components'],
)
parser.add_argument(
'--dlrn-url',
help=(
'The URL for the delorean server to use. Defaults to '
'https://trunk.rdoproject.org'
),
)
parser.add_argument(
'--os-version',
default='centos8',
choices=config['os_versions'],
help=('The operating system and version to fetch the build tag for'),
)
parser.add_argument(
'--tag',
default='current-tripleo',
choices=config['rdo_named_tags'],
help=('The known tag to retrieve the hash_info for'),
)
parser.add_argument(
'--release',
default='master',
help=('The release of OpenStack you want the hash info for. '
'Default master'),
choices=config['tripleo_releases'],
)
parser.add_argument(
'--verbose',
action='store_true',
help=('Enable verbose log level for debugging'),
)
args = parser.parse_args()
if args.verbose:
logging.getLogger().setLevel(logging.DEBUG)
logging.debug("Logging level set to DEBUG")
_validate_args(args)
if args.dlrn_url is not None:
logging.debug(
"Overriding configuration dlrn_url. Original value {}. "
"New value {}".format(config['dlrn_url'], args.dlrn_url)
)
config['dlrn_url'] = args.dlrn_url
logging.debug(
"Proceeding with the following configuration: {}".format(config)
)
tripleo_hash_info = TripleOHashInfo(
args.os_version,
args.release,
args.component,
args.tag,
config,
)
print(tripleo_hash_info)
return tripleo_hash_info
def cli_entrypoint():
try:
main()
sys.exit(0)
except KeyboardInterrupt:
logging.info("Exiting on user interrupt")
raise
if __name__ == "__main__":
main()

53
plugins/module_utils/tripleo_repos/get_hash/config.yaml
View File

@ -1,53 +0,0 @@
# This file is installed to the path in [options.data_files] of the project
# setup.cfg file. It *must* contain all the keys specified in constants
# CONFIG_KEYS or there will be explosions.
dlrn_url: 'https://trunk.rdoproject.org'
tripleo_releases:
- master
- zed
- wallaby
- victoria
- ussuri
- train
- stein
- queens
- osp16-2
- osp17
- osp17-1
- osp18
tripleo_ci_components:
- baremetal
- cinder
- clients
- cloudops
- common
- compute
- glance
- manila
- network
- octavia
- security
- swift
- tempest
- tripleo
- ui
- validation
rdo_named_tags:
- current
- consistent
- component-ci-testing
- promoted-components
- tripleo-ci-testing
- current-tripleo
- current-tripleo-rdo
os_versions:
- centos7
- centos8
- centos9
- rhel8
- rhel9

90
plugins/module_utils/tripleo_repos/get_hash/constants.py
View File

@ -1,90 +0,0 @@
# Copyright 2021 Red Hat, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
#
#
from __future__ import (absolute_import, division, print_function)
__metaclass__ = type
"""
These are the keys we expect to find in a well-formed config.yaml
If any keys are missing from the configuration hash resolution doesn't proceed.
"""
CONFIG_KEYS = [
'dlrn_url',
'tripleo_releases',
'tripleo_ci_components',
'rdo_named_tags',
'os_versions',
]
"""
This is the path that we expect to find the system installed config.yaml.
The path is specified in [options.data_files] of the project setup.cfg.
"""
CONFIG_PATH = '/usr/local/etc/tripleo_get_hash/config.yaml'
DEFAULT_CONFIG = {
"tripleo_releases": [
"master",
"zed",
"wallaby",
"victoria",
"ussuri",
"train",
"stein",
"queens",
"osp16-2",
"osp17",
"osp17-1",
"osp18"
],
"dlrn_url": "https://trunk.rdoproject.org",
"rdo_named_tags": [
"current",
"consistent",
"component-ci-testing",
"promoted-components",
"tripleo-ci-testing",
"current-tripleo",
"current-tripleo-rdo"
],
"tripleo_ci_components": [
"baremetal",
"cinder",
"clients",
"cloudops",
"common",
"compute",
"glance",
"manila",
"network",
"octavia",
"security",
"swift",
"tempest",
"tripleo",
"ui",
"validation"
],
"os_versions": [
"centos7",
"centos8",
"centos9",
"rhel8",
"rhel9"
]
}

62
plugins/module_utils/tripleo_repos/get_hash/exceptions.py
View File

@ -1,62 +0,0 @@
# Copyright 2021 Red Hat, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
#
#
from __future__ import (absolute_import, division, print_function)
__metaclass__ = type
class Base(Exception):
"""Base Exception"""
class TripleOHashMissingConfig(Base):
"""Missing configuration file for TripleOHashInfo. This is thrown
when there is no config.yaml in constants.CONFIG_PATH or the local
directory assuming execution from a source checkout.
"""
def __init__(self, error_msg):
super(TripleOHashMissingConfig, self).__init__(error_msg)
class TripleOHashInvalidConfig(Base):
"""Invalid configuration file for TripleOHashInfo. This is used when
any of they keys in constants.CONFIG_KEYS is not found in config.yaml.
"""
def __init__(self, error_msg):
super(TripleOHashInvalidConfig, self).__init__(error_msg)
class TripleOHashInvalidParameter(Base):
"""Invalid parameters passed for TripleOHashInfo. This is thrown when
the user passed invalid combination ofparameters parameters to the cli
entrypoint, for example specifying --component with centos7.
"""
def __init__(self, error_msg):
super(TripleOHashInvalidParameter, self).__init__(error_msg)
class TripleOHashInvalidDLRNResponse(Base):
"""Invalid response received from the DLRN server. This is seen if
the delorean server replies with a status code other than 200 OK for
a query to commit.yaml or delorean.repo.md5.
"""
def __init__(self, error_msg):
super(TripleOHashInvalidDLRNResponse, self).__init__(error_msg)

226
plugins/module_utils/tripleo_repos/get_hash/tripleo_hash_info.py
View File

@ -1,226 +0,0 @@
# Copyright 2021 Red Hat, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
#
#
from __future__ import (absolute_import, division, print_function)
import logging
import os
from .constants import CONFIG_PATH, CONFIG_KEYS, DEFAULT_CONFIG
from .exceptions import (
TripleOHashInvalidConfig, TripleOHashInvalidDLRNResponse
)
try:
from tripleo_repos.utils import http_get
except ImportError:
from ansible_collections.tripleo.repos.plugins.module_utils. \
tripleo_repos.utils import http_get
__metaclass__ = type
class TripleOHashInfo:
"""
Objects of type TripleOHashInfo contain the attributes required to
represent a particular delorean build hash. This includes the full, commit,
distro and extended hashes (where applicable), as well as the release,
OS name and version, component name (if applicable), named tag
(current-tripleo, tripleo-ci-testing etc) as well as the URL to the
delorean server that provided the information used to build each object
instance.
"""
@classmethod
def load_yaml(cls, filename):
import yaml
return yaml.safe_load(filename)
@classmethod
def _resolve_local_config_path(cls):
""" Load local config from disk from expected locations. """
paths = [
# pip install --user
os.path.expanduser(
"~/.local/etc/tripleo_get_hash/config.yaml"),
# root install
"/etc/tripleo_get_hash/config.yaml",
# embedded config.yaml as fallback
os.path.join(
os.path.dirname(os.path.abspath(__file__)), "config.yaml")
]
for _local_config in paths:
if cls._check_read_file(_local_config):
return _local_config
@classmethod
def _check_read_file(cls, filepath):
if os.path.isfile(filepath) and os.access(filepath, os.R_OK):
return True
return False
@classmethod
def load_config(cls, passed_config=None):
"""
This is a class method since we call it from the CLI entrypoint
before the TripleOHashInfo object is created. The method will first
try to use constants.CONFIG_PATH. If that is missing it tries to use
a local config.yaml for example for invocations from a source checkout
directory. If the file is not found TripleOHashMissingConfig is raised.
If any of the contants.CONFIG_KEYS is missing from config.yaml then
TripleOHashInvalidConfig is raised. If the passed_config dict contains
a given config value then that is used instead of the value from the
loaded config file. Returns a dictionary containing
the key->value for all the keys in constants.CONFIG_KEYS.
:param passed_config: dict with configuration overrides
:raises TripleOHashMissingConfig for missing config.yaml
:raises TripleOHashInvalidConfig for missing keys in config.yaml
:return: a config dictionary with the keys in constants.CONFIG_KEYS
"""
passed_config = passed_config or {}
result_config = {}
config_path = ''
local_config = cls._resolve_local_config_path()
# prefer const.CONFIG_PATH then local_config
if cls._check_read_file(CONFIG_PATH):
config_path = CONFIG_PATH
elif local_config:
config_path = local_config
else:
logging.info("Using embedded config file")
loaded_config = DEFAULT_CONFIG
logging.info("Using config file at %s", config_path)
if config_path != '':
with open(config_path, 'r') as config_yaml:
loaded_config = cls.load_yaml(config_yaml)
for k in CONFIG_KEYS:
if k not in loaded_config:
error_str = (
"Malformed config file - missing {0}. Expected all"
"of these configuration items: {1}"
).format(
k, ", ".join(CONFIG_KEYS)
)
logging.error(error_str)
raise TripleOHashInvalidConfig(error_str)
# if the passed config contains the key then use that value
if passed_config.get(k):
result_config[k] = passed_config[k]
else:
result_config[k] = loaded_config[k]
return result_config
def __init__(self, os_version, release, component, tag, config=None):
"""Create a new TripleOHashInfo object
:param os_version: The OS and version e.g. centos8
:param release: The OpenStack release e.g. wallaby
:param component: The tripleo-ci component e.g. 'common' or None
:param tag: The Delorean server named tag e.g. current-tripleo
:param config: Use an existing config dictionary and don't load it
"""
config = TripleOHashInfo.load_config(config)
self.os_version = os_version
self.release = release
self.component = component
self.tag = tag
repo_url = self._resolve_repo_url(config['dlrn_url'])
self.dlrn_url = repo_url
repo_url_response, status = http_get(repo_url)
if status != 200:
error_str = (
"Invalid response received from the delorean server. Queried "
"URL: {0}. Response code: {1}. Response text: {2}. Failed to "
"create TripleOHashInfo object."
).format(repo_url, status, repo_url_response)
logging.error(error_str)
raise TripleOHashInvalidDLRNResponse(error_str)
if repo_url.endswith('commit.yaml'):
from_commit_yaml = self._hashes_from_commit_yaml(repo_url_response)
self.full_hash = from_commit_yaml[0]
self.commit_hash = from_commit_yaml[1]
self.distro_hash = from_commit_yaml[2]
self.extended_hash = from_commit_yaml[3]
else:
self.full_hash = repo_url_response
self.commit_hash = None
self.distro_hash = None
self.extended_hash = None
def _resolve_repo_url(self, dlrn_url):
"""Resolve the delorean server URL given the various attributes of
this TripleOHashInfo object. The only passed parameter is the
dlrn_url. There are three main cases:
* centos8/rhel8 component https://trunk.rdoproject.org/centos8/component/common/current-tripleo/commit.yaml
* centos7 https://trunk.rdoproject.org/centos7/current-tripleo/commit.yaml
* centos8/rhel8 non component https://trunk.rdoproject.org/centos8/current-tripleo/delorean.repo.md5
Returns a string which is the full URL to the required item (i.e.
commit.yaml or repo.md5 depending on the case).
:param dlrn_url: The base url for the delorean server
:returns string URL to required commit.yaml or repo.md5
""" # noqa
repo_url = ''
if 'centos7' in self.os_version:
repo_url = "%s/%s-%s/%s/commit.yaml" % (
dlrn_url,
self.os_version,
self.release,
self.tag,
)
elif self.component is not None:
repo_url = "%s/%s-%s/component/%s/%s/commit.yaml" % (
dlrn_url,
self.os_version,
self.release,
self.component,
self.tag,
)
else:
repo_url = "%s/%s-%s/%s/delorean.repo.md5" % (
dlrn_url,
self.os_version,
self.release,
self.tag,
)
logging.debug("repo_url is %s", repo_url)
return repo_url
def _hashes_from_commit_yaml(self, delorean_result):
"""This function is used when a commit.yaml file is returned
by _resolve_repo_url. Returns a tuple containing the various
extracted hashes: full, commit, distro and extended
:returns tuple of strings full, commit, distro, extended hashes
"""
parsed_yaml = self.load_yaml(delorean_result)
commit = parsed_yaml['commits'][0]['commit_hash']
distro = parsed_yaml['commits'][0]['distro_hash']
full = "%s_%s" % (commit, distro[0:8])
extended = parsed_yaml['commits'][0]['extended_hash']
logging.debug(
"delorean commit.yaml results %s", parsed_yaml['commits'][0])
return full, commit, distro, extended
def __repr__(self):
"""Returns a string representation of the object"""
attrs = vars(self)
return ',\n'.join('%s: %s' % item for item in attrs.items())

632
plugins/module_utils/tripleo_repos/main.py
View File

@ -1,632 +0,0 @@
#!/usr/bin/env python
# Copyright 2016 Red Hat, Inc.
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.
from __future__ import (absolute_import, division, print_function)
import argparse
import os
import platform
import re
import subprocess
import sys
__metaclass__ = type
TITLE_RE = re.compile('\\[(.*)\\]')
NAME_RE = re.compile('name=(.+)')
PRIORITY_RE = re.compile('priority=\\d+')
# Packages to be included from delorean-current when using current-tripleo
INCLUDE_PKGS = ('includepkgs=instack,instack-undercloud,'
'os-apply-config,os-collect-config,os-net-config,'
'os-refresh-config,python*-tripleoclient,'
'openstack-tripleo-*,openstack-puppet-modules,'
'ansible-role-tripleo*,puppet-*,python*-tripleo-common,'
'python*-paunch*,tripleo-ansible,ansible-config_template')
DEFAULT_OUTPUT_PATH = '/etc/yum.repos.d'
DEFAULT_RDO_MIRROR = 'https://trunk.rdoproject.org'
# RHEL is only provided to licensed cloud providers via RHUI
DEFAULT_MIRROR_MAP = {
'fedora': 'https://mirrors.fedoraproject.org',
'centos7': 'http://mirror.centos.org',
'centos8': 'http://mirror.centos.org',
'centos9': 'http://mirror.stream.centos.org',
'ubi8': 'http://mirror.centos.org',
'ubi9': 'http://mirror.stream.centos.org',
'rhel8': 'https://trunk.rdoproject.org',
'rhel9': 'https://trunk.rdoproject.org',
}
CEPH_REPO_TEMPLATE = '''
[tripleo-centos-ceph-%(ceph_release)s]
name=tripleo-centos-ceph-%(ceph_release)s
baseurl=%(mirror)s/centos/%(centos_release)s/storage/$basearch/ceph-%(ceph_release)s/
gpgcheck=0
enabled=1
'''
CEPH_SIG_REPO_TEMPLATE = '''
[tripleo-centos-ceph-%(ceph_release)s]
name=tripleo-centos-ceph-%(ceph_release)s
baseurl=%(mirror)s/SIGs/%(centos_release)s/storage/$basearch/ceph-%(ceph_release)s/
gpgcheck=0
enabled=1
'''
CEPH_RDO_REPO_TEMPLATE = '''
[tripleo-centos-ceph-%(ceph_release)s]
name=tripleo-centos-ceph-%(ceph_release)s
baseurl=https://trunk.rdoproject.org/centos8-master/deps/storage/%(ceph_release)s/
gpgcheck=0
enabled=1
'''
OPSTOOLS_REPO_TEMPLATE = '''
[tripleo-centos-opstools]
name=tripleo-centos-opstools
baseurl=%(mirror)s/centos/7/opstools/$basearch/
gpgcheck=0
enabled=1
'''
# centos-8 only
HIGHAVAILABILITY_REPO_TEMPLATE = '''
[tripleo-centos-highavailability]
name=tripleo-centos-highavailability
baseurl=%(mirror)s/%(legacy_url)s%(stream)s/HighAvailability/$basearch/os/
gpgcheck=0
enabled=1
'''
# centos-8 only
POWERTOOLS_REPO_TEMPLATE = '''
[tripleo-centos-powertools]
name=tripleo-centos-powertools
baseurl=%(mirror)s/%(legacy_url)s%(stream)s/%(pt_name)s/$basearch/os/
gpgcheck=0
enabled=1
'''
# ubi-8 only
APPSTREAM_REPO_TEMPLATE = '''
[tripleo-centos-appstream]
name=tripleo-centos-appstream
baseurl=%(mirror)s/%(legacy_url)s%(stream)s/AppStream/$basearch/os/
gpgcheck=0
enabled=1
%(extra)s
'''
BASE_REPO_TEMPLATE = '''
[tripleo-centos-baseos]
name=tripleo-centos-baseos
baseurl=%(mirror)s/%(legacy_url)s%(stream)s/BaseOS/$basearch/os/
gpgcheck=0
enabled=1
'''
# unversioned fedora added for backwards compatibility
SUPPORTED_DISTROS = [
('centos', '7'),
('centos', '8'),
('centos', '9'),
('fedora', ''),
('rhel', '8'),
('rhel', '9'),
('ubi', '8'),
('ubi', '9') # a subcase of the rhel distro
]
DISTRO_CHOICES = ["".join(distro_pair)
for distro_pair in SUPPORTED_DISTROS]
class InvalidArguments(Exception):
pass
class NoRepoTitle(Exception):
pass
def _get_distro():
"""Get distro info from os-release
returns: distro_id, distro_major_version_id, distro_name
"""
# Avoids a crash on unsupported platforms which would prevent even
# running with `--help`.
if not os.path.exists('/etc/os-release'):
return platform.system(), 'unknown', 'unknown'
output = subprocess.Popen(
'source /etc/os-release && echo -e -n "$ID\n$VERSION_ID\n$NAME"',
shell=True,
stdout=subprocess.PIPE,
stderr=open(os.devnull, 'w'),
executable='/bin/bash',
universal_newlines=True).communicate()
# distro_id and distro_version_id will always be at least an empty string
distro_id, distro_version_id, distro_name = output[0].split('\n')
# if distro_version_id is empty string the major version will be empty
# string too
distro_major_version_id = distro_version_id.split('.')[0]
# check if that is UBI subcase?
if os.path.exists('/etc/yum.repos.d/ubi.repo'):
distro_id = 'ubi'
if (distro_id, distro_major_version_id) not in SUPPORTED_DISTROS:
print(
"WARNING: Unsupported platform '{0}{1}' detected by tripleo-repos,"
" centos7 will be used unless you use CLI param to change it."
"".format(distro_id, distro_major_version_id), file=sys.stderr)
distro_id = 'centos'
distro_major_version_id = '7'
if distro_id == 'ubi':
print(
"WARNING: Centos{0} Base and AppStream will be installed for "
"this UBI distro".format(distro_major_version_id))
return distro_id, distro_major_version_id, distro_name
def _parse_args(distro_id, distro_major_version_id):
distro = "{0}{1}".format(distro_id, distro_major_version_id)
parser = argparse.ArgumentParser(
description='Download and install repos necessary for TripleO. Note '
'that some of these repos require yum-plugin-priorities, '
'so that will also be installed.',
formatter_class=argparse.ArgumentDefaultsHelpFormatter)
parser.add_argument('repos', metavar='REPO', nargs='+',
choices=['current', 'deps', 'current-tripleo',
'current-tripleo-dev', 'ceph', 'opstools',
'tripleo-ci-testing', 'current-tripleo-rdo'],
help='A list of repos. Available repos: '
'%(choices)s. The deps repo will always be '
'included when using current or '
'current-tripleo. current-tripleo-dev '
'downloads the current-tripleo, current, and '
'deps repos, but sets the current repo to only '
'be used for TripleO projects. It also modifies '
'each repo\'s priority so packages are installed '
'from the appropriate location.')
parser.add_argument('-d', '--distro',
default=distro,
choices=DISTRO_CHOICES,
nargs='?',
help='Target distro with default detected at runtime. '
)
parser.add_argument('-b', '--branch',
default='master',
help='Target branch. Should be the lowercase name of '
'the OpenStack release. e.g. liberty')
parser.add_argument('-o', '--output-path',
default=DEFAULT_OUTPUT_PATH,
help='Directory in which to save the selected repos.')
parser.add_argument('--mirror',
help='Server from which to install base OS packages. '
'Default value is based on distro param.')
parser.add_argument('--rdo-mirror',
default=DEFAULT_RDO_MIRROR,
help='Server from which to install RDO packages.')
stream_group = parser.add_mutually_exclusive_group()
stream_group.add_argument('--stream',
action='store_true',
default=True,
help='Enable stream support for CentOS repos')
stream_group.add_argument('--no-stream',
action='store_true',
default=False,
help='Disable stream support for CentOS repos')
args = parser.parse_args()
if args.no_stream:
args.stream = False
# Default mirror for args.distro (which defaults to 'distro')
default_mirror = DEFAULT_MIRROR_MAP.get(args.distro, None)
if default_mirror is None and 'fedora' in args.distro:
# We don't have different mirrors for specific fedora releases
default_mirror = DEFAULT_MIRROR_MAP.get('fedora', None)
if args.mirror is None:
args.mirror = default_mirror
args.old_mirror = default_mirror
return args
def _get_repo(path, args):
# lazy import
if 'requests' not in globals():
import requests
r = requests.get(path)
if r.status_code == 200:
return _inject_mirrors(r.text, args)
else:
r.raise_for_status()
def _write_repo(content, target, name=None):
if not name:
m = TITLE_RE.search(content)
if not m:
raise NoRepoTitle('Could not find repo title in: \n%s' % content)
name = m.group(1)
# centos-8 dlrn repos have changed. repos per component
# are folded into a single repo.
if 'component' in name:
name = 'delorean'
filename = name + '.repo'
filename = os.path.join(target, filename)
with open(filename, 'w') as f:
f.write(content)
print('Installed repo %s to %s' % (name, filename))
def _validate_distro_repos(args):
"""Validate requested repos are valid for the distro"""
valid_repos = []
if 'fedora' in args.distro:
valid_repos = ['current', 'current-tripleo', 'ceph', 'deps',
'tripleo-ci-testing']
elif args.distro in DISTRO_CHOICES:
valid_repos = ['ceph', 'current', 'current-tripleo',
'current-tripleo-dev', 'deps', 'tripleo-ci-testing',
'opstools', 'current-tripleo-rdo']
invalid_repos = [x for x in args.repos if x not in valid_repos]
if len(invalid_repos) > 0:
raise InvalidArguments(
'{0} repo(s) are not valid for {1}. Valid repos '
'are: {2}'.format(invalid_repos, args.distro, valid_repos))
return True
def _validate_current_tripleo(repos):
"""Validate current usage
current and current-tripleo cannot be specified with each other and
current-tripleo-dev is a mix of current, current-tripleo and deps
so they should not be specified on the command line with each other.
"""
if 'current-tripleo' in repos and 'current' in repos:
raise InvalidArguments('Cannot use current and current-tripleo at the '
'same time.')
if 'current-tripleo-dev' not in repos:
return True
if 'current' in repos or 'current-tripleo' in repos or 'deps' in repos:
raise InvalidArguments('current-tripleo-dev should not be used with '
'any other RDO Trunk repos.')
return True
def _validate_tripleo_ci_testing(repos):
"""Validate tripleo-ci-testing
With tripleo-ci-testing for repo (currently only periodic container build)
no other repos expected except optionally deps|ceph|opstools
which is enabled regardless.
"""
if 'tripleo-ci-testing' in repos and len(repos) > 1:
if 'deps' in repos or 'ceph' in repos or 'opstools' in repos:
return True
else:
raise InvalidArguments('Cannot use tripleo-ci-testing at the '
'same time as other repos, except '
'deps|ceph|opstools.')
return True
def _validate_distro_stream(args, distro_name, distro_major_version_id):
"""Validate stream related args vs host
Fails if stream is to be used but the host isn't a stream OS or vice versa
"""
if args.output_path != DEFAULT_OUTPUT_PATH:
# don't validate distro name because the output path is not
# /etc/yum.repos.d, so the repo files may not be used to install
# packages on this host
return True
if 'centos' not in distro_name.lower():
return True
if distro_name.lower() == 'centos' and distro_major_version_id != '8':
return True
is_stream = args.stream and not args.no_stream
if is_stream and 'stream' not in distro_name.lower():
raise InvalidArguments('--stream provided, but OS is not the Stream '
'version. Please ensure the host is Stream.')
elif not is_stream and 'stream' in distro_name.lower():
raise InvalidArguments('--no-stream provided, but OS is the Stream '
'version. Please ensure the host is not the '
'Stream version.')
return True
def _validate_args(args, distro_name, distro_major_version_id):
_validate_current_tripleo(args.repos)
_validate_distro_repos(args)
_validate_tripleo_ci_testing(args.repos)
_validate_distro_stream(args, distro_name, distro_major_version_id)
def _remove_existing(args):
"""Remove any delorean* or opstools repos that already exist"""
if args.distro in ['ubi8', 'ubi9']:
regex = '^(BaseOS|AppStream|delorean|tripleo-centos-' \
'(opstools|ceph|highavailability|powertools)).*.repo'
else:
regex = '^(delorean|tripleo-centos-' \
'(opstools|ceph|highavailability|powertools)).*.repo'
pattern = re.compile(regex)
if os.path.exists("/etc/distro.repos.d"):
paths = set(
os.listdir(args.output_path) + os.listdir("/etc/distro.repos.d"))
else:
paths = os.listdir(args.output_path)
for f in paths:
if pattern.match(f):
filename = os.path.join(args.output_path, f)
if os.path.exists(filename):
os.remove(filename)
print('Removed old repo "%s"' % filename)
filename = os.path.join("/etc/distro.repos.d", f)
if os.path.exists(filename):
os.remove(filename)
print('Removed old repo "%s"' % filename)
def _get_base_path(args):
if args.distro in ['ubi8', 'ubi9']:
# there are no base paths for UBI that work well
distro = args.distro.replace('ubi', 'centos')
else:
distro = args.distro
# The mirror url with /$DISTRO$VERSION path for master branch is
# deprecated.
# The default for rdo mirrors is $DISTRO$VERSION-$BRANCH
# it should work for every (distro, branch) pair that
# makes sense
# Any exception should be corrected at source, not here.
distro_branch = '%s-%s' % (distro, args.branch)
return '%s/%s/' % (args.rdo_mirror, distro_branch)
def _install_priorities():
try:
subprocess.check_call(['yum', 'install', '-y',
'yum-plugin-priorities'])
except subprocess.CalledProcessError as e:
print('ERROR: Failed to install yum-plugin-priorities\n%s\n%s' %
(e.cmd, e.output))
raise
def _create_ceph(args, release):
"""Generate a Ceph repo file for release"""
centos_release = '9-stream'
template = CEPH_SIG_REPO_TEMPLATE
if args.distro == 'centos7':
centos_release = '7'
template = CEPH_REPO_TEMPLATE
elif args.distro == 'centos8' and release == 'nautilus':
template = CEPH_RDO_REPO_TEMPLATE
elif args.distro == 'centos8':
centos_release = '8-stream'
template = CEPH_REPO_TEMPLATE
return template % {'centos_release': centos_release,
'ceph_release': release,
'mirror': args.mirror}
def _change_priority(content, new_priority):
new_content = PRIORITY_RE.sub('priority=%d' % new_priority, content)
# This shouldn't happen, but let's be safe.
if not PRIORITY_RE.search(new_content):
new_content = []
for line in content.split("\n"):
new_content.append(line)
if line.startswith('['):
new_content.append('priority=%d' % new_priority)
new_content = "\n".join(new_content)
return new_content
def _add_includepkgs(content):
new_content = []
for line in content.split("\n"):
new_content.append(line)
if line.startswith('['):
new_content.append(INCLUDE_PKGS)
return "\n".join(new_content)
def _inject_mirrors(content, args):
"""Replace any references to the default mirrors in repo content
In some cases we want to use mirrors whose repo files still point to the
default servers. If the user specified to use the mirror, we want to
replace any such references with the mirror address. This function
handles that by using a regex to swap out the baseurl server.
"""
content = re.sub('baseurl=%s' % DEFAULT_RDO_MIRROR,
'baseurl=%s' % args.rdo_mirror,
content)
if args.old_mirror:
content = re.sub('baseurl=%s' % args.old_mirror,
'baseurl=%s' % args.mirror,
content)
return content
def _install_repos(args, base_path):
def install_deps(args, base_path):
content = _get_repo(base_path + 'delorean-deps.repo', args)
_write_repo(content, args.output_path)
for repo in args.repos:
if repo == 'current':
content = _get_repo(base_path + 'current/delorean.repo', args)
_write_repo(content, args.output_path, name='delorean')
install_deps(args, base_path)
elif repo == 'deps':
install_deps(args, base_path)
elif repo == 'current-tripleo':
content = _get_repo(base_path + 'current-tripleo/delorean.repo',
args)
_write_repo(content, args.output_path)
install_deps(args, base_path)
elif repo == 'current-tripleo-dev':
content = _get_repo(base_path + 'delorean-deps.repo', args)
_write_repo(content, args.output_path)
content = _get_repo(base_path + 'current-tripleo/delorean.repo',
args)
content = TITLE_RE.sub('[\\1-current-tripleo]', content)
content = NAME_RE.sub('name=\\1-current-tripleo', content)
# We need to twiddle priorities since we're mixing multiple repos
# that are generated with the same priority.
content = _change_priority(content, 20)
_write_repo(content, args.output_path,
name='delorean-current-tripleo')
content = _get_repo(base_path + 'current/delorean.repo', args)
content = _add_includepkgs(content)
content = _change_priority(content, 10)
_write_repo(content, args.output_path, name='delorean')
elif repo == 'tripleo-ci-testing':
content = _get_repo(base_path + 'tripleo-ci-testing/delorean.repo',
args)
_write_repo(content, args.output_path)
install_deps(args, base_path)
elif repo == 'current-tripleo-rdo':
content = _get_repo(
base_path + 'current-tripleo-rdo/delorean.repo', args)
_write_repo(content, args.output_path)
install_deps(args, base_path)
elif repo == 'ceph':
if args.branch in ['liberty', 'mitaka']:
content = _create_ceph(args, 'hammer')
elif args.branch in ['newton', 'ocata', 'pike']:
content = _create_ceph(args, 'jewel')
elif args.branch in ['queens', 'rocky']:
content = _create_ceph(args, 'luminous')
elif args.branch in ['stein', 'train', 'ussuri', 'victoria']:
content = _create_ceph(args, 'nautilus')
else:
content = _create_ceph(args, 'pacific')
_write_repo(content, args.output_path)
elif repo == 'opstools':
content = OPSTOOLS_REPO_TEMPLATE % {'mirror': args.mirror}
_write_repo(content, args.output_path)
else:
raise InvalidArguments('Invalid repo "%s" specified' % repo)
distro = args.distro
# CentOS-8 AppStream is required for UBI-8
legacy_url = 'centos/'
if distro in ['ubi8', 'ubi9']:
if not os.path.exists("/etc/distro.repos.d"):
print('WARNING: For UBI it is recommended to create '
'/etc/distro.repos.d and rerun!')
dp_exists = False
else:
dp_exists = True
if args.output_path == DEFAULT_OUTPUT_PATH and dp_exists:
distro_path = "/etc/distro.repos.d"
else:
distro_path = args.output_path
# TODO: Remove it once bugs are fixed
# Add extra options to APPSTREAM_REPO_TEMPLATE because of
# rhbz/1961558 and lpbz/1929634
extra = ''
if args.branch in ['train', 'ussuri', 'victoria']:
extra = 'exclude=edk2-ovmf-20200602gitca407c7246bf-5*'
distro_name = str(distro[-1]) + '-stream'
content = APPSTREAM_REPO_TEMPLATE % {'mirror': args.mirror,
'extra': extra,
'legacy_url': legacy_url,
'stream': distro_name}
_write_repo(content, distro_path)
content = BASE_REPO_TEMPLATE % {'mirror': args.mirror,
'legacy_url': legacy_url,
'stream': distro_name}
_write_repo(content, distro_path)
if distro in ['centos8', 'centos9', 'ubi8', 'ubi9']:
distro = 'centos' + str(distro[-1])
if 'centos' in distro:
stream = str(distro[-1])
# HA, Powertools are required for CentOS-8
if int(stream) >= 8:
if args.stream and not args.no_stream:
stream = stream + '-stream'
pt_name = 'PowerTools'
if '9' in stream:
legacy_url = ''
pt_name = 'CRB'
content = HIGHAVAILABILITY_REPO_TEMPLATE % {
'mirror': args.mirror,
'stream': stream,
'legacy_url': legacy_url}
_write_repo(content, args.output_path)
content = POWERTOOLS_REPO_TEMPLATE % {'mirror': args.mirror,
'stream': stream,
'legacy_url': legacy_url,
'pt_name': pt_name}
_write_repo(content, args.output_path)
if '9' in stream:
content = APPSTREAM_REPO_TEMPLATE % {'mirror': args.mirror,
'extra': '',
'legacy_url': legacy_url,
'stream': stream}
_write_repo(content, args.output_path)
content = BASE_REPO_TEMPLATE % {'mirror': args.mirror,
'legacy_url': legacy_url,
'stream': stream}
_write_repo(content, args.output_path)
def _run_pkg_clean(distro):
pkg_mgr = 'yum' if distro == 'centos7' else 'dnf'
try:
subprocess.check_call([pkg_mgr, 'clean', 'metadata'])
except subprocess.CalledProcessError:
print('ERROR: Failed to clean yum metadata.')
raise
def main():
distro_id, distro_major_version_id, distro_name = _get_distro()
args = _parse_args(distro_id, distro_major_version_id)
_validate_args(args, distro_name, distro_major_version_id)
base_path = _get_base_path(args)
if (distro_name.lower(), distro_major_version_id) == ("centos", "7"):
_install_priorities()
_remove_existing(args)
_install_repos(args, base_path)
_run_pkg_clean(args.distro)
if __name__ == '__main__':
main()

83
plugins/module_utils/tripleo_repos/utils.py
View File

@ -1,83 +0,0 @@
# Copyright 2021 Red Hat, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
#
from __future__ import (absolute_import, division, print_function)
import logging
import sys
__metaclass__ = type
# portable http_get that uses either ansible recommended way or python native
# urllib. Also deals with python2 vs python3 for centos7 train jobs.
py_version = sys.version_info.major
if py_version < 3:
import urllib2
def http_get(url):
try:
response = urllib2.urlopen(url)
return (
response.read().decode('utf-8'),
int(response.code))
except Exception as e:
return (str(e), -1)
else:
try:
from ansible.module_utils.urls import open_url
def http_get(url):
try:
response = open_url(url, method='GET')
return (response.read().decode('utf-8'), response.status)
except Exception as e:
return (str(e), -1)
except ImportError:
from urllib.request import urlopen
def http_get(url):
try:
response = urlopen(url)
return (
response.read().decode('utf-8'),
int(response.status))
except Exception as e:
return (str(e), -1)
def load_logging(level=logging.INFO, module_name="tripleo-repos"):
"""Load and set logging level. Default is set to logging.INFO level."""
logger = logging.getLogger()
# Only add logger once to avoid duplicated streams in tests
if not logger.handlers:
stdout_handlers = [
_handler
for _handler in logger.handlers
if
(
hasattr(_handler, 'stream') and 'stdout' in
_handler.stream.name
)
]
if stdout_handlers == []:
formatter = logging.Formatter(
(
"%(asctime)s - " + module_name + " - %(levelname)s - "
"%(message)s"
)
)
handler = logging.StreamHandler(sys.stdout)
handler.setFormatter(formatter)
logger.addHandler(handler)
logger.setLevel(level)

0
plugins/module_utils/tripleo_repos/yum_config/__init__.py
View File

306
plugins/module_utils/tripleo_repos/yum_config/__main__.py
View File

@ -1,306 +0,0 @@
# Copyright 2021 Red Hat, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
import argparse
import logging
import os
import sys
from tripleo_repos.utils import load_logging
import tripleo_repos.yum_config.constants as const
import tripleo_repos.yum_config.yum_config as cfg
import tripleo_repos.yum_config.utils as utils
def options_to_dict(options):
opt_dict = {}
if options:
for opt in options:
try:
k, v = opt.split('=')
except ValueError:
msg = 'Set options must be provided as "key=value" pairs'
logging.error(msg)
sys.exit(2)
opt_dict[k] = v
return opt_dict
def main():
load_logging(module_name="tripleo-yum-config")
# Get release model and version
distro, major_version, __ = utils.get_distro_info()
py_version = sys.version_info.major
if py_version < 3:
logging.warning("Some operations will be disabled when running with "
"python 2.")
# Repo arguments
repo_args_parser = argparse.ArgumentParser(add_help=False)
repo_args_parser.add_argument(
'--name',
help='name of the repo to be modified'
)
environment_parse = argparse.ArgumentParser(add_help=False)
environment_parse.add_argument(
'--environment-file',
dest='env_file',
default=None,
help=('path to an environment file to be read before creating repo '
'files'),
)
parser_enable_group = repo_args_parser.add_mutually_exclusive_group()
parser_enable_group.add_argument(
'--enable',
action='store_true',
dest='enable',
default=None,
help='enable a yum repo or module'
)
parser_enable_group.add_argument(
'--disable',
action='store_false',
dest='enable',
default=None,
help='disable a yum repo or module'
)
repo_args_parser.add_argument(
'--config-dir-path',
dest='config_dir_path',
default=const.YUM_REPO_DIR,
help=(
'set the absolute directory path that holds all repo '
'configuration files')
)
repo_args_parser.add_argument(
'--down-url',
dest='down_url',
help=(
'URL of a repo file to be used as base to create or update '
'a repo configuration file.')
)
# Generic key-value options
options_parse = argparse.ArgumentParser(add_help=False)
options_parse.add_argument(
'--set-opts',
dest='set_opts',
nargs='+',
help='sets config options as key=value pairs for a specific '
'configuration file'
)
# dnf module parser
dnf_module_parser = argparse.ArgumentParser(add_help=False)
dnf_module_parser.add_argument(
'operation',
choices=['enable', 'disable', 'install', 'remove', 'reset'],
help="dnf module operation to be executed"
)
dnf_module_parser.add_argument(
'name',
help='name of the module to be modified'
)
dnf_module_parser.add_argument(
'--stream',
help="sets module stream"
)
dnf_module_parser.add_argument(
'--profile',
help="sets module profile"
)
# Compose repo arguments
compose_args_parser = argparse.ArgumentParser(add_help=False)
compose_args_parser.add_argument(
'--compose-url',
dest='compose_url',
required=True,
help='CentOS compose URL'
)
compose_args_parser.add_argument(
'--release',
dest='release',
choices=const.COMPOSE_REPOS_RELEASES,
default='centos-stream-8',
help='target CentOS release.'
)
compose_args_parser.add_argument(
'--arch',
choices=const.COMPOSE_REPOS_SUPPORTED_ARCHS,
default='x86_64',
help='set the architecture for the destination repos.'
)
compose_args_parser.add_argument(
'--disable-repos',
nargs='+',
help='list of repo names or repo absolute file paths to be disabled.'
)
compose_args_parser.add_argument(
'--disable-all-conflicting',
action='store_true',
dest='disable_conflicting',
default=False,
help='after enabling compose repos, disable all other repos that '
'match variant names.'
)
compose_args_parser.add_argument(
'--variants',
nargs='+',
help='Name of the repos to be enabled. Default behavior is to enable '
'all that match a specific release and architecture.'
)
compose_args_parser.add_argument(
'--config-dir-path',
dest='config_dir_path',
default=const.YUM_REPO_DIR,
help='set the absolute directory path that holds all repo '
'configuration files'
)
# Common file path argument
common_parse = argparse.ArgumentParser(add_help=False)
common_parse.add_argument(
'--config-file-path',
dest='config_file_path',
help=('set the absolute file path of the configuration file to be '
'updated.')
)
# Main parser
main_parser = argparse.ArgumentParser()
main_parser.add_argument(
'--verbose', '-v',
action='store_true',
default=False,
help='enable verbose log level for debugging',
)
subparsers = main_parser.add_subparsers(dest='command')
# Subcommands
subparsers.add_parser(
'repo',
parents=[common_parse, environment_parse, repo_args_parser,
options_parse],
help='updates a yum repository options'
)
subparsers.add_parser(
'global',
parents=[common_parse, environment_parse, options_parse],
help='updates global yum configuration options'
)
if py_version >= 3:
subparsers.add_parser(
'enable-compose-repos',
parents=[compose_args_parser, environment_parse],
help='enable CentOS compose repos based on an compose url.'
)
for min_distro_ver in const.DNF_MODULE_MINIMAL_DISTRO_VERSIONS:
if (distro == min_distro_ver.get('distro') and int(
major_version) >= min_distro_ver.get('min_version')):
subparsers.add_parser(
'module',
parents=[dnf_module_parser],
help='updates yum module options'
)
break
args = main_parser.parse_args()
if args.command is None:
main_parser.print_help()
sys.exit(2)
if args.verbose:
logging.getLogger().setLevel(logging.DEBUG)
logging.debug('Logging level set to DEBUG')
if args.command == 'repo':
set_dict = options_to_dict(args.set_opts)
config_obj = cfg.TripleOYumRepoConfig(
dir_path=args.config_dir_path,
environment_file=args.env_file)
if args.name is not None:
config_obj.add_or_update_section(args.name, set_dict=set_dict,
file_path=args.config_file_path,
enabled=args.enable,
from_url=args.down_url)
else:
# When no section (name) is provided, we consider all sections from
# repo file downloaded from the URL, otherwise fail.
if args.down_url is None:
logging.error("You must provide a repo 'name' or a valid "
"'url' where repo info can be downloaded.")
sys.exit(2)
config_obj.add_or_update_all_sections_from_url(
args.down_url, file_path=args.config_file_path,
set_dict=set_dict, enabled=args.enable)
elif args.command == 'module':
import tripleo_repos.yum_config.dnf_manager as dnf_mgr
dnf_mod_mgr = dnf_mgr.DnfModuleManager()
dnf_method = getattr(dnf_mod_mgr, args.operation + "_module")
dnf_method(args.name, stream=args.stream, profile=args.profile)
elif args.command == 'global':
set_dict = options_to_dict(args.set_opts)
config_obj = cfg.TripleOYumGlobalConfig(
file_path=args.config_file_path,
environment_file=args.env_file)
config_obj.update_section('main', set_dict)
elif args.command == 'enable-compose-repos':
import tripleo_repos.yum_config.compose_repos as compose_repos
repo_obj = compose_repos.TripleOYumComposeRepoConfig(
args.compose_url,
args.release,
dir_path=args.config_dir_path,
arch=args.arch,
environment_file=args.env_file)
repo_obj.enable_compose_repos(variants=args.variants,
override_repos=args.disable_conflicting)
if args.disable_repos:
for file in args.disable_repos:
valid_path = None
rel_path = os.path.join(args.config_dir_path, file)
if cfg.validated_file_path(file):
valid_path = file
elif cfg.validated_file_path(rel_path):
valid_path = rel_path
if valid_path is not None:
repo_obj.update_all_sections(valid_path, enabled=False)
def cli_entrypoint():
try:
main()
sys.exit(0)
except KeyboardInterrupt:
logging.info("Exiting on user interrupt")
sys.exit(2)
except Exception as e:
logging.error(str(e))
sys.exit(2)
if __name__ == "__main__":
cli_entrypoint()

206
plugins/module_utils/tripleo_repos/yum_config/compose_repos.py
View File

@ -1,206 +0,0 @@
# Copyright 2021 Red Hat, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
#
from __future__ import (absolute_import, division, print_function)
import logging
import json
import os
import re
from .constants import (
YUM_REPO_DIR,
YUM_REPO_FILE_EXTENSION,
YUM_REPO_SUPPORTED_OPTIONS,
COMPOSE_REPOS_RELEASES,
COMPOSE_REPOS_INFO_PATH,
COMPOSE_REPOS_URL_PATTERN,
COMPOSE_REPOS_URL_REPLACE_STR,
)
from .exceptions import (
TripleOYumConfigInvalidSection,
TripleOYumConfigComposeError,
)
from .yum_config import (
TripleOYumConfig
)
__metaclass__ = type
class TripleOYumComposeRepoConfig(TripleOYumConfig):
"""Manages yum repo configuration files for CentOS Compose."""
def __init__(self, compose_url, release, dir_path=None, arch=None,
environment_file=None):
conf_dir_path = dir_path or YUM_REPO_DIR
self.arch = arch or 'x86_64'
# 1. validate release name
if release not in COMPOSE_REPOS_RELEASES:
msg = 'CentOS release not supported.'
raise TripleOYumConfigComposeError(error_msg=msg)
self.release = release
# 2. Validate URL
pattern = re.compile(COMPOSE_REPOS_URL_PATTERN[self.release])
if not pattern.match(compose_url):
msg = 'The provided URL does not match the expect pattern.'
raise TripleOYumConfigComposeError(error_msg=msg)
# 3. Get compose info from url
segments = [compose_url,
COMPOSE_REPOS_INFO_PATH[self.release]]
self.compose_info_url = '/'.join(s.strip('/') for s in segments)
self.compose_info = self._get_compose_info()
# 4. Get compose-id from metadata
self.compose_id = self.compose_info['compose']['id']
# 5. Replace the compose-id from url to avoid 'labels'
repl_args = {'compose_id': self.compose_id}
self.compose_url = (
pattern.sub(
COMPOSE_REPOS_URL_REPLACE_STR[self.release] % repl_args,
compose_url)
)
super(TripleOYumComposeRepoConfig, self).__init__(
valid_options=YUM_REPO_SUPPORTED_OPTIONS,
dir_path=conf_dir_path,
file_extension=YUM_REPO_FILE_EXTENSION,
environment_file=environment_file)
def _get_compose_info(self):
"""Retrieve compose info for a provided compose-id url."""
# NOTE(dviroel): works for both centos 8 and 9
import urllib.request
try:
logging.debug("Retrieving compose info from url: %s",
self.compose_info_url)
res = urllib.request.urlopen(self.compose_info_url)
except Exception:
msg = ("Failed to retrieve compose info from url: %s"
% self.compose_info_url)
raise TripleOYumConfigComposeError(error_msg=msg)
compose_info = json.loads(res.read())
if compose_info['header']['version'] != "1.2":
# NOTE(dviroel): Log a warning just in case we receive a different
# version here. Code may fail depending on the change.
logging.warning("Expecting compose info version '1.2' but got %s.",
compose_info['header']['version'])
return compose_info['payload']
def _get_repo_name(self, variant):
return " ".join([self.compose_id, variant])
def _get_repo_filename(self, variant):
return "-".join([self.compose_id, variant]) + '.repo'
def _get_repo_base_url(self, variant):
"""Build the base_url based on variant name and system architecture."""
variant_info = self.compose_info['variants'][variant]
if not variant_info['paths'].get('repository', {}).get(self.arch):
# Variant has no support yet
return None
segments = [self.compose_url,
variant_info['paths']['repository'][self.arch]]
return '/'.join(s.strip('/') for s in segments)
def get_compose_variants(self):
return self.compose_info['variants'].keys()
def enable_compose_repos(self, variants=None, override_repos=False):
"""Enable CentOS compose repos of a given variant list.
This function will build from scratch all repos for a given compose-id
url. If a list of variants is not provided, it will enable all for all
variants returned from compose info.
:param variants: A list of variant names to be enabled.
:param override_repos: True if all matching variants in the same
repo directory should be disable in favor of the new repos.
"""
if variants:
for var in variants:
if not (var in self.compose_info['variants'].keys()):
msg = 'One or more provided variants are invalid.'
raise TripleOYumConfigComposeError(error_msg=msg)
else:
variants = self.compose_info['variants'].keys()
updated_repos = {}
for var in variants:
base_url = self._get_repo_base_url(var)
if not base_url:
continue
add_dict = {
'name': self._get_repo_name(var),
'baseurl': base_url,
'enabled': '1',
'gpgcheck': '0',
}
filename = self._get_repo_filename(var)
file_path = os.path.join(self.dir_path, filename)
# create a file if doesn't exist and add a section to it
try:
self.add_section(var.lower(), add_dict, file_path)
except TripleOYumConfigInvalidSection:
logging.debug("Section '%s' that already exists in this file. "
"Trying to update it...", var)
self.update_section(var.lower(),
set_dict=add_dict,
file_path=file_path)
# needed to override other repos
updated_repos[var.lower()] = file_path
if override_repos:
for var in updated_repos:
config_files = self._get_config_files(var)
for file in config_files:
if file != updated_repos[var]:
msg = ("Disabling matching section '%(section)s' in "
"configuration file: %(file)s.")
msg_args = {
'section': var,
'file': file,
}
logging.debug(msg, msg_args)
self.update_section(var, enabled=False, file_path=file)
def add_section(self, section, add_dict, file_path):
# Create a new file if it does not exists
if not os.path.isfile(file_path):
with open(file_path, 'w+'):
pass
super(TripleOYumComposeRepoConfig, self).add_section(
section, add_dict, file_path)
def update_section(
self, section, set_dict=None, enabled=None, file_path=None):
update_dict = set_dict or {}
if enabled is not None:
update_dict['enabled'] = '1' if enabled else '0'
if update_dict:
super(TripleOYumComposeRepoConfig, self).update_section(
section, update_dict, file_path=file_path)
def update_all_sections(self, file_path, set_dict=None, enabled=None):
update_dict = set_dict or {}
if enabled is not None:
update_dict['enabled'] = '1' if enabled else '0'
if update_dict:
super(TripleOYumComposeRepoConfig, self).update_all_sections(
update_dict, file_path)

87
plugins/module_utils/tripleo_repos/yum_config/constants.py
View File

@ -1,87 +0,0 @@
# Copyright 2021 Red Hat, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
#
#
from __future__ import (absolute_import, division, print_function)
"""
List of options that can be updated for yum repo files.
"""
__metaclass__ = type
YUM_REPO_SUPPORTED_OPTIONS = [
'baseurl',
'cost',
'enabled',
'exclude',
'excludepkgs',
'gpgcheck',
'gpgkey',
'includepkgs',
'metalink',
'mirrorlist',
'module_hotfixes',
'name',
'priority',
'skip_if_unavailable',
]
"""
Default constants for yum repo operations.
"""
YUM_REPO_DIR = '/etc/yum.repos.d'
YUM_REPO_FILE_EXTENSION = '.repo'
"""
Default constants for yum/dnf global configurations.
"""
YUM_GLOBAL_CONFIG_FILE_PATH = '/etc/yum.conf'
"""
CentOS Stream compose repos defaults
"""
COMPOSE_REPOS_RELEASES = [
"centos-stream-8",
"centos-stream-9"
]
COMPOSE_REPOS_SUPPORTED_ARCHS = [
"aarch64",
"ppc64le",
"x86_64"
]
COMPOSE_REPOS_URL_PATTERN = {
"centos-stream-8": r"(^https:.*.centos.org/)([^/]*)(/compose/?$)",
"centos-stream-9": r"(^https:.*.centos.org/.*/)(.*)(/compose/?$)",
}
COMPOSE_REPOS_URL_REPLACE_STR = {
"centos-stream-8": r"\1%(compose_id)s\3",
"centos-stream-9": r"\1%(compose_id)s\3",
}
COMPOSE_REPOS_INFO_PATH = {
"centos-stream-8": "metadata/composeinfo.json",
"centos-stream-9": "metadata/composeinfo.json",
}
"""
DNF Manager constants
"""
DNF_MODULE_MINIMAL_DISTRO_VERSIONS = [
{'distro': 'centos', 'min_version': 8},
{'distro': 'rhel', 'min_version': 8},
{'distro': 'fedora', 'min_version': 22},
]

92
plugins/module_utils/tripleo_repos/yum_config/dnf_manager.py
View File

@ -1,92 +0,0 @@
# Copyright 2021 Red Hat, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
from __future__ import (absolute_import, division, print_function)
import logging
__metaclass__ = type
class DnfModuleManager:
"""Class that manages dnf modules."""
def __init__(self):
# lazy import to allow CLI to start without dnf
import dnf
self.base = dnf.Base()
self.base.conf.read()
self.base.conf.best = True
self.base.read_all_repos()
self.base.fill_sack()
self.module_base = dnf.module.module_base.ModuleBase(self.base)
def _get_module_spec(self, name, stream=None, profile=None):
"""Return a module spec string based on stream and/or profile."""
module_spec = name
if stream:
module_spec += ':{0}'.format(stream)
if profile:
module_spec += '/{0}'.format(profile)
return module_spec
def _do_transaction(self):
"""Perform the resolved transaction."""
try:
self.base.do_transaction()
except RuntimeError:
logging.error('This command has to be run with superuser '
'privileges.')
raise
def enable_module(self, name, stream=None, profile=None):
"""Enable a module stream."""
self.module_base.enable(
[self._get_module_spec(name, stream=stream, profile=profile)]
)
self._do_transaction()
logging.info("Module %s was enabled.", name)
def disable_module(self, name, stream=None, profile=None):
"""Disable a module stream."""
self.module_base.disable(
[self._get_module_spec(name, stream=stream, profile=profile)]
)
self._do_transaction()
logging.info("Module %s was disabled.", name)
def reset_module(self, name, stream=None, profile=None):
"""Reset a module. It will no longer be enabled or disabled."""
self.module_base.reset(
[self._get_module_spec(name, stream=stream, profile=profile)]
)
self._do_transaction()
logging.info("Module %s was reset.", name)
def install_module(self, name, stream=None, profile=None):
"""Install packages of a module profile."""
self.module_base.install(
[self._get_module_spec(name, stream=stream, profile=profile)]
)
self.base.resolve()
self.base.download_packages(self.base.transaction.install_set)
self._do_transaction()
logging.info("Module %s was installed.", name)
def remove_module(self, name, stream=None, profile=None):
"""Remove packages of a module profile."""
self.module_base.remove(
[self._get_module_spec(name, stream=stream, profile=profile)]
)
self._do_transaction()
logging.info("Module %s was removed.", name)

76
plugins/module_utils/tripleo_repos/yum_config/exceptions.py
View File

@ -1,76 +0,0 @@
# Copyright 2021 Red Hat, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
#
#
from __future__ import (absolute_import, division, print_function)
__metaclass__ = type
class Base(Exception):
"""Base Exception class."""
class TripleOYumConfigNotFound(Base):
"""A configuration file was not found in the provided file path."""
def __init__(self, error_msg):
super(TripleOYumConfigNotFound, self).__init__(error_msg)
class TripleOYumConfigPermissionDenied(Base):
"""THh user has no permission to modify the configuration file."""
def __init__(self, error_msg):
super(TripleOYumConfigPermissionDenied, self).__init__(error_msg)
class TripleOYumConfigFileParseError(Base):
"""Encountered an error while parsing the configuration file."""
def __init__(self, error_msg):
super(TripleOYumConfigFileParseError, self).__init__(error_msg)
class TripleOYumConfigInvalidSection(Base):
"""The configuration file does not have the requested section.
This exception is raised when the expected section in the configuration
file does not exist and the class will not create a new one.
"""
def __init__(self, error_msg):
super(TripleOYumConfigInvalidSection, self).__init__(error_msg)
class TripleOYumConfigInvalidOption(Base):
"""One or more options are not valid for this configuration file."""
def __init__(self, error_msg):
super(TripleOYumConfigInvalidOption, self).__init__(error_msg)
class TripleOYumConfigComposeError(Base):
"""An error occurred while configuring CentOS compose repos."""
def __init__(self, error_msg):
super(TripleOYumConfigComposeError, self).__init__(error_msg)
class TripleOYumConfigUrlError(Base):
"""An error occurred while fetching repo from the url."""
def __init__(self, error_msg):
super(TripleOYumConfigUrlError, self).__init__(error_msg)

50
plugins/module_utils/tripleo_repos/yum_config/utils.py
View File

@ -1,50 +0,0 @@
# Copyright 2021 Red Hat, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
from __future__ import (absolute_import, division, print_function)
import os
import platform
import subprocess
__metaclass__ = type
# TODO(dviroel): Merge in a utils file when refactoring tripleo-repos.
def get_distro_info():
"""Get distro info from os-release file.
:return: distro_id, distro_major_version_id and distro_name
"""
if not os.path.exists('/etc/os-release'):
return platform.system(), 'unknown', 'unknown'
output = subprocess.Popen(
'source /etc/os-release && echo -e -n "$ID\n$VERSION_ID\n$NAME"',
shell=True,
stdout=subprocess.PIPE,
stderr=open(os.devnull, 'w'),
executable='/bin/bash',
universal_newlines=True).communicate()
# distro_id and distro_version_id will always be at least an empty string
distro_id, distro_version_id, distro_name = output[0].split('\n')
# if distro_version_id is empty string the major version will be empty
# string too
distro_major_version_id = distro_version_id.split('.')[0]
# check if that is UBI subcase?
if os.path.exists('/etc/yum.repos.d/ubi.repo'):
distro_id = 'ubi'
return distro_id, distro_major_version_id, distro_name

449
plugins/module_utils/tripleo_repos/yum_config/yum_config.py
View File

@ -1,449 +0,0 @@
# Copyright 2021 Red Hat, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
#
from __future__ import (absolute_import, division, print_function)
import io
import logging
import os
import subprocess
import sys
from .constants import (
YUM_GLOBAL_CONFIG_FILE_PATH,
YUM_REPO_DIR,
YUM_REPO_FILE_EXTENSION,
YUM_REPO_SUPPORTED_OPTIONS,
)
from .exceptions import (
TripleOYumConfigFileParseError,
TripleOYumConfigInvalidOption,
TripleOYumConfigInvalidSection,
TripleOYumConfigNotFound,
TripleOYumConfigUrlError,
)
try:
import tripleo_repos.utils as repos_utils
except ImportError:
import ansible_collections.tripleo.repos.plugins.module_utils.\
tripleo_repos.utils as repos_utils
py_version = sys.version_info.major
if py_version < 3:
import ConfigParser as cfg_parser
def save_section_to_file(file_path, config, section, updates):
"""Updates a specific 'section' in a 'config' and write to disk.
:param file_path: Absolute path to the file to be updated.
:param config: configparser object created from the file.
:param section: section name to be updated.
:param updates: dict with options to update in section.
"""
for k, v in updates.items():
config.set(section, k, v)
with open(file_path, 'w') as f:
config.write(f)
# NOTE(dviroel) Need to manually remove whitespaces around "=", to
# avoid legacy scripts failing on parsing ini files.
with open(file_path, 'r+') as f:
lines = f.readlines()
# erase content before writing again
f.truncate(0)
f.seek(0)
for line in lines:
line = line.strip()
if "=" in line:
option_kv = line.split("=", 1)
option_kv = list(map(str.strip, option_kv))
f.write("%s%s%s\n" % (option_kv[0], "=", option_kv[1]))
else:
f.write(line + "\n")
else:
import configparser as cfg_parser
def save_section_to_file(file_path, config, section, updates):
"""Updates a specific 'section' in a 'config' and write to disk.
:param file_path: Absolute path to the file to be updated.
:param config: configparser object created from the file.
:param section: section name to be updated.
:param updates: dict with options to update in section.
"""
config[section].update(updates)
with open(file_path, 'w') as f:
config.write(f, space_around_delimiters=False)
__metaclass__ = type
def validated_file_path(file_path):
if os.path.isfile(file_path) and os.access(file_path, os.W_OK):
return True
return False
def source_env_file(source_file, update=True):
"""Source a file and get all environment variables in a dict format."""
p_open = subprocess.Popen(". %s; env" % source_file,
stdout=subprocess.PIPE,
shell=True)
data = p_open.communicate()[0].decode('ascii')
env_dict = dict(
line.split("=", 1) for line in data.splitlines()
if len(line.split("=", 1)) > 1)
if update:
os.environ.update(env_dict)
return env_dict
class TripleOYumConfig:
"""
This class is a base class for updating yum configuration files in
ini format. The class validates the if the configuration files exists and
if it has the the permissions needed. A list of updatable options may be
provided to the class constructor.
"""
def __init__(self, valid_options=None, dir_path=None, file_extension=None,
environment_file=None):
"""
Creates a TripleOYumConfig object that holds configuration file
information.
:param valid_options: A list of options that can be updated on this
file.
:param dir_path: The directory path that this class can use to search
for configuration files to be updated.
:param: file_extension: File extension to filter configuration files
in the search directory.
:param environment_file: File to be read before updating environment
variables.
"""
self.dir_path = dir_path
self.file_extension = file_extension
self.valid_options = valid_options
self.env_file = environment_file
# Sanity checks
if dir_path:
if not os.path.isdir(dir_path):
msg = ('The configuration dir "{0}" was not found in the '
'provided path.').format(dir_path)
raise TripleOYumConfigNotFound(error_msg=msg)
if self.env_file:
source_env_file(os.path.expanduser(self.env_file), update=True)
def _read_config_file(self, file_path, section=None):
"""Reads a configuration file.
:param section: The name of the section that will be update. Only used
to fail earlier if the section is not found.
:return: a config parser object and the full file path.
"""
config = cfg_parser.ConfigParser()
file_paths = [file_path]
if self.dir_path:
# if dir_path is configured, we can search for filename there
file_paths.append(os.path.join(self.dir_path, file_path))
valid_file_path = None
for file in file_paths:
if validated_file_path(file):
valid_file_path = file
break
if not valid_file_path:
msg = ('The configuration file "{0}" was '
'not found.'.format(file_path))
raise TripleOYumConfigNotFound(error_msg=msg)
try:
config.read(valid_file_path)
except cfg_parser.Error:
msg = 'Unable to parse configuration file {0}.'.format(
valid_file_path)
raise TripleOYumConfigFileParseError(error_msg=msg)
if section and section not in config.sections():
msg = ('The provided section "{0}" was not found in the '
'configuration file {1}.').format(
section, valid_file_path)
raise TripleOYumConfigInvalidSection(error_msg=msg)
return config, valid_file_path
def _get_config_files(self, section):
"""Gets all configuration file paths for a given section.
This method will search for a 'section' name in all files inside the
configuration directory. All files with 'section' will be returned.
:param section: Section to be found inside configuration files.
:return: A list of config file paths.
"""
# Search for a configuration file that has the provided section
config_files_path = []
if section and self.dir_path:
for file in os.listdir(self.dir_path):
# Skip files that don't match the file extension or are not
# writable
if self.file_extension and not file.endswith(
self.file_extension):
continue
if not os.access(os.path.join(self.dir_path, file), os.W_OK):
continue
tmp_config = cfg_parser.ConfigParser()
try:
tmp_config.read(os.path.join(self.dir_path, file))
except cfg_parser.Error:
continue
if section in tmp_config.sections():
config_files_path.append(os.path.join(self.dir_path, file))
return config_files_path
def update_section(self, section, set_dict, file_path=None):
"""Updates a set of options of a section.
If a file path is not provided by the caller, this function will search
for the section in all files located in the working directory and
update each one of them.
:param section: Name of the section on the configuration file that will
be updated.
:param set_dict: Dict with all options and values to be updated in the
configuration file section.
:param file_path: Path to the configuration file to be updated.
"""
if self.valid_options:
if not all(key in self.valid_options for key in set_dict.keys()):
msg = 'One or more provided options are not valid.'
raise TripleOYumConfigInvalidOption(error_msg=msg)
files = [file_path] if file_path else self._get_config_files(section)
if not files:
msg = ('No configuration files were found for the provided '
'section {0}'.format(section))
raise TripleOYumConfigNotFound(error_msg=msg)
for k, v in set_dict.items():
set_dict[k] = os.path.expandvars(v)
for file in files:
config, file = self._read_config_file(file, section=section)
# Update configuration file with dict updates
save_section_to_file(file, config, section, set_dict)
logging.info("Section '%s' was successfully "
"updated.", section)
def add_section(self, section, add_dict, file_path):
""" Adds a new section with options in a provided config file.
:param section: Section name to be added to the config file.
:param add_dict: Dict with all options and values to be added into the
new section.
:param file_path: Path to the configuration file to be updated.
"""
if self.valid_options:
if not all(key in self.valid_options for key in add_dict.keys()):
msg = 'One or more provided options are not valid.'
raise TripleOYumConfigInvalidOption(error_msg=msg)
# This section shouldn't exist in the provided file
config, file_path = self._read_config_file(file_path=file_path)
if section in config.sections():
msg = ("Section '%s' already exists in the configuration "
"file.", section)
raise TripleOYumConfigInvalidSection(error_msg=msg)
for k, v in add_dict.items():
add_dict[k] = os.path.expandvars(v)
# Add new section
config.add_section(section)
# Update configuration file with dict updates
save_section_to_file(file_path, config, section, add_dict)
logging.info("Section '%s' was successfully "
"added.", section)
def update_all_sections(self, set_dict, file_path):
"""Updates all section of a given configuration file.
:param set_dict: Dict with all options and values to be updated in
the configuration file.
:param file_path: Path to the configuration file to be updated.
"""
if self.valid_options:
if not all(key in self.valid_options for key in set_dict.keys()):
msg = 'One or more provided options are not valid.'
raise TripleOYumConfigInvalidOption(error_msg=msg)
config, file_path = self._read_config_file(file_path)
for section in config.sections():
save_section_to_file(file_path, config, section, set_dict)
logging.info("All sections for '%s' were successfully "
"updated.", file_path)
def get_config_from_url(self, url):
content, status = repos_utils.http_get(url)
if status != 200:
msg = ("Invalid response code received from provided url: "
"{0}. Response code: {1}."
).format(url, status)
logging.error(msg)
raise TripleOYumConfigUrlError(error_msg=msg)
config = cfg_parser.ConfigParser()
if py_version < 3:
sfile = io.StringIO(content)
config.readfp(sfile)
else:
config.read_string(content)
return config
def get_options_from_url(self, url, section):
config = self.get_config_from_url(url)
if section not in config.sections():
msg = ("Section '{0}' was not found in the configuration file "
"provided by the url {1}.").format(section, url)
raise TripleOYumConfigInvalidSection(error_msg=msg)
return dict(config.items(section))
class TripleOYumRepoConfig(TripleOYumConfig):
"""Manages yum repo configuration files."""
def __init__(self, dir_path=None, environment_file=None):
conf_dir_path = dir_path or YUM_REPO_DIR
super(TripleOYumRepoConfig, self).__init__(
valid_options=YUM_REPO_SUPPORTED_OPTIONS,
dir_path=conf_dir_path,
file_extension=YUM_REPO_FILE_EXTENSION,
environment_file=environment_file)
def update_section(
self, section, set_dict=None, file_path=None, enabled=None,
from_url=None):
update_dict = (
self.get_options_from_url(from_url, section) if from_url else {})
if set_dict:
update_dict.update(set_dict)
if enabled is not None:
update_dict['enabled'] = '1' if enabled else '0'
if update_dict:
super(TripleOYumRepoConfig, self).update_section(
section, update_dict, file_path=file_path)
def add_section(self, section, add_dict, file_path, enabled=None,
from_url=None):
update_dict = (
self.get_options_from_url(from_url, section) if from_url else {})
update_dict.update(add_dict)
if enabled is not None:
update_dict['enabled'] = '1' if enabled else '0'
super(TripleOYumRepoConfig, self).add_section(
section, update_dict, file_path)
def add_or_update_section(self, section, set_dict=None,
file_path=None, enabled=None,
create_if_not_exists=True, from_url=None):
new_set_dict = (
self.get_options_from_url(from_url, section) if from_url else {})
new_set_dict.update(set_dict)
# make sure that it has a name
if 'name' not in new_set_dict.keys():
new_set_dict['name'] = section
# Try to update existing repos
try:
self.update_section(
section, set_dict=new_set_dict, file_path=file_path,
enabled=enabled)
except TripleOYumConfigNotFound:
if not create_if_not_exists or file_path is None:
# there is nothing to do, we can't create a new config file
raise
# Create a new file if it does not exists
with open(file_path, 'w+'):
pass
self.add_section(section, new_set_dict, file_path, enabled=enabled)
except TripleOYumConfigInvalidSection:
self.add_section(section, new_set_dict, file_path, enabled=enabled)
def add_or_update_all_sections_from_url(
self, from_url, file_path=None, set_dict=None, enabled=None,
create_if_not_exists=True):
"""Adds or updates all sections based on repo file from a URL."""
tmp_config = self.get_config_from_url(from_url)
if file_path is None:
# Build a file_path based on download url. If not compatible,
# don't fill file_path and let the code search for sections in all
# repo files inside config dir_path.
file_name = from_url.split('/')[-1]
if file_name.endswith(".repo"):
# Expecting a '*.repo' filename here, since the file can't be
# created with a different extension
file_path = os.path.join(self.dir_path, file_name)
for section in tmp_config.sections():
update_dict = dict(tmp_config.items(section))
update_dict.update(set_dict)
self.add_or_update_section(
section, set_dict=update_dict,
file_path=file_path, enabled=enabled,
create_if_not_exists=create_if_not_exists)
class TripleOYumGlobalConfig(TripleOYumConfig):
"""Manages yum global configuration file."""
def __init__(self, file_path=None, environment_file=None):
self.conf_file_path = file_path or YUM_GLOBAL_CONFIG_FILE_PATH
logging.info("Using '%s' as yum global configuration "
"file.", self.conf_file_path)
if file_path is not None:
# validate user provided file path
validated_file_path(file_path)
else:
# If there is no default 'yum.conf' configuration file, we need to
# create it. If the user specify another conf file that doesn't
# exists, the operation will fail.
if not os.path.isfile(self.conf_file_path):
config = cfg_parser.ConfigParser()
config.read(self.conf_file_path)
config.add_section('main')
with open(self.conf_file_path, 'w+') as file:
config.write(file)
super(TripleOYumGlobalConfig, self).__init__(
environment_file=environment_file)
def update_section(self, section, set_dict, file_path=None):
super(TripleOYumGlobalConfig, self).update_section(
section, set_dict, file_path=(file_path or self.conf_file_path))
def add_section(self, section, add_dict, file_path=None):
add_file_path = file_path or self.conf_file_path
super(TripleOYumGlobalConfig, self).add_section(
section, add_dict, add_file_path)

145
plugins/modules/get_hash.py
View File

@ -1,145 +0,0 @@
#!/usr/bin/python
# Copyright 2021 Red Hat, Inc.
# GNU General Public License v3.0+ (see COPYING or
# https://www.gnu.org/licenses/gpl-3.0.txt)
from __future__ import (absolute_import, division, print_function)
__metaclass__ = type
DOCUMENTATION = r'''
---
module: get_hash
short_description: Resolve rdo named tag to commit, full and distro hashes
version_added: "1.0.0"
description: ""
options:
os_version:
description: The operating system and version to fetch hashes for
required: false
type: str
default: centos8
release:
description: The release of OpenStack you want the hash info for
required: false
type: str
default: master
component:
description: The tripleo-ci component you are interested in
required: false
type: str
tag:
description: The named tag to fetch
required: false
type: str
default: current-tripleo
dlrn_url:
description: The url of the DLRN server to use for hash queries
required: false
type: str
default: https://trunk.rdoproject.org
author:
- Marios Andreou (@marios)
'''
EXAMPLES = r'''
- name: Get the latest hash info for victoria centos8 tripleo component
tripleo_get_hash:
os_version: centos8
release: victoria
component: tripleo
dlrn_url: 'https://foo.bar.baz'
'''
RETURN = r'''
full_hash:
description: The full hash that identifies the build
type: str
returned: always
sample: 'f47f1db5af04ddd1ab4cc3ccadf95884d335b3f3_92f50ace'
distro_hash:
description: The distro hash
type: str
returned: when available
sample: '92f50acecd0a218936b7163e8362e75913b62af2'
commit_hash:
description: The commit hash
type: str
returned: when available
sample: 'f47f1db5af04ddd1ab4cc3ccadf95884d335b3f3'
extended_hash:
description: The extended hash
type: str
returned: when available
sample: 'f47f1db5af04ddd1ab4cc3ccadf95884d335b3f3'
dlrn_url:
description: The dlrn server url from which hash info was collected.
type: str
returned: always
sample: 'https://trunk.rdoproject.org/centos8-master/current-tripleo/delorean.repo.md5' # noqa E501
'''
from ansible.module_utils.basic import AnsibleModule # noqa: E402
def run_module():
result = dict(
success=False,
changed=False,
error="",
)
argument_spec = dict(
os_version=dict(type='str', required=False, default='centos8'),
release=dict(type='str', required=False, default='master'),
component=dict(type='str', required=False, default=None),
tag=dict(type='str', required=False, default='current-tripleo'),
dlrn_url=dict(type='str',
required=False,
default='https://trunk.rdoproject.org'),
)
module = AnsibleModule(
argument_spec,
supports_check_mode=False
)
try:
from ansible_collections.tripleo.repos.plugins.module_utils.\
tripleo_repos.get_hash.tripleo_hash_info import TripleOHashInfo
os_version = module.params.get('os_version')
release = module.params.get('release')
component = module.params.get('component')
tag = module.params.get('tag')
dlrn_url = module.params.get('dlrn_url')
hash_result = TripleOHashInfo(os_version, release, component, tag,
config={'dlrn_url': dlrn_url})
result['commit_hash'] = hash_result.commit_hash
result['distro_hash'] = hash_result.distro_hash
result['full_hash'] = hash_result.full_hash
result['extended_hash'] = hash_result.extended_hash
result['dlrn_url'] = hash_result.dlrn_url
result['success'] = True
except Exception as exc:
result['error'] = str(exc)
result['msg'] = "Error something went wrong fetching hash info"
module.fail_json(**result)
module.exit_json(**result)
def main():
run_module()
if __name__ == '__main__':
main()

373
plugins/modules/yum_config.py
View File

@ -1,373 +0,0 @@
#!/usr/bin/python
# Copyright 2021 Red Hat, Inc.
# GNU General Public License v3.0+ (see COPYING or
# https://www.gnu.org/licenses/gpl-3.0.txt)
from __future__ import (absolute_import, division, print_function)
__metaclass__ = type
DOCUMENTATION = r'''
---
module: yum_config
short_description: Update yum configuration files for TripleO deployments.
version_added: "1.0.0"
description:
- Update specific options for different yum configuration files like
yum repos, yum modules and yum global configuration.
options:
type:
description:
- The type of yum configuration to be changed.
required: true
type: str
choices: [repo, module, global, 'enable-compose-repos']
name:
description:
- Name of the repo or module to be changed. This options is
mandatory only for 'repo' when no 'down_url' is provided. This
options is always mandatory for 'module' type.
type: str
enabled:
description:
- Change the yum repo or module to enabled or disabled.
- This options is ignored for yum global configuration.
type: bool
default: true
down_url:
description:
- URL of a downloadable repo file to be used as base to construct a
new repo file. When used together with 'name', will update only the
requested section, without a specific section 'name' will add or
update all sections available in the downloaded file.
type: str
operation:
description:
- Operation to be execute within a dnf module.
type: str
choices: [install, remove, reset]
stream:
description:
- Sets a module stream. This options is recommended when enabling a
module that doesn't have a default stream.
type: str
profile:
description:
- Sets a module profile. This options is recommended when installing
a module that doesn't have a default profile.
type: str
set_options:
description:
- Dictionary with options to be updated. All dictionary values must
be string or list of strings.
type: dict
file_path:
description:
- Absolute path of the configuration file to be changed.
type: path
dir_path:
description:
- Absolute path of the directory that contains the configuration
file to be changed.
type: path
default: /etc/yum.repos.d
environment_file:
description:
- Absolute path to an environment file to be read before updating or
creating yum config and repo files.
type: path
compose_url:
description:
- URL that contains CentOS compose repositories.
type: str
centos_release:
description:
- Target CentOS release.
type: str
choices: [centos-stream-8, centos-stream-9]
arch:
description:
- System architecture which the repos will be configure.
type: str
choices: [aarch64, ppc64le, x86_64]
default: x86_64
variants:
description:
- Repository variants that should be configured. If not provided,
all available variants will be configured.
type: list
elements: str
disable_conflicting_variants:
description:
- Disable all repos from the same directory that match variants'
name.
type: bool
default: false
disable_repos:
description:
- List with file path of repos that should be disabled after
successfully enabling all compose repos.
type: list
elements: str
author:
- Douglas Viroel (@viroel)
'''
EXAMPLES = r'''
# Set yum 'appstream' repo to enabled and exclude a list of packages
- name: Enable appstream repo and exclude nodejs and mariadb packages
become: true
become_user: root
tripleo_yum_config:
type: repo
name: appstream
enabled: true
set_options:
exclude:
- nodejs*
- mariadb*
# Enable and install a yum/dnf module
- name: Enable nginx module
become: true
become_user: root
tripleo_yum_config:
type: module
name: tomcat
enabled: false
stream: "1.18"
- name: Enable nginx module
become: true
become_user: root
tripleo_yum_config:
type: module
name: nginx
operation: install
profile: common
# Set yum global configuration options
- name: Set yum global options
become: true
become_user: root
tripleo_yum_config:
type: global
file_path: /etc/dnf/dnf.conf
set_options:
skip_if_unavailable: "False"
keepcache: "0"
- name: Configure a set of repos based on latest CentOS Stream 8 compose
become: true
become_user: root
tripleo_yum_config:
compose_url: https://composes.centos.org/latest-CentOS-Stream-8/compose/
centos_release: centos-stream-8
variants:
- AppStream
- BaseOS
disable_conflicting_variants: true
disable_repos:
- /etc/yum.repos.d/CentOS-Linux-AppStream.repo
- /etc/yum.repos.d/CentOS-Linux-BaseOS.repo
'''
RETURN = r''' # '''
import os # noqa: E402
from ansible.module_utils import six # noqa: E402
from ansible.module_utils.basic import AnsibleModule # noqa: E402
def run_module():
try:
import ansible_collections.tripleo.repos.plugins.module_utils. \
tripleo_repos.yum_config.constants as const
from ansible_collections.tripleo.repos.plugins.module_utils. \
tripleo_repos.yum_config import utils
except ImportError:
import tripleo_repos.yum_config.constants as const
from tripleo_repos.yum_config import utils
supported_config_types = ['repo', 'global', 'module',
'enable-compose-repos']
supported_module_operations = ['install', 'remove', 'reset']
module_args = dict(
type=dict(type='str', required=True, choices=supported_config_types),
name=dict(type='str'),
enabled=dict(type='bool', default=True),
down_url=dict(type='str'),
operation=dict(type='str', choices=supported_module_operations),
stream=dict(type='str'),
profile=dict(type='str'),
set_options=dict(type='dict', default={}),
file_path=dict(type='path'),
dir_path=dict(type='path', default=const.YUM_REPO_DIR),
environment_file=dict(type='path'),
compose_url=dict(type='str'),
centos_release=dict(type='str',
choices=const.COMPOSE_REPOS_RELEASES),
arch=dict(type='str', choices=const.COMPOSE_REPOS_SUPPORTED_ARCHS,
default='x86_64'),
variants=dict(type='list', default=[],
elements='str'),
disable_conflicting_variants=dict(type='bool', default=False),
disable_repos=dict(type='list', default=[],
elements='str'),
)
required_if_params = [
["type", "module", ["name"]],
["type", "enable-compose-repos", ["compose_url"]]
]
module = AnsibleModule(
argument_spec=module_args,
required_if=required_if_params,
supports_check_mode=False
)
operations_not_supp_in_py2 = ['module', 'enable-compose-repos']
if six.PY2 and module.params['type'] in operations_not_supp_in_py2:
msg = ("The configuration type '{0}' is not "
"supported with python 2.").format(module.params['type'])
module.fail_json(msg=msg)
if (module.params['type'] == 'repo' and not
module.params['name'] and not module.params['down_url']):
msg = ("When using configuration type '{0}' you must provide a repo "
"'name' or a 'down_url'.").format(module.params['type'])
module.fail_json(msg=msg)
distro, major_version, __ = utils.get_distro_info()
dnf_module_support = False
for min_distro_ver in const.DNF_MODULE_MINIMAL_DISTRO_VERSIONS:
if (distro == min_distro_ver.get('distro') and int(
major_version) >= min_distro_ver.get('min_version')):
dnf_module_support = True
break
if module.params['type'] == 'module' and not dnf_module_support:
msg = ("The configuration type 'module' is not "
"supported in this distro version "
"({0}-{1}).".format(distro, major_version))
module.fail_json(msg=msg)
# 'set_options' expects a dict that can also contains a list of values.
# List of elements will be converted to a comma-separated list
m_set_opts = module.params.get('set_options')
if m_set_opts:
for k, v in m_set_opts.items():
if isinstance(v, list):
m_set_opts[k] = ','.join([str(elem) for elem in v])
elif not isinstance(v, str):
m_set_opts[k] = str(v)
# Module execution
try:
try:
import ansible_collections.tripleo.repos.plugins.module_utils.\
tripleo_repos.yum_config.yum_config as cfg
except ImportError:
import tripleo_repos.yum_config.yum_config as cfg
if module.params['type'] == 'repo':
config_obj = cfg.TripleOYumRepoConfig(
dir_path=module.params['dir_path'],
environment_file=module.params['environment_file'])
if module.params['name']:
config_obj.add_or_update_section(
module.params['name'],
set_dict=m_set_opts,
file_path=module.params['file_path'],
enabled=module.params['enabled'],
from_url=module.params['down_url'])
else:
config_obj.add_or_update_all_sections_from_url(
module.params['down_url'],
set_dict=m_set_opts,
file_path=module.params['file_path'],
enabled=module.params['enabled'])
elif module.params['type'] == 'global':
config_obj = cfg.TripleOYumGlobalConfig(
file_path=module.params['file_path'],
environment_file=module.params['environment_file'])
config_obj.update_section('main', m_set_opts)
elif module.params['type'] == 'enable-compose-repos':
try:
import ansible_collections.tripleo.repos.plugins.module_utils.\
tripleo_repos.yum_config.compose_repos as repos
except ImportError:
import tripleo_repos.yum_config.compose_repos as repos
# 1. Create compose repo config object
repo_obj = repos.TripleOYumComposeRepoConfig(
module.params['compose_url'],
module.params['centos_release'],
dir_path=module.params['dir_path'],
arch=module.params['arch'],
environment_file=module.params['environment_file'])
# 2. enable CentOS compose repos
repo_obj.enable_compose_repos(
variants=module.params['variants'],
override_repos=module.params['disable_conflicting_variants'])
# 3. Disable all repos provided in disable_repos
for file in module.params['disable_repos']:
valid_path = None
rel_path = os.path.join(module.params['dir_path'], file)
if cfg.validated_file_path(file):
valid_path = file
elif cfg.validated_file_path(rel_path):
valid_path = rel_path
if valid_path is not None:
repo_obj.update_all_sections(valid_path, enabled=False)
elif module.params['type'] == 'module':
try:
import ansible_collections.tripleo.repos.plugins.module_utils.\
tripleo_repos.yum_config.dnf_manager as dnf_mgr
except ImportError:
import tripleo_repos.yum_config.dnf_manager as dnf_mgr
dnf_mod_mgr = dnf_mgr.DnfModuleManager()
if module.params['enabled']:
dnf_mod_mgr.enable_module(module.params['name'],
stream=module.params['stream'],
profile=module.params['profile'])
else:
dnf_mod_mgr.disable_module(module.params['name'],
stream=module.params['stream'],
profile=module.params['profile'])
if module.params['operation']:
dnf_method = getattr(dnf_mod_mgr,
module.params['operation'] + "_module")
dnf_method(module.params['name'],
stream=module.params['stream'],
profile=module.params['profile'])
except Exception as exc:
module.fail_json(msg=str(exc))
# Successful module execution
result = {
'changed': True,
'msg': "Yum {0} configuration was successfully updated.".format(
module.params['type'])
}
module.exit_json(**result)
def main():
run_module()
if __name__ == '__main__':
main()

5
releasenotes/notes/stream_by_default-783cc6a2208b942c.yaml
View File

@ -1,5 +0,0 @@
---
features:
- |
tripleo-repos now runs with the --stream argument enabled by default. Use
the --no-stream argument when using tripleo-repos without CentOS stream.

7
requirements.txt
View File

@ -1,7 +0,0 @@
# The order of packages is significant, because pip processes them in the order
# of appearance. Changing the order has an impact on the overall integration
# process, which may cause wedges in the gate later.
pbr!=2.1.0,>=2.0.0 # Apache-2.0
requests>=2.10.0 # Apache-2.0
PyYAML>=3.12 # MIT

40
setup.cfg
View File

@ -1,40 +0,0 @@
[metadata]
name = tripleo-repos
summary = A tool for managing tripleo repos
description-file =
README.rst
author = OpenStack
author-email = openstack-discuss@lists.openstack.org
home-page = https://docs.openstack.org/tripleo-docs/latest/
classifier =
Environment :: OpenStack
Intended Audience :: Information Technology
Intended Audience :: System Administrators
License :: OSI Approved :: Apache Software License
Operating System :: POSIX :: Linux
Programming Language :: Python
Programming Language :: Python :: Implementation :: CPython
Programming Language :: Python :: 3 :: Only
Programming Language :: Python :: 3
Programming Language :: Python :: 3.6
Programming Language :: Python :: 3.7
Programming Language :: Python :: 3.8
Programming Language :: Python :: 3.9
[files]
packages =
tripleo_repos
data_files =
etc/tripleo_get_hash/ = tripleo_repos/get_hash/config.yaml
share/ansible/plugins/modules/ = plugins/modules/*
share/ansible/plugins/module_utils/ = plugins/module_utils/*
[entry_points]
console_scripts =
tripleo-repos = tripleo_repos.main:main
tripleo-yum-config = tripleo_repos.yum_config.__main__:cli_entrypoint
tripleo-get-hash = tripleo_repos.get_hash.__main__:cli_entrypoint
[pbr]
skip_authors = True
skip_changelog = True

20
setup.py
View File

@ -1,20 +0,0 @@
# Copyright (c) 2013 Hewlett-Packard Development Company, L.P.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import setuptools
setuptools.setup(
setup_requires=['pbr>=2.0'],
pbr=True)

15
test-requirements.txt
View File

@ -1,15 +0,0 @@
# The order of packages is significant, because pip processes them in the order
# of appearance. Changing the order has an impact on the overall integration
# process, which may cause wedges in the gate later.
hacking>=3.0,<=3.1.0 # Apache-2.0
coverage>=4.0,!=4.4 # Apache-2.0
ddt>=1.0.1 # MIT
python-subunit>=0.0.18 # Apache-2.0/BSD
oslotest>=1.10.0 # Apache-2.0
testrepository>=0.0.18 # Apache-2.0/BSD
testscenarios>=0.4 # Apache-2.0/BSD
testtools>=1.4.0 # MIT
stestr>=2.0.0 # Apache-2.0
fixtures>=3.0.0 # Apache-2.0/BSD

1
tests/sanity/ignore-2.10.txt
View File

@ -1 +0,0 @@
ignore.txt

1
tests/sanity/ignore-2.11.txt
View File

@ -1 +0,0 @@
ignore.txt

1
tests/sanity/ignore-2.12.txt
View File

@ -1 +0,0 @@
ignore.txt

1
tests/sanity/ignore-2.9.txt
View File

@ -1 +0,0 @@
ignore.txt

3
tests/sanity/ignore.txt
View File

@ -1,3 +0,0 @@
plugins/module_utils/tripleo_repos/utils.py replace-urlopen
plugins/module_utils/tripleo_repos/utils.py pylint:ansible-bad-import
plugins/module_utils/tripleo_repos/yum_config/compose_repos.py replace-urlopen

1
tests/sanity/requirements.txt
View File

@ -1 +0,0 @@
PyYAML

0
tests/unit/__init__.py
View File

0
tests/unit/get_hash/__init__.py
View File

115
tests/unit/get_hash/fakes.py
View File

@ -1,115 +0,0 @@
# Copyright 2021 Red Hat, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
#
#
TEST_COMMIT_YAML_COMPONENT = """
commits:
- artifacts: repos/component/common/47/6a/476a52df13202a44336c8b01419f8b73b93d93eb_1f5a41f3/openstack-tacker-4.1.0-0.20210325043415.476a52d.el8.src.rpm,repos/component/common/47/6a/476a52df13202a44336c8b01419f8b73b93d93eb_1f5a41f3/python3-tacker-doc-4.1.0-0.20210325043415.476a52d.el8.noarch.rpm,repos/component/common/47/6a/476a52df13202a44336c8b01419f8b73b93d93eb_1f5a41f3/python3-tacker-tests-4.1.0-0.20210325043415.476a52d.el8.noarch.rpm,repos/component/common/47/6a/476a52df13202a44336c8b01419f8b73b93d93eb_1f5a41f3/openstack-tacker-common-4.1.0-0.20210325043415.476a52d.el8.noarch.rpm,repos/component/common/47/6a/476a52df13202a44336c8b01419f8b73b93d93eb_1f5a41f3/python3-tacker-4.1.0-0.20210325043415.476a52d.el8.noarch.rpm,repos/component/common/47/6a/476a52df13202a44336c8b01419f8b73b93d93eb_1f5a41f3/openstack-tacker-4.1.0-0.20210325043415.476a52d.el8.noarch.rpm
civotes: '[]'
commit_branch: master
commit_hash: 476a52df13202a44336c8b01419f8b73b93d93eb
component: common
distgit_dir: /home/centos8-master-uc/data/openstack-tacker_distro/
distro_hash: 1f5a41f31db8e3eb51caa9c0e201ab0583747be8
dt_build: '1616646776'
dt_commit: '1616646661.0'
dt_distro: '1616411951'
dt_extended: '0'
extended_hash: None
flags: '0'
id: '21047'
notes: OK
project_name: openstack-tacker
promotions: '[]'
repo_dir: /home/centos8-master-uc/data/openstack-tacker
status: SUCCESS
type: rpm
""" # noqa
TEST_COMMIT_YAML_CENTOS_7 = """
commits:
- artifacts: repos/b5/ef/b5ef03c9c939db551b03e9490edc6981ff582035_76ebc465/openstack-tripleo-heat-templates-12.1.1-0.20200227052810.b5ef03c.el7.src.rpm,repos/b5/ef/b5ef03c9c939db551b03e9490edc6981ff582035_76ebc465/openstack-tripleo-heat-templates-12.1.1-0.20200227052810.b5ef03c.el7.noarch.rpm
commit_branch: master
commit_hash: b5ef03c9c939db551b03e9490edc6981ff582035
component: None
distgit_dir: /home/centos-master-uc/data/openstack-tripleo-heat-templates_distro/
distro_hash: 76ebc4655502820b7677579349fd500eeca292e6
dt_build: '1582781227'
dt_commit: '1582780705.0'
dt_distro: '1580409403'
dt_extended: '0'
extended_hash: None
flags: '0'
id: '86894'
notes: OK
project_name: openstack-tripleo-heat-templates
repo_dir: /home/centos-master-uc/data/openstack-tripleo-heat-templates
status: SUCCESS
type: rpm
""" # noqa
TEST_REPO_MD5 = 'a96366960d5f9b08f78075b7560514e7'
BAD_CONFIG_FILE = """
awoo: 'foo'
"""
CONFIG_FILE = """
dlrn_url: 'https://trunk.rdoproject.org'
tripleo_releases:
- master
- zed
- wallaby
- victoria
- ussuri
- train
- osp16-2
- osp17
tripleo_ci_components:
- baremetal
- cinder
- clients
- cloudops
- common
- compute
- glance
- manila
- network
- octavia
- security
- swift
- tempest
- tripleo
- ui
- validation
rdo_named_tags:
- current
- consistent
- component-ci-testing
- tripleo-ci-testing
- current-tripleo
- current-tripleo-rdo
os_versions:
- centos7
- centos8
- centos9
- rhel8
- rhel9
"""

231
tests/unit/get_hash/test_tripleo_get_hash.py
View File

@ -1,231 +0,0 @@
# Copyright 2021 Red Hat, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
#
#
import sys
import unittest
from unittest import mock
from unittest.mock import mock_open, MagicMock, patch
import yaml
import tripleo_repos.get_hash.exceptions as exc
import tripleo_repos.get_hash.__main__ as tgh
from . import fakes as test_fakes
@mock.patch(
'builtins.open', new_callable=mock_open, read_data=test_fakes.CONFIG_FILE
)
class TestGetHash(unittest.TestCase):
"""In this class we test the CLI invocations for this module.
The builtin 'open' function is mocked at a
class level so we can mock the config.yaml with the contents of the
fakes.CONFIG_FILE
"""
def test_centos_8_current_tripleo_stable(self, mock_config):
mocked = MagicMock(
return_value=(test_fakes.TEST_REPO_MD5, 200))
with patch(
'tripleo_repos.get_hash.tripleo_hash_info.http_get', mocked):
args = ['--os-version', 'centos8', '--release', 'victoria']
sys.argv[1:] = args
main_res = tgh.main()
self.assertEqual(main_res.full_hash, test_fakes.TEST_REPO_MD5)
self.assertEqual(
'https://trunk.rdoproject.org/centos8-victoria/current-tripleo/delorean.repo.md5', # noqa
main_res.dlrn_url,
)
self.assertEqual('centos8', main_res.os_version)
self.assertEqual('victoria', main_res.release)
# TODO(marios) reenable https://bugs.launchpad.net/tripleo/+bug/2002112
# def test_verbose_logging_on(self, mock_config):
# args = ['--verbose']
# debug_msgs = []
#
# mocked = MagicMock(
# return_value=(test_fakes.TEST_REPO_MD5, 200))
# with patch(
# 'tripleo_repos.get_hash.tripleo_hash_info.http_get', mocked):
# with self.assertLogs() as captured:
# sys.argv[1:] = args
# tgh.main()
# debug_msgs = [
# record.message
# for record in captured.records
# if record.levelname == 'DEBUG'
# ]
# self.assertIn('Logging level set to DEBUG', debug_msgs)
#
# def test_verbose_logging_off(self, mock_config):
# debug_msgs = []
#
# mocked = MagicMock(
# return_value=(test_fakes.TEST_REPO_MD5, 200))
# with patch(
# 'tripleo_repos.get_hash.tripleo_hash_info.http_get', mocked):
#
# args = ['--tag', 'current-tripleo', '--os-version', 'centos8']
# with self.assertLogs() as captured:
# sys.argv[1:] = args
# tgh.main()
# debug_msgs = [
# record.message
# for record in captured.records
# if record.levelname == 'DEBUG'
# ]
# self.assertEqual(debug_msgs, [])
def test_invalid_unknown_components(self, mock_config):
args = ['--component', 'nosuchcomponent']
sys.argv[1:] = args
self.assertRaises(SystemExit, lambda: tgh.main())
def test_valid_tripleo_ci_components(self, mock_config):
config_file = open("fake_config_file") # open is mocked at class level
config_yaml = yaml.safe_load(config_file.read())
config_file.close()
# interate for each of config components
for component in config_yaml['tripleo_ci_components']:
mocked = MagicMock(
return_value=(test_fakes.TEST_COMMIT_YAML_COMPONENT, 200))
with patch(
'tripleo_repos.get_hash.tripleo_hash_info.http_get',
mocked):
args = ['--component', "{}".format(component)]
sys.argv[1:] = args
main_res = tgh.main()
self.assertEqual(
"https://trunk.rdoproject.org/centos8-master/component"
"/{}/current-tripleo/commit.yaml".format(
component
),
main_res.dlrn_url,
)
self.assertEqual("{}".format(component), main_res.component)
def test_invalid_component_centos7(self, mock_config):
args = ['--os-version', 'centos7', '--component', 'tripleo']
sys.argv[1:] = args
self.assertRaises(exc.TripleOHashInvalidParameter, lambda: tgh.main())
def test_valid_os_version(self, mock_config):
config_file = open("fake_config_file") # open is mocked at class level
config_yaml = yaml.safe_load(config_file.read())
config_file.close()
# interate for each supported os_version
for os_v in config_yaml['os_versions']:
if '7' in os_v:
mocked = MagicMock(
return_value=(test_fakes.TEST_COMMIT_YAML_CENTOS_7, 200))
expected_url = (
"https://trunk.rdoproject.org/{}-master/"
"current-tripleo/commit.yaml".format(os_v)
)
else:
mocked = MagicMock(
return_value=(test_fakes.TEST_REPO_MD5, 200))
expected_url = (
"https://trunk.rdoproject.org/{}-master/"
"current-tripleo/delorean.repo.md5".format(os_v)
)
with patch(
'tripleo_repos.get_hash.tripleo_hash_info.http_get',
mocked):
args = ['--os-version', "{}".format(os_v)]
sys.argv[1:] = args
main_res = tgh.main()
self.assertEqual(main_res.dlrn_url, expected_url)
self.assertEqual("{}".format(os_v), main_res.os_version)
def test_invalid_os_version(self, mock_config):
args = ['--os-version', 'rhelos99', '--component', 'tripleo']
sys.argv[1:] = args
self.assertRaises(SystemExit, lambda: tgh.main())
def test_invalid_unknown_tag(self, mock_config):
args = ['--tag', 'nosuchtag']
sys.argv[1:] = args
self.assertRaises(SystemExit, lambda: tgh.main())
def test_valid_rdo_named_tags(self, mock_config):
config_file = open("fake_config_file") # open is mocked at class level
config_yaml = yaml.safe_load(config_file.read())
config_file.close()
# iterate for each of config named tags
for tag in config_yaml['rdo_named_tags']:
mocked = MagicMock(
return_value=(test_fakes.TEST_REPO_MD5, 200))
with patch(
'tripleo_repos.get_hash.tripleo_hash_info.http_get',
mocked):
args = ['--tag', "{}".format(tag)]
sys.argv[1:] = args
main_res = tgh.main()
self.assertEqual(
"https://trunk.rdoproject.org/centos8-master"
"/{}/delorean.repo.md5".format(
tag
),
main_res.dlrn_url,
)
self.assertEqual(tag, main_res.tag)
def test_override_dlrn_url(self, mock_config):
mocked = MagicMock(
return_value=(test_fakes.TEST_REPO_MD5, 200))
with patch(
'tripleo_repos.get_hash.tripleo_hash_info.http_get',
mocked):
args = ['--dlrn-url', 'https://awoo.com/awoo']
sys.argv[1:] = args
main_res = tgh.main()
self.assertEqual(
"https://awoo.com/awoo/centos8-master/current-tripleo"
"/delorean.repo.md5",
main_res.dlrn_url,
)
def test_override_os_version_release_rhel8(self, mock_config):
mocked = MagicMock(
return_value=(test_fakes.TEST_REPO_MD5, 200))
with patch(
'tripleo_repos.get_hash.tripleo_hash_info.http_get',
mocked):
args = [
'--dlrn-url',
'https://awoo.com/awoo',
'--os-version',
'rhel8',
'--release',
'osp16-2',
]
sys.argv[1:] = args
main_res = tgh.main()
self.assertEqual('rhel8', main_res.os_version)
self.assertEqual('osp16-2', main_res.release)
self.assertEqual(
"https://awoo.com/awoo/rhel8-osp16-2/current-tripleo"
"/delorean.repo.md5", main_res.dlrn_url,
)
if __name__ == '__main__':
unittest.main()

227
tests/unit/get_hash/test_tripleo_get_hash_info.py
View File

@ -1,227 +0,0 @@
# Copyright 2021 Red Hat, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
#
#
import unittest
import tripleo_repos.get_hash.tripleo_hash_info as thi
import tripleo_repos.get_hash.exceptions as exc
from . import fakes as test_fakes
from unittest import mock
from unittest.mock import mock_open, MagicMock, patch
@mock.patch(
'builtins.open', new_callable=mock_open, read_data=test_fakes.CONFIG_FILE
)
class TestGetHashInfo(unittest.TestCase):
"""In this class we test the functions and instantiation of the
TripleOHashInfo class. The builtin 'open' function is mocked at a
class level so we can mock the config.yaml with the contents of the
fakes.CONFIG_FILE
"""
def test_hashes_from_commit_yaml(self, mock_config):
sample_commit_yaml = test_fakes.TEST_COMMIT_YAML_COMPONENT
expected_result = (
'476a52df13202a44336c8b01419f8b73b93d93eb_1f5a41f3',
'476a52df13202a44336c8b01419f8b73b93d93eb',
'1f5a41f31db8e3eb51caa9c0e201ab0583747be8',
'None',
)
mocked = MagicMock(
return_value=(test_fakes.TEST_COMMIT_YAML_COMPONENT, 200))
with patch(
'tripleo_repos.get_hash.tripleo_hash_info.http_get', mocked):
mock_hash_info = thi.TripleOHashInfo(
'centos8', 'master', 'common', 'current-tripleo'
)
actual_result = mock_hash_info._hashes_from_commit_yaml(
sample_commit_yaml
)
self.assertEqual(expected_result, actual_result)
def test_resolve_repo_url_component_commit_yaml(self, mock_config):
mocked = MagicMock(
return_value=(test_fakes.TEST_COMMIT_YAML_COMPONENT, 200))
with patch(
'tripleo_repos.get_hash.tripleo_hash_info.http_get', mocked):
c8_component_hash_info = thi.TripleOHashInfo(
'centos8', 'master', 'common', 'current-tripleo'
)
repo_url = c8_component_hash_info._resolve_repo_url("https://woo")
self.assertEqual(
repo_url,
'https://woo/centos8-master/component/common/current-tripleo/commit.yaml', # noqa
)
def test_resolve_repo_url_centos8_repo_md5(self, mock_config):
mocked = MagicMock(
return_value=(test_fakes.TEST_REPO_MD5, 200))
with patch(
'tripleo_repos.get_hash.tripleo_hash_info.http_get', mocked):
c8_hash_info = thi.TripleOHashInfo(
'centos8', 'master', None, 'current-tripleo'
)
repo_url = c8_hash_info._resolve_repo_url("https://woo")
self.assertEqual(
repo_url, 'https://woo/centos8-master/current-tripleo/delorean.repo.md5' # noqa
)
def test_resolve_repo_url_centos7_commit_yaml(self, mock_config):
mocked = MagicMock(
return_value=(test_fakes.TEST_COMMIT_YAML_CENTOS_7, 200))
with patch(
'tripleo_repos.get_hash.tripleo_hash_info.http_get', mocked):
c7_hash_info = thi.TripleOHashInfo(
'centos7', 'master', None, 'current-tripleo'
)
repo_url = c7_hash_info._resolve_repo_url("https://woo")
self.assertEqual(
repo_url, 'https://woo/centos7-master/current-tripleo/commit.yaml' # noqa
)
def test_get_tripleo_hash_info_centos8_md5(self, mock_config):
mocked = MagicMock(
return_value=(test_fakes.TEST_REPO_MD5, 200))
with patch(
'tripleo_repos.get_hash.tripleo_hash_info.http_get', mocked):
created_hash_info = thi.TripleOHashInfo(
'centos8', 'master', None, 'current-tripleo'
)
self.assertIsInstance(created_hash_info, thi.TripleOHashInfo)
self.assertEqual(
created_hash_info.full_hash, test_fakes.TEST_REPO_MD5
)
self.assertEqual(created_hash_info.tag, 'current-tripleo')
self.assertEqual(created_hash_info.os_version, 'centos8')
self.assertEqual(created_hash_info.release, 'master')
def test_get_tripleo_hash_info_component(self, mock_config):
expected_commit_hash = '476a52df13202a44336c8b01419f8b73b93d93eb'
expected_distro_hash = '1f5a41f31db8e3eb51caa9c0e201ab0583747be8'
expected_full_hash = '476a52df13202a44336c8b01419f8b73b93d93eb_1f5a41f3' # noqa
mocked = MagicMock(
return_value=(test_fakes.TEST_COMMIT_YAML_COMPONENT, 200))
with patch(
'tripleo_repos.get_hash.tripleo_hash_info.http_get', mocked):
created_hash_info = thi.TripleOHashInfo(
'centos8', 'victoria', 'common', 'tripleo-ci-testing'
)
self.assertIsInstance(created_hash_info, thi.TripleOHashInfo)
self.assertEqual(created_hash_info.full_hash, expected_full_hash)
self.assertEqual(
created_hash_info.distro_hash, expected_distro_hash
)
self.assertEqual(
created_hash_info.commit_hash, expected_commit_hash
)
self.assertEqual(created_hash_info.component, 'common')
self.assertEqual(created_hash_info.tag, 'tripleo-ci-testing')
self.assertEqual(created_hash_info.release, 'victoria')
def test_get_tripleo_hash_info_centos7_commit_yaml(self, mock_config):
expected_commit_hash = 'b5ef03c9c939db551b03e9490edc6981ff582035'
expected_distro_hash = '76ebc4655502820b7677579349fd500eeca292e6'
expected_full_hash = 'b5ef03c9c939db551b03e9490edc6981ff582035_76ebc465' # noqa
mocked = MagicMock(
return_value=(test_fakes.TEST_COMMIT_YAML_CENTOS_7, 200))
with patch(
'tripleo_repos.get_hash.tripleo_hash_info.http_get', mocked):
created_hash_info = thi.TripleOHashInfo(
'centos7', 'master', None, 'tripleo-ci-testing'
)
self.assertIsInstance(created_hash_info, thi.TripleOHashInfo)
self.assertEqual(created_hash_info.full_hash, expected_full_hash)
self.assertEqual(
created_hash_info.distro_hash, expected_distro_hash
)
self.assertEqual(
created_hash_info.commit_hash, expected_commit_hash
)
self.assertEqual(created_hash_info.os_version, 'centos7')
def test_bad_config_file(self, mock_config):
mocked = MagicMock(
return_value=test_fakes.TEST_COMMIT_YAML_CENTOS_7)
with patch(
'tripleo_repos.get_hash.tripleo_hash_info.http_get', mocked):
with mock.patch(
'builtins.open',
new_callable=mock_open,
read_data=test_fakes.BAD_CONFIG_FILE,
):
self.assertRaises(
exc.TripleOHashInvalidConfig,
thi.TripleOHashInfo,
'centos7',
'master',
None,
'tripleo-ci-testing',
)
def test_override_config_dlrn_url(self, mock_config):
expected_dlrn_url = 'https://foo.bar.baz/centos8-master/component/common/current-tripleo/commit.yaml' # noqa
mocked = MagicMock(
return_value=(test_fakes.TEST_COMMIT_YAML_COMPONENT, 200))
with patch(
'tripleo_repos.get_hash.tripleo_hash_info.http_get', mocked):
mock_hash_info = thi.TripleOHashInfo(
'centos8', 'master', 'common', 'current-tripleo',
{'dlrn_url': 'https://foo.bar.baz'}
)
self.assertEqual(expected_dlrn_url, mock_hash_info.dlrn_url)
def test_override_config_dlrn_url_empty_ignored(self, mock_config):
expected_dlrn_url = 'https://trunk.rdoproject.org/centos8-master/component/common/current-tripleo/commit.yaml' # noqa
mocked = MagicMock(
return_value=(test_fakes.TEST_COMMIT_YAML_COMPONENT, 200))
with patch(
'tripleo_repos.get_hash.tripleo_hash_info.http_get', mocked):
mock_hash_info = thi.TripleOHashInfo(
'centos8', 'master', 'common', 'current-tripleo',
{'dlrn_url': ''}
)
self.assertEqual(expected_dlrn_url, mock_hash_info.dlrn_url)
def test_404_dlrn_http_status_code(self, mock_config):
bad_dlrn_url = 'https://server.ok/centos8-master/component/common/current-tripleo/commit.yaml' # noqa
response_text_404 = "Some kind of 404 text NOT FOUND!"
mocked = MagicMock(
return_value=(response_text_404, 404))
with patch(
'tripleo_repos.get_hash.tripleo_hash_info.http_get', mocked):
with self.assertLogs() as captured:
self.assertRaises(
exc.TripleOHashInvalidDLRNResponse,
thi.TripleOHashInfo,
'centos8',
'master',
'common',
'current-tripleo',
{'dlrn_url': 'https://server.ok'},
)
debug_msgs = [
record.message
for record in captured.records
if record.levelname == 'ERROR'
]
error_str = (
"Invalid response received from the delorean server. Queried "
"URL: {0}. Response code: {1}. Response text: {2}. Failed to "
"create TripleOHashInfo object."
).format(bad_dlrn_url, '404', response_text_404)
self.assertIn(error_str, debug_msgs)

0
tests/unit/tripleo_repos/__init__.py
View File

773
tests/unit/tripleo_repos/test_main.py
View File

@ -1,773 +0,0 @@
# Copyright 2016 Red Hat, Inc.
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import subprocess
import sys
from unittest import mock
import ddt
import testtools
from tripleo_repos import main
@ddt.ddt
class TestTripleORepos(testtools.TestCase):
@mock.patch('tripleo_repos.main._get_distro')
@mock.patch('sys.argv', ['tripleo-repos', 'current', '-d', 'centos8'])
@mock.patch('tripleo_repos.main._run_pkg_clean')
@mock.patch('tripleo_repos.main._validate_args')
@mock.patch('tripleo_repos.main._get_base_path')
@mock.patch('tripleo_repos.main._remove_existing')
@mock.patch('tripleo_repos.main._install_repos')
def test_main_centos8(self, mock_install, mock_remove, mock_gbp,
mock_validate, mock_clean, mock_distro):
mock_distro.return_value = ('centos', '8', 'CentOS 8')
args = main._parse_args('centos', '8')
mock_path = mock.Mock()
mock_gbp.return_value = mock_path
main.main()
mock_validate.assert_called_once_with(args, 'CentOS 8', '8')
mock_gbp.assert_called_once_with(args)
mock_remove.assert_called_once_with(args)
mock_clean.assert_called_once_with('centos8')
@mock.patch('tripleo_repos.main._get_distro')
@mock.patch('sys.argv', ['tripleo-repos', 'current', '-d', 'fedora'])
@mock.patch('tripleo_repos.main._run_pkg_clean')
@mock.patch('tripleo_repos.main._validate_args')
@mock.patch('tripleo_repos.main._get_base_path')
@mock.patch('tripleo_repos.main._install_priorities')
@mock.patch('tripleo_repos.main._remove_existing')
@mock.patch('tripleo_repos.main._install_repos')
def test_main_fedora(self, mock_install, mock_remove, mock_ip, mock_gbp,
mock_validate, mock_clean, mock_distro):
mock_distro.return_value = ('centos', '8', 'CentOS 8')
args = main._parse_args('centos', '8')
mock_path = mock.Mock()
mock_gbp.return_value = mock_path
main.main()
mock_validate.assert_called_once_with(args, 'CentOS 8', '8')
mock_gbp.assert_called_once_with(args)
assert not mock_ip.called, '_install_priorities should no tbe called'
mock_remove.assert_called_once_with(args)
mock_install.assert_called_once_with(args, mock_path)
mock_clean.assert_called_once_with('fedora')
@mock.patch('requests.get')
def test_get_repo(self, mock_get):
mock_response = mock.Mock()
mock_response.status_code = 200
mock_response.text = '88MPH'
mock_get.return_value = mock_response
fake_addr = 'http://lone/pine/mall'
args = mock.Mock()
args.distro = 'centos'
content = main._get_repo(fake_addr, args)
self.assertEqual('88MPH', content)
mock_get.assert_called_once_with(fake_addr)
@mock.patch('requests.get')
def test_get_repo_404(self, mock_get):
mock_response = mock.Mock()
mock_response.status_code = 404
mock_get.return_value = mock_response
fake_addr = 'http://twin/pines/mall'
main._get_repo(fake_addr, mock.Mock())
mock_get.assert_called_once_with(fake_addr)
mock_response.raise_for_status.assert_called_once_with()
@mock.patch('os.listdir')
@mock.patch('os.remove')
@mock.patch('os.path.exists')
def test_remove_existing(self, mock_exists, mock_remove, mock_listdir):
fake_list = ['foo.repo', 'delorean.repo',
'delorean-current-tripleo.repo',
'tripleo-centos-opstools.repo',
'tripleo-centos-highavailability.repo']
mock_exists.return_value = [True, False, True, False, True,
False, False, True]
mock_listdir.return_value = fake_list
mock_args = mock.Mock()
mock_args.output_path = '/etc/yum.repos.d'
main._remove_existing(mock_args)
self.assertIn(mock.call('/etc/yum.repos.d/delorean.repo'),
mock_remove.mock_calls)
self.assertIn(mock.call('/etc/yum.repos.d/'
'delorean-current-tripleo.repo'),
mock_remove.mock_calls)
self.assertIn(
mock.call('/etc/yum.repos.d/tripleo-centos-opstools.repo'),
mock_remove.mock_calls)
self.assertIn(
mock.call('/etc/distro.repos.d/'
'tripleo-centos-highavailability.repo'),
mock_remove.mock_calls)
self.assertNotIn(mock.call('/etc/yum.repos.d/foo.repo'),
mock_remove.mock_calls)
# There is no $DISTRO single path anymore, every path has branch
# specification, even master
def test_get_base_path(self):
args = mock.Mock()
args.branch = 'master'
args.distro = 'centos7'
args.rdo_mirror = 'http://trunk.rdoproject.org'
path = main._get_base_path(args)
self.assertEqual('http://trunk.rdoproject.org/centos7-master/', path)
def test_get_base_path_fedora(self):
args = mock.Mock()
args.branch = 'master'
args.distro = 'fedora'
args.rdo_mirror = 'http://trunk.rdoproject.org'
path = main._get_base_path(args)
self.assertEqual('http://trunk.rdoproject.org/fedora-master/', path)
@mock.patch('subprocess.check_call')
def test_install_priorities(self, mock_check_call):
main._install_priorities()
mock_check_call.assert_called_once_with(['yum', 'install', '-y',
'yum-plugin-priorities'])
@mock.patch('subprocess.check_call')
def test_install_priorities_fails(self, mock_check_call):
mock_check_call.side_effect = subprocess.CalledProcessError(88, '88')
self.assertRaises(subprocess.CalledProcessError,
main._install_priorities)
@mock.patch('tripleo_repos.main._get_repo')
@mock.patch('tripleo_repos.main._write_repo')
def test_install_repos_current(self, mock_write, mock_get):
args = mock.Mock()
args.repos = ['current']
args.branch = 'master'
args.output_path = 'test'
args.distro = 'fake'
mock_get.return_value = '[delorean]\nMr. Fusion'
main._install_repos(args, 'roads/')
self.assertEqual([mock.call('roads/current/delorean.repo', args),
mock.call('roads/delorean-deps.repo', args),
],
mock_get.mock_calls)
self.assertEqual([mock.call('[delorean]\nMr. Fusion', 'test',
name='delorean'),
mock.call('[delorean]\nMr. Fusion', 'test'),
],
mock_write.mock_calls)
@mock.patch('tripleo_repos.main._get_repo')
@mock.patch('tripleo_repos.main._write_repo')
def test_install_repos_current_mitaka(self, mock_write, mock_get):
args = mock.Mock()
args.repos = ['current']
args.branch = 'mitaka'
args.output_path = 'test'
args.distro = 'fake'
mock_get.return_value = '[delorean]\nMr. Fusion'
main._install_repos(args, 'roads/')
self.assertEqual([mock.call('roads/current/delorean.repo', args),
mock.call('roads/delorean-deps.repo', args),
],
mock_get.mock_calls)
self.assertEqual([mock.call('[delorean]\nMr. Fusion', 'test',
name='delorean'),
mock.call('[delorean]\nMr. Fusion', 'test'),
],
mock_write.mock_calls)
@mock.patch('tripleo_repos.main._get_repo')
@mock.patch('tripleo_repos.main._write_repo')
def test_install_repos_deps(self, mock_write, mock_get):
args = mock.Mock()
args.repos = ['deps']
args.branch = 'master'
args.output_path = 'test'
args.distro = 'fake'
mock_get.return_value = '[delorean-deps]\nMr. Fusion'
main._install_repos(args, 'roads/')
mock_get.assert_called_once_with('roads/delorean-deps.repo', args)
mock_write.assert_called_once_with('[delorean-deps]\nMr. Fusion',
'test')
@mock.patch('tripleo_repos.main._get_repo')
@mock.patch('tripleo_repos.main._write_repo')
def test_install_repos_current_tripleo(self, mock_write, mock_get):
args = mock.Mock()
args.repos = ['current-tripleo']
args.branch = 'master'
args.output_path = 'test'
args.distro = 'fake'
mock_get.return_value = '[delorean]\nMr. Fusion'
main._install_repos(args, 'roads/')
self.assertEqual([mock.call('roads/current-tripleo/delorean.repo',
args),
mock.call('roads/delorean-deps.repo', args),
],
mock_get.mock_calls)
self.assertEqual([mock.call('[delorean]\nMr. Fusion', 'test'),
mock.call('[delorean]\nMr. Fusion', 'test'),
],
mock_write.mock_calls)
@mock.patch('tripleo_repos.main._get_repo')
@mock.patch('tripleo_repos.main._write_repo')
def test_install_repos_current_tripleo_dev(self, mock_write, mock_get):
args = mock.Mock()
args.repos = ['current-tripleo-dev']
args.branch = 'master'
args.output_path = 'test'
args.distro = 'fake'
mock_get.return_value = '[delorean]\nMr. Fusion'
main._install_repos(args, 'roads/')
mock_get.assert_any_call('roads/delorean-deps.repo', args)
# This is the wrong name for the deps repo, but I'm not bothered
# enough by that to mess with mocking multiple different calls.
mock_write.assert_any_call('[delorean]\nMr. Fusion', 'test')
mock_get.assert_any_call('roads/current-tripleo/delorean.repo', args)
mock_write.assert_any_call('[delorean-current-tripleo]\n'
'priority=20\nMr. Fusion', 'test',
name='delorean-current-tripleo')
mock_get.assert_called_with('roads/current/delorean.repo', args)
mock_write.assert_called_with('[delorean]\npriority=10\n%s\n'
'Mr. Fusion' %
main.INCLUDE_PKGS, 'test',
name='delorean')
@mock.patch('tripleo_repos.main._get_repo')
@mock.patch('tripleo_repos.main._write_repo')
def test_install_repos_tripleo_ci_testing(self, mock_write, mock_get):
args = mock.Mock()
args.repos = ['tripleo-ci-testing']
args.branch = 'master'
args.output_path = 'test'
args.distro = 'fake'
mock_get.return_value = '[delorean]\nMr. Fusion'
main._install_repos(args, 'roads/')
self.assertEqual([mock.call('roads/tripleo-ci-testing/delorean.repo',
args),
mock.call('roads/delorean-deps.repo', args),
],
mock_get.mock_calls)
self.assertEqual([mock.call('[delorean]\nMr. Fusion', 'test'),
mock.call('[delorean]\nMr. Fusion', 'test'),
],
mock_write.mock_calls)
@mock.patch('tripleo_repos.main._get_repo')
@mock.patch('tripleo_repos.main._write_repo')
def test_install_repos_current_tripleo_rdo(self, mock_write, mock_get):
args = mock.Mock()
args.repos = ['current-tripleo-rdo']
args.branch = 'master'
args.output_path = 'test'
args.distro = 'fake'
mock_get.return_value = '[delorean]\nMr. Fusion'
main._install_repos(args, 'roads/')
self.assertEqual([mock.call('roads/current-tripleo-rdo/delorean.repo',
args),
mock.call('roads/delorean-deps.repo', args),
],
mock_get.mock_calls)
self.assertEqual([mock.call('[delorean]\nMr. Fusion', 'test'),
mock.call('[delorean]\nMr. Fusion', 'test'),
],
mock_write.mock_calls)
@ddt.data('liberty', 'mitaka', 'newton', 'ocata', 'pike', 'queens',
'rocky', 'stein', 'master')
@mock.patch('tripleo_repos.main._write_repo')
@mock.patch('tripleo_repos.main._create_ceph')
def test_install_repos_ceph(self,
branch,
mock_create_ceph,
mock_write_repo):
ceph_release = {
'liberty': 'hammer',
'mitaka': 'hammer',
'newton': 'jewel',
'ocata': 'jewel',
'pike': 'jewel',
'queens': 'luminous',
'rocky': 'luminous',
'stein': 'nautilus',
'train': 'nautilus',
'ussuri': 'nautilus',
'victoria': 'nautilus',
'master': 'pacific',
}
args = mock.Mock()
args.repos = ['ceph']
args.branch = branch
args.output_path = 'test'
args.distro = 'fake'
mock_repo = '[centos-ceph-luminous]\nMr. Fusion'
mock_create_ceph.return_value = mock_repo
main._install_repos(args, 'roads/')
mock_create_ceph.assert_called_once_with(args, ceph_release[branch])
mock_write_repo.assert_called_once_with(mock_repo, 'test')
@mock.patch('tripleo_repos.main._write_repo')
def test_install_repos_opstools(self, mock_write):
args = mock.Mock()
args.repos = ['opstools']
args.branch = 'master'
args.output_path = 'test'
args.mirror = 'http://foo'
args.distro = 'fake'
main._install_repos(args, 'roads/')
expected_repo = ('\n[tripleo-centos-opstools]\n'
'name=tripleo-centos-opstools\n'
'baseurl=http://foo/centos/7/opstools/$basearch/\n'
'gpgcheck=0\n'
'enabled=1\n')
mock_write.assert_called_once_with(expected_repo,
'test')
@mock.patch('requests.get')
@mock.patch('tripleo_repos.main._write_repo')
def test_install_repos_deps_mirror(self, mock_write, mock_get):
args = mock.Mock()
args.repos = ['deps']
args.branch = 'master'
args.output_path = 'test'
args.old_mirror = 'http://mirror.centos.org'
args.mirror = 'http://foo'
args.distro = 'centos7'
args.rdo_mirror = 'http://bar'
# Abbreviated repos to verify the regex works
fake_repo = '''
[delorean-current-tripleo]
name=test repo
baseurl=https://trunk.rdoproject.org/centos7/some-repo-hash
enabled=1
[rdo-qemu-ev]
name=test qemu-ev
baseurl=http://mirror.centos.org/centos/7/virt/$basearch/kvm-common
enabled=1
'''
expected_repo = '''
[delorean-current-tripleo]
name=test repo
baseurl=http://bar/centos7/some-repo-hash
enabled=1
[rdo-qemu-ev]
name=test qemu-ev
baseurl=http://foo/centos/7/virt/$basearch/kvm-common
enabled=1
'''
mock_get.return_value = mock.Mock(text=fake_repo,
status_code=200)
main._install_repos(args, 'roads/')
mock_write.assert_called_once_with(expected_repo,
'test')
def test_install_repos_invalid(self):
args = mock.Mock()
args.repos = ['roads?']
self.assertRaises(main.InvalidArguments, main._install_repos, args,
'roads/')
@mock.patch('tripleo_repos.main._get_repo')
@mock.patch('tripleo_repos.main._write_repo')
def test_install_repos_centos8(self, mock_write, mock_get):
args = mock.Mock()
args.repos = ['current']
args.branch = 'master'
args.output_path = 'test'
args.distro = 'centos8'
args.stream = False
args.mirror = 'mirror'
mock_get.return_value = '[delorean]\nMr. Fusion'
main._install_repos(args, 'roads/')
self.assertEqual([mock.call('roads/current/delorean.repo', args),
mock.call('roads/delorean-deps.repo', args),
],
mock_get.mock_calls)
self.assertEqual([mock.call('[delorean]\nMr. Fusion', 'test',
name='delorean'),
mock.call('[delorean]\nMr. Fusion', 'test'),
mock.call((
'\n[tripleo-centos-highavailability]\n'
'name=tripleo-centos-highavailability\n'
'baseurl=mirror/centos/8/HighAvailability'
'/$basearch/os/\ngpgcheck=0\nenabled=1\n'),
'test'),
mock.call((
'\n[tripleo-centos-powertools]\n'
'name=tripleo-centos-powertools\n'
'baseurl=mirror/centos/8/PowerTools'
'/$basearch/os/\ngpgcheck=0\nenabled=1\n'),
'test')
],
mock_write.mock_calls)
@mock.patch('tripleo_repos.main._get_repo')
@mock.patch('tripleo_repos.main._write_repo')
def test_install_repos_centos8_stream(self, mock_write, mock_get):
args = mock.Mock()
args.repos = ['current']
args.branch = 'master'
args.output_path = 'test'
args.distro = 'centos8'
args.stream = True
args.no_stream = False
args.mirror = 'mirror'
mock_get.return_value = '[delorean]\nMr. Fusion'
main._install_repos(args, 'roads/')
self.assertEqual([mock.call('roads/current/delorean.repo', args),
mock.call('roads/delorean-deps.repo', args),
],
mock_get.mock_calls)
self.assertEqual([mock.call('[delorean]\nMr. Fusion', 'test',
name='delorean'),
mock.call('[delorean]\nMr. Fusion', 'test'),
mock.call((
'\n[tripleo-centos-highavailability]\n'
'name=tripleo-centos-highavailability\n'
'baseurl=mirror/centos/8-stream/HighAvailability'
'/$basearch/os/\ngpgcheck=0\nenabled=1\n'),
'test'),
mock.call((
'\n[tripleo-centos-powertools]\n'
'name=tripleo-centos-powertools\n'
'baseurl=mirror/centos/8-stream/PowerTools'
'/$basearch/os/\ngpgcheck=0\nenabled=1\n'),
'test')
],
mock_write.mock_calls)
@mock.patch('tripleo_repos.main._get_repo')
@mock.patch('tripleo_repos.main._write_repo')
def test_install_repos_centos9_stream(self, mock_write, mock_get):
args = mock.Mock()
args.repos = ['current']
args.branch = 'master'
args.output_path = 'test'
args.distro = 'centos9'
args.stream = True
args.no_stream = False
args.mirror = 'mirror'
mock_get.return_value = '[delorean]\nMr. Fusion'
main._install_repos(args, 'roads/')
self.assertEqual([mock.call('roads/current/delorean.repo', args),
mock.call('roads/delorean-deps.repo', args),
],
mock_get.mock_calls)
self.assertEqual([mock.call('[delorean]\nMr. Fusion', 'test',
name='delorean'),
mock.call('[delorean]\nMr. Fusion', 'test'),
mock.call((
'\n[tripleo-centos-highavailability]\n'
'name=tripleo-centos-highavailability\n'
'baseurl=mirror/9-stream/HighAvailability'
'/$basearch/os/\ngpgcheck=0\nenabled=1\n'),
'test'),
mock.call((
'\n[tripleo-centos-powertools]\n'
'name=tripleo-centos-powertools\n'
'baseurl=mirror/9-stream/CRB'
'/$basearch/os/\ngpgcheck=0\nenabled=1\n'),
'test'),
mock.call((
'\n[tripleo-centos-appstream]\n'
'name=tripleo-centos-appstream\n'
'baseurl=mirror/9-stream/AppStream'
'/$basearch/os/\ngpgcheck=0\nenabled=1\n\n'),
'test'),
mock.call((
'\n[tripleo-centos-baseos]\n'
'name=tripleo-centos-baseos\n'
'baseurl=mirror/9-stream/BaseOS'
'/$basearch/os/\ngpgcheck=0\nenabled=1\n'),
'test')
],
mock_write.mock_calls)
@mock.patch('tripleo_repos.main._get_repo')
@mock.patch('tripleo_repos.main._write_repo')
def test_install_repos_centos8_no_stream(self, mock_write, mock_get):
args = mock.Mock()
args.repos = ['current']
args.branch = 'master'
args.output_path = 'test'
args.distro = 'centos8'
args.stream = False
args.no_stream = True
args.mirror = 'mirror'
mock_get.return_value = '[delorean]\nMr. Fusion'
main._install_repos(args, 'roads/')
self.assertEqual([mock.call('roads/current/delorean.repo', args),
mock.call('roads/delorean-deps.repo', args),
],
mock_get.mock_calls)
self.assertEqual([mock.call('[delorean]\nMr. Fusion', 'test',
name='delorean'),
mock.call('[delorean]\nMr. Fusion', 'test'),
mock.call((
'\n[tripleo-centos-highavailability]\n'
'name=tripleo-centos-highavailability\n'
'baseurl=mirror/centos/8/HighAvailability'
'/$basearch/os/\ngpgcheck=0\nenabled=1\n'),
'test'),
mock.call((
'\n[tripleo-centos-powertools]\n'
'name=tripleo-centos-powertools\n'
'baseurl=mirror/centos/8/PowerTools'
'/$basearch/os/\ngpgcheck=0\nenabled=1\n'),
'test')
],
mock_write.mock_calls)
def test_write_repo(self):
m = mock.mock_open()
with mock.patch('tripleo_repos.main.open', m, create=True):
main._write_repo('#Doc\n[delorean]\nThis=Heavy', 'test')
m.assert_called_once_with('test/delorean.repo', 'w')
m().write.assert_called_once_with('#Doc\n[delorean]\nThis=Heavy')
def test_write_repo_invalid(self):
self.assertRaises(main.NoRepoTitle, main._write_repo, 'Great Scot!',
'test')
def test_parse_args(self):
with mock.patch.object(sys, 'argv', ['', 'current', 'deps', '-d',
'centos7', '-b', 'liberty',
'-o', 'test']):
args = main._parse_args('centos', '8')
self.assertEqual(['current', 'deps'], args.repos)
self.assertEqual('centos7', args.distro)
self.assertEqual('liberty', args.branch)
self.assertEqual('test', args.output_path)
def test_parse_args_long(self):
with mock.patch.object(sys, 'argv', ['', 'current', '--distro',
'centos7', '--branch',
'mitaka', '--output-path',
'test']):
args = main._parse_args('centos', '8')
self.assertEqual(['current'], args.repos)
self.assertEqual('centos7', args.distro)
self.assertEqual('mitaka', args.branch)
self.assertEqual('test', args.output_path)
def test_change_priority(self):
result = main._change_priority('[delorean]\npriority=1', 10)
self.assertEqual('[delorean]\npriority=10', result)
def test_change_priority_none(self):
result = main._change_priority('[delorean]', 10)
self.assertEqual('[delorean]\npriority=10', result)
def test_change_priority_none_muilti(self):
data = "[repo1]\n[repo2]\n"
expected = "[repo1]\n{0}\n[repo2]\n{0}\n".format("priority=10")
result = main._change_priority(data, 10)
self.assertEqual(expected, result)
def test_add_includepkgs(self):
data = "[repo1]\n[repo2]"
expected = "[repo1]\n{0}\n[repo2]\n{0}".format(main.INCLUDE_PKGS)
result = main._add_includepkgs(data)
self.assertEqual(expected, result)
def test_create_ceph(self):
mock_args = mock.Mock(mirror='http://foo')
result = main._create_ceph(mock_args, 'jewel')
expected_repo = '''
[tripleo-centos-ceph-jewel]
name=tripleo-centos-ceph-jewel
baseurl=http://foo/SIGs/9-stream/storage/$basearch/ceph-jewel/
gpgcheck=0
enabled=1
'''
self.assertEqual(expected_repo, result)
mock_args = mock.Mock(mirror='http://foo', distro='centos8')
result = main._create_ceph(mock_args, 'jewel')
expected_repo = '''
[tripleo-centos-ceph-jewel]
name=tripleo-centos-ceph-jewel
baseurl=http://foo/centos/8-stream/storage/$basearch/ceph-jewel/
gpgcheck=0
enabled=1
'''
self.assertEqual(expected_repo, result)
def test_inject_mirrors_centos(self):
start_repo = '''
[delorean]
name=delorean
baseurl=https://trunk.rdoproject.org/centos7/some-repo-hash
enabled=1
[centos]
name=centos
baseurl=http://mirror.centos.org/centos/7/virt/$basearch/kvm-common
enabled=1
'''
expected = '''
[delorean]
name=delorean
baseurl=http://bar/centos7/some-repo-hash
enabled=1
[centos]
name=centos
baseurl=http://foo/centos/7/virt/$basearch/kvm-common
enabled=1
'''
mock_args = mock.Mock(mirror='http://foo',
rdo_mirror='http://bar',
distro='centos',
old_mirror='http://mirror.centos.org')
result = main._inject_mirrors(start_repo, mock_args)
self.assertEqual(expected, result)
def test_inject_mirrors_rhel(self):
start_repo = '''
[delorean]
name=delorean
baseurl=https://trunk.rdoproject.org/centos7/some-repo-hash
enabled=1
[rhel]
name=rhel
baseurl=https://some/stuff
enabled=1
'''
expected = '''
[delorean]
name=delorean
baseurl=http://bar/centos7/some-repo-hash
enabled=1
[rhel]
name=rhel
baseurl=http://foo/stuff
enabled=1
'''
mock_args = mock.Mock(mirror='http://foo',
rdo_mirror='http://bar',
distro='rhel',
old_mirror='https://some')
result = main._inject_mirrors(start_repo, mock_args)
self.assertEqual(expected, result)
def test_inject_mirrors_no_match(self):
start_repo = '''
[delorean]
name=delorean
baseurl=https://some.mirror.com/centos7/some-repo-hash
enabled=1
'''
mock_args = mock.Mock(rdo_mirror='http://some.mirror.com',
distro='centos')
# If a user has a mirror whose repos already point at itself then
# the _inject_mirrors call should be a noop.
self.assertEqual(start_repo, main._inject_mirrors(start_repo,
mock_args))
@mock.patch('subprocess.check_call')
def test_run_pkg_clean(self, mock_check_call):
main._run_pkg_clean('centos7')
mock_check_call.assert_called_once_with(['yum', 'clean', 'metadata'])
@mock.patch('subprocess.check_call')
def test_run_pkg_clean_fedora(self, mock_check_call):
main._run_pkg_clean('fedora')
mock_check_call.assert_called_once_with(['dnf', 'clean', 'metadata'])
@mock.patch('subprocess.check_call')
def test_run_pkg_clean_fails(self, mock_check_call):
mock_check_call.side_effect = subprocess.CalledProcessError(88, '88')
self.assertRaises(subprocess.CalledProcessError,
main._run_pkg_clean, ['centos7'])
class TestValidate(testtools.TestCase):
def setUp(self):
super(TestValidate, self).setUp()
self.args = mock.Mock()
self.args.repos = ['current']
self.args.branch = 'master'
self.args.distro = 'centos7'
self.distro_major_version_id = "7"
self.args.stream = False
self.args.no_stream = False
def test_good(self):
main._validate_args(self.args, '', '')
def test_current_and_tripleo_dev(self):
self.args.repos = ['current', 'current-tripleo-dev']
self.assertRaises(main.InvalidArguments, main._validate_args,
self.args, '', '')
def test_tripleo_ci_testing_and_current_tripleo(self):
self.args.repos = ['current-tripleo', 'tripleo-ci-testing']
self.assertRaises(main.InvalidArguments, main._validate_args,
self.args, '', '')
def test_tripleo_ci_testing_and_ceph_opstools_allowed(self):
self.args.repos = ['ceph', 'opstools', 'tripleo-ci-testing']
main._validate_args(self.args, '', '')
def test_tripleo_ci_testing_and_deps_allowed(self):
self.args.repos = ['deps', 'tripleo-ci-testing']
main._validate_args(self.args, '', '')
def test_ceph_and_tripleo_dev(self):
self.args.repos = ['current-tripleo-dev', 'ceph']
self.args.output_path = main.DEFAULT_OUTPUT_PATH
main._validate_args(self.args, '', '')
def test_deps_and_tripleo_dev(self):
self.args.repos = ['deps', 'current-tripleo-dev']
self.assertRaises(main.InvalidArguments, main._validate_args,
self.args, '', '')
def test_current_and_tripleo(self):
self.args.repos = ['current', 'current-tripleo']
self.assertRaises(main.InvalidArguments, main._validate_args,
self.args, '', '')
def test_deps_and_tripleo_allowed(self):
self.args.repos = ['deps', 'current-tripleo']
main._validate_args(self.args, '', '')
def test_invalid_distro(self):
self.args.distro = 'Jigawatts 1.21'
self.assertRaises(main.InvalidArguments, main._validate_args,
self.args, '', '')
def test_invalid_stream(self):
self.args.output_path = main.DEFAULT_OUTPUT_PATH
self.args.stream = True
self.assertRaises(main.InvalidArguments, main._validate_args,
self.args, 'CentOS 8', '8')
def test_invalid_no_stream(self):
self.args.output_path = main.DEFAULT_OUTPUT_PATH
self.args.stream = False
self.args.no_stream = True
self.assertRaises(main.InvalidArguments, main._validate_args,
self.args, 'CentOS 8 Stream', '8')
def test_validate_distro_repos(self):
self.assertTrue(main._validate_distro_repos(self.args))
def test_validate_distro_repos_fedora_tripleo_dev(self):
self.args.distro = 'fedora'
self.args.repos = ['current-tripleo-dev']
self.assertRaises(main.InvalidArguments, main._validate_distro_repos,
self.args)

0
tests/unit/yum_config/__init__.py
View File

113
tests/unit/yum_config/fakes.py
View File

@ -1,113 +0,0 @@
# Copyright 2021 Red Hat, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
FAKE_FILE_PATH = '/path/to/file'
FAKE_DIR_PATH = '/path/to/dir'
FAKE_SUPP_OPTIONS = ['fake_option1', 'fake_option2']
FAKE_OPTION1 = 'fake_option1'
FAKE_DIR_FILES = ['fake_file1.conf', 'fake_file2.conf', 'fake.md']
FAKE_SECTIONS = ['fake_section1', 'fake_section2']
FAKE_SECTION1 = 'fake_section1'
FAKE_SECTION2 = 'fake_section2'
FAKE_SET_DICT = {
'key1': 'value1',
'key2': 'value2',
}
FAKE_REPO_DOWN_URL = '/fake/down/url/fake.repo'
FAKE_COMPOSE_URL = (
'https://composes.centos.org/fake-CentOS-Stream/compose/')
FAKE_REPO_PATH = '/etc/yum.repos.d/fake.repo'
FAKE_RELEASE_NAME = 'fake_release'
FAKE_COMPOSE_INFO = {
"header": {
"version": "1.2",
},
"payload": {
"compose": {
"id": "fake_compose_id",
},
"release": {
"name": "CentOS Stream",
"short": "CentOS-Stream",
"version": "8",
},
"variants": {
"AppStream": {
"arches": [
"aarch64",
"ppc64le",
"x86_64"
],
"id": "AppStream",
"name": "AppStream",
"paths": {
"packages": {
"aarch64": "AppStream/aarch64/os/Packages",
"ppc64le": "AppStream/ppc64le/os/Packages",
"x86_64": "AppStream/x86_64/os/Packages",
},
},
},
"BaseOS": {
"arches": [
"aarch64",
"ppc64le",
"x86_64",
],
"id": "BaseOS",
"name": "BaseOS",
"paths": {
"packages": {
"aarch64": "BaseOS/aarch64/os/Packages",
"ppc64le": "BaseOS/ppc64le/os/Packages",
"x86_64": "BaseOS/x86_64/os/Packages",
},
},
},
},
},
}
FAKE_ENV_OUTPUT = """
LANG=C.utf8
HOSTNAME=4cb7d7db1907
which_declare=declare -f
container=oci
PWD=/
HOME=/root
TERM=xterm
SHLVL=1
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
_=/usr/bin/env
"""
class FakeConfigParser(dict):
def __init__(self, *args, **kwargs):
super(FakeConfigParser, self).__init__(*args, **kwargs)
self.__dict__ = self
def write(self, file, space_around_delimiters=False):
pass
def read(self, file):
pass
def add_section(self, section):
self[section] = {}
def sections(self):
return self.keys()

18
tests/unit/yum_config/mock_modules.py
View File

@ -1,18 +0,0 @@
# Copyright 2021 Red Hat, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
import sys
from unittest import mock
sys.modules["dnf"] = mock.Mock()

119
tests/unit/yum_config/test_compose_repos.py
View File

@ -1,119 +0,0 @@
# Copyright 2021 Red Hat, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
import copy
import json
import os
from unittest import mock
import urllib.request
from . import fakes
from . import test_main
import tripleo_repos.yum_config.constants as const
import tripleo_repos.yum_config.exceptions as exc
import tripleo_repos.yum_config.compose_repos as repos
import tripleo_repos.yum_config.yum_config as yum_config
class TestTripleOComposeRepos(test_main.TestTripleoYumConfigBase):
"""Tests for TripleComposeRepos class and its methods."""
def setUp(self):
super(TestTripleOComposeRepos, self).setUp()
self.repos = self._create_compose_repos_obj(
dir_path='/tmp'
)
def _create_compose_repos_obj(
self,
compose_url=fakes.FAKE_COMPOSE_URL,
release=const.COMPOSE_REPOS_RELEASES[0],
dir_path=None,
arch=const.COMPOSE_REPOS_SUPPORTED_ARCHS[0]):
url_res = mock.Mock()
json_data = json.dumps(fakes.FAKE_COMPOSE_INFO)
self.mock_object(urllib.request, "urlopen",
mock.Mock(return_value=url_res))
self.mock_object(url_res, 'read',
mock.Mock(return_value=json_data))
return repos.TripleOYumComposeRepoConfig(
compose_url, release, dir_path=dir_path, arch=arch)
def test_tripleo_compose_repos_invalid_release(self):
self.assertRaises(exc.TripleOYumConfigComposeError,
repos.TripleOYumComposeRepoConfig,
fakes.FAKE_COMPOSE_URL,
'invalid_release')
def test_tripleo_compose_repos_invalid_url(self):
self.assertRaises(exc.TripleOYumConfigComposeError,
repos.TripleOYumComposeRepoConfig,
"http://invalid_url.org",
const.COMPOSE_REPOS_RELEASES[0])
def test__get_compose_info_exc(self):
self.mock_object(urllib.request, "urlopen",
mock.Mock(side_effect=Exception))
self.assertRaises(exc.TripleOYumConfigComposeError,
self.repos._get_compose_info)
def test_enable_compose_repos(self):
self.mock_object(self.repos, 'add_section')
self.repos.enable_compose_repos(
variants=fakes.FAKE_COMPOSE_INFO['payload']['variants'].keys(),
override_repos=False
)
@mock.patch('builtins.open')
def test_add_section(self, open):
self.mock_object(os.path, 'isfile', mock.Mock(return_value=False))
mock_add_section = self.mock_object(yum_config.TripleOYumConfig,
"add_section")
self.repos.add_section(fakes.FAKE_SECTION1, fakes.FAKE_SET_DICT,
fakes.FAKE_FILE_PATH)
mock_add_section.assert_called_once_with(
fakes.FAKE_SECTION1, fakes.FAKE_SET_DICT, fakes.FAKE_FILE_PATH
)
def test_update_section(self):
mock_update = self.mock_object(yum_config.TripleOYumConfig,
"update_section")
expected_set_dict = copy.deepcopy(fakes.FAKE_SET_DICT)
expected_set_dict['enabled'] = '1'
self.repos.update_section(fakes.FAKE_SECTION1,
set_dict=fakes.FAKE_SET_DICT,
enabled=True,
file_path=fakes.FAKE_FILE_PATH)
mock_update.assert_called_once_with(fakes.FAKE_SECTION1,
expected_set_dict,
file_path=fakes.FAKE_FILE_PATH)
def test_update_all_sections(self):
mock_update = self.mock_object(yum_config.TripleOYumConfig,
"update_all_sections")
expected_set_dict = copy.deepcopy(fakes.FAKE_SET_DICT)
expected_set_dict['enabled'] = '0'
self.repos.update_all_sections(fakes.FAKE_FILE_PATH,
set_dict=fakes.FAKE_SET_DICT,
enabled=False)
mock_update.assert_called_once_with(expected_set_dict,
fakes.FAKE_FILE_PATH)

71
tests/unit/yum_config/test_dnf_manager.py
View File

@ -1,71 +0,0 @@
# Copyright 2021 Red Hat, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
import ddt
from unittest import mock
from . import test_main
import tripleo_repos.yum_config.dnf_manager as dnf_mgr
@ddt.ddt
class TestTripleODnfManager(test_main.TestTripleoYumConfigBase):
"""Tests for DnfModuleManager class and its methods."""
def setUp(self):
super(TestTripleODnfManager, self).setUp()
self.dnf = dnf_mgr.DnfModuleManager()
@ddt.data(
{'module': 'fake', 'stream': None, 'profile': None},
{'module': 'fake', 'stream': 'fake_stream', 'profile': None},
{'module': 'fake', 'stream': None, 'profile': 'fake_prof'},
{'module': 'fake', 'stream': 'fake_stream', 'profile': 'fake_prof'},
)
@ddt.unpack
def test__get_module_spec(self, module, stream, profile):
exp_str = module
exp_str += ':' + stream if stream else ''
exp_str += '/' + profile if profile else ''
result = self.dnf._get_module_spec(module, stream=stream,
profile=profile)
self.assertEqual(exp_str, result)
@ddt.data(Exception, RuntimeError)
def test__do_transaction_failure(self, exc):
mock_transaction = self.mock_object(
self.dnf.base, 'do_transaction',
mock.Mock(side_effect=exc))
self.assertRaises(exc, self.dnf._do_transaction)
mock_transaction.assert_called_once()
@ddt.data('enable', 'disable', 'reset', 'install', 'remove')
def test_module_operations(self, operation):
fake_module = 'fake_module'
fake_stream = 'fake_stream'
fake_profile = 'fake_profile'
mock_get_mod_spec = self.mock_object(self.dnf, '_get_module_spec')
mock_op = self.mock_object(self.dnf.module_base, operation)
mock_transaction = self.mock_object(self.dnf, '_do_transaction')
dnf_method = getattr(self.dnf, operation + "_module")
dnf_method(fake_module, stream=fake_stream, profile=fake_profile)
mock_get_mod_spec.assert_called_once_with(
fake_module, stream=fake_stream, profile=fake_profile)
mock_op.assert_called_once()
mock_transaction.assert_called_once()

209
tests/unit/yum_config/test_main.py
View File

@ -1,209 +0,0 @@
# Copyright 2021 Red Hat, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
import ddt
import sys
import unittest
from unittest import mock
from . import fakes
from . import mock_modules # noqa: F401
import tripleo_repos.yum_config.__main__ as main
import tripleo_repos.yum_config.compose_repos as repos
import tripleo_repos.yum_config.constants as const
import tripleo_repos.yum_config.dnf_manager as dnf_mgr
import tripleo_repos.yum_config.utils as utils
import tripleo_repos.yum_config.yum_config as yum_cfg
class TestTripleoYumConfigBase(unittest.TestCase):
"""Base test class for tripleo yum config module."""
def mock_object(self, obj, attr, new_attr=None):
if not new_attr:
new_attr = mock.Mock()
patcher = mock.patch.object(obj, attr, new_attr)
patcher.start()
# stop patcher at the end of the test
self.addCleanup(patcher.stop)
return new_attr
@ddt.ddt
class TestTripleoYumConfigMain(TestTripleoYumConfigBase):
"""Test class for main method operations."""
def setUp(self):
super(TestTripleoYumConfigMain, self).setUp()
self.mock_object(utils, 'get_distro_info',
mock.Mock(return_value=("centos", "8", None)))
def test_main_repo(self):
sys.argv[1:] = ['repo', '--name', 'fake_repo', '--enable',
'--set-opts', 'key1=value1', 'key2=value2',
'--config-file-path', fakes.FAKE_FILE_PATH,
'--down-url', fakes.FAKE_REPO_DOWN_URL]
yum_repo_obj = mock.Mock()
mock_update_section = self.mock_object(yum_repo_obj,
'add_or_update_section')
mock_yum_repo_obj = self.mock_object(
yum_cfg, 'TripleOYumRepoConfig',
mock.Mock(return_value=yum_repo_obj))
main.main()
expected_dict = {'key1': 'value1', 'key2': 'value2'}
mock_yum_repo_obj.assert_called_once_with(dir_path=const.YUM_REPO_DIR,
environment_file=None)
mock_update_section.assert_called_once_with(
'fake_repo', set_dict=expected_dict,
file_path=fakes.FAKE_FILE_PATH, enabled=True,
from_url=fakes.FAKE_REPO_DOWN_URL)
def test_main_repo_from_url(self):
sys.argv[1:] = ['repo', '--enable',
'--set-opts', 'key1=value1', 'key2=value2',
'--config-file-path', fakes.FAKE_FILE_PATH,
'--down-url', fakes.FAKE_REPO_DOWN_URL]
yum_repo_obj = mock.Mock()
mock_update_all_sections = self.mock_object(
yum_repo_obj, 'add_or_update_all_sections_from_url')
mock_yum_repo_obj = self.mock_object(
yum_cfg, 'TripleOYumRepoConfig',
mock.Mock(return_value=yum_repo_obj))
main.main()
expected_dict = {'key1': 'value1', 'key2': 'value2'}
mock_yum_repo_obj.assert_called_once_with(dir_path=const.YUM_REPO_DIR,
environment_file=None)
mock_update_all_sections.assert_called_once_with(
fakes.FAKE_REPO_DOWN_URL, file_path=fakes.FAKE_FILE_PATH,
set_dict=expected_dict, enabled=True)
@ddt.data('enable', 'disable', 'reset', 'install', 'remove')
def test_main_module(self, operation):
sys.argv[1:] = ['module', operation, 'fake_module', '--stream',
'fake_stream', '--profile', 'fake_profile']
mock_dnf_mod = mock.Mock()
mock_op = self.mock_object(mock_dnf_mod, operation + '_module')
mock_dnf_mod_obj = self.mock_object(
dnf_mgr, 'DnfModuleManager',
mock.Mock(return_value=mock_dnf_mod))
main.main()
mock_dnf_mod_obj.assert_called_once()
mock_op.assert_called_once_with(
'fake_module', stream='fake_stream', profile='fake_profile')
def test_main_global_conf(self):
sys.argv[1:] = ['global', '--set-opts', 'key1=value1', 'key2=value2']
yum_global_obj = mock.Mock()
mock_update_section = self.mock_object(
yum_global_obj, 'update_section')
mock_yum_global_obj = self.mock_object(
yum_cfg, 'TripleOYumGlobalConfig',
mock.Mock(return_value=yum_global_obj))
main.main()
expected_dict = {'key1': 'value1', 'key2': 'value2'}
mock_yum_global_obj.assert_called_once_with(file_path=None,
environment_file=None)
mock_update_section.assert_called_once_with('main', expected_dict)
def test_main_no_command(self):
sys.argv[1:] = []
with self.assertRaises(SystemExit) as command:
main.main()
self.assertEqual(2, command.exception.code)
@ddt.data('repo')
def test_main_repo_mod_without_name(self, command):
sys.argv[1:] = [command, '--set-opts', 'key1=value1',
'--config-dir-path', '/tmp']
with self.assertRaises(SystemExit) as command:
main.main()
self.assertEqual(2, command.exception.code)
def test_main_repo_without_name_and_url(self):
sys.argv[1:] = ['repo', '--enable',
'--set-opts', 'key1=value1', 'key2=value2',
'--config-file-path', fakes.FAKE_FILE_PATH,
'--config-dir-path', '/tmp']
with self.assertRaises(SystemExit) as command:
main.main()
self.assertEqual(2, command.exception.code)
@ddt.data('key:value', 'value', 'key value')
def test_main_invalid_options_format(self, option):
sys.argv[1:] = ['global', '--set-opts', option]
with self.assertRaises(SystemExit) as command:
main.main()
self.assertEqual(2, command.exception.code)
def test_main_enable_compose_repos(self):
sys.argv[1:] = [
'enable-compose-repos', '--compose-url', fakes.FAKE_COMPOSE_URL,
'--release', const.COMPOSE_REPOS_RELEASES[0],
'--variants', 'fake_variant',
'--disable-repos', fakes.FAKE_REPO_PATH,
'--arch', const.COMPOSE_REPOS_SUPPORTED_ARCHS[0],
]
repos_obj = mock.Mock()
mock_yum_global_obj = self.mock_object(
repos, 'TripleOYumComposeRepoConfig',
mock.Mock(return_value=repos_obj))
mock_enable_composes = self.mock_object(
repos_obj, 'enable_compose_repos')
mock_update_all = self.mock_object(
repos_obj, 'update_all_sections')
self.mock_object(yum_cfg, 'validated_file_path',
mock.Mock(return_value=True))
main.main()
mock_yum_global_obj.assert_called_once_with(
fakes.FAKE_COMPOSE_URL,
const.COMPOSE_REPOS_RELEASES[0],
dir_path=const.YUM_REPO_DIR,
arch=const.COMPOSE_REPOS_SUPPORTED_ARCHS[0],
environment_file=None)
mock_enable_composes.assert_called_once_with(
variants=['fake_variant'], override_repos=False)
mock_update_all.assert_called_once_with(
fakes.FAKE_REPO_PATH, enabled=False
)
def test_main_invalid_release_for_dnf_module(self):
self.mock_object(utils, 'get_distro_info',
mock.Mock(return_value=("centos", "7", None)))
sys.argv[1:] = ['module', 'enable', 'fake_module']
with self.assertRaises(SystemExit) as command:
main.main()
self.assertEqual(2, command.exception.code)

462
tests/unit/yum_config/test_yum_config.py
View File

@ -1,462 +0,0 @@
# Copyright 2021 Red Hat, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License"); you may
# not use this file except in compliance with the License. You may obtain
# a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
# License for the specific language governing permissions and limitations
# under the License.
import configparser
import copy
import ddt
import os
import subprocess
from unittest import mock
from . import fakes
from . import test_main
import tripleo_repos.yum_config.constants as const
import tripleo_repos.yum_config.exceptions as exc
import tripleo_repos.yum_config.yum_config as yum_cfg
import tripleo_repos.utils as repos_utils
@ddt.ddt
class TestTripleOYumConfig(test_main.TestTripleoYumConfigBase):
"""Tests for TripleYumConfig class and its methods."""
def _create_yum_config_obj(self, dir_path=None, valid_options=None,
file_extension=None):
self.mock_object(os.path, 'isfile')
self.mock_object(os, 'access')
self.mock_object(os.path, 'isdir')
return yum_cfg.TripleOYumConfig(dir_path=dir_path,
valid_options=valid_options,
file_extension=file_extension)
def test_tripleo_yum_config_invalid_dir_path(self):
self.mock_object(os.path, 'isdir',
mock.Mock(return_value=False))
self.assertRaises(exc.TripleOYumConfigNotFound,
yum_cfg.TripleOYumConfig,
dir_path='fake_dir_path')
def test_read_config_file_path(self):
yum_config = self._create_yum_config_obj()
parser_mock = mock.Mock()
self.mock_object(configparser, 'ConfigParser',
mock.Mock(return_value=parser_mock))
read_mock = self.mock_object(parser_mock, 'read')
self.mock_object(parser_mock, 'sections',
mock.Mock(return_value=fakes.FAKE_SECTIONS))
config_parser, file_path = yum_config._read_config_file(
fakes.FAKE_FILE_PATH,
section=fakes.FAKE_SECTION1
)
self.assertEqual(parser_mock, config_parser)
self.assertEqual(fakes.FAKE_FILE_PATH, file_path)
read_mock.assert_called_once_with(fakes.FAKE_FILE_PATH)
def test_read_config_file_path_parse_error(self):
yum_config = self._create_yum_config_obj()
parser_mock = mock.Mock()
self.mock_object(configparser, 'ConfigParser',
mock.Mock(return_value=parser_mock))
read_mock = self.mock_object(parser_mock, 'read',
mock.Mock(side_effect=configparser.Error))
self.assertRaises(exc.TripleOYumConfigFileParseError,
yum_config._read_config_file,
fakes.FAKE_FILE_PATH,
section=fakes.FAKE_SECTION1)
read_mock.assert_called_once_with(fakes.FAKE_FILE_PATH)
def test_read_config_file_path_invalid_section(self):
yum_config = self._create_yum_config_obj()
parser_mock = mock.Mock()
self.mock_object(configparser, 'ConfigParser',
mock.Mock(return_value=parser_mock))
read_mock = self.mock_object(parser_mock, 'read')
self.mock_object(parser_mock, 'sections',
mock.Mock(return_value=fakes.FAKE_SECTIONS))
self.assertRaises(exc.TripleOYumConfigInvalidSection,
yum_config._read_config_file,
fakes.FAKE_FILE_PATH,
section='invalid_section')
read_mock.assert_called_once_with(fakes.FAKE_FILE_PATH)
def test_get_config_files(self):
yum_config = self._create_yum_config_obj(
dir_path=fakes.FAKE_DIR_PATH,
file_extension='.conf')
parser_mocks = []
for i in range(3):
parser_mock = mock.Mock()
parser_mocks.append(parser_mock)
self.mock_object(parser_mock, 'read')
self.mock_object(parser_mocks[0], 'sections',
mock.Mock(return_value=[]))
# second file inside dir will have the expected sections
self.mock_object(parser_mocks[1], 'sections',
mock.Mock(return_value=fakes.FAKE_SECTIONS))
self.mock_object(parser_mocks[2], 'sections',
mock.Mock(return_value=[]))
self.mock_object(os, 'listdir',
mock.Mock(return_value=fakes.FAKE_DIR_FILES))
self.mock_object(os, 'access', mock.Mock(return_value=True))
self.mock_object(configparser, 'ConfigParser',
mock.Mock(side_effect=parser_mocks))
result = yum_config._get_config_files(fakes.FAKE_SECTION1)
expected_dir_path = [os.path.join(fakes.FAKE_DIR_PATH,
fakes.FAKE_DIR_FILES[1])]
self.assertEqual(expected_dir_path, result)
@mock.patch('builtins.open')
def test_update_section(self, open):
yum_config = self._create_yum_config_obj(
valid_options=fakes.FAKE_SUPP_OPTIONS)
config_parser = fakes.FakeConfigParser({fakes.FAKE_SECTION1: {}})
mock_read_config = self.mock_object(
yum_config, '_read_config_file',
mock.Mock(return_value=(config_parser, fakes.FAKE_FILE_PATH)))
updates = {fakes.FAKE_OPTION1: 'new_fake_value'}
yum_config.update_section(fakes.FAKE_SECTION1, updates,
file_path=fakes.FAKE_FILE_PATH)
mock_read_config.assert_called_once_with(fakes.FAKE_FILE_PATH,
section=fakes.FAKE_SECTION1)
def test_update_section_invalid_options(self):
yum_config = self._create_yum_config_obj(
valid_options=fakes.FAKE_SUPP_OPTIONS)
updates = {'invalid_option': 'new_fake_value'}
self.assertRaises(exc.TripleOYumConfigInvalidOption,
yum_config.update_section,
fakes.FAKE_SECTION1,
updates,
file_path=fakes.FAKE_FILE_PATH)
def test_update_section_file_not_found(self):
yum_config = self._create_yum_config_obj(
valid_options=fakes.FAKE_SUPP_OPTIONS)
mock_get_configs = self.mock_object(
yum_config, '_get_config_files',
mock.Mock(return_value=[]))
updates = {fakes.FAKE_OPTION1: 'new_fake_value'}
self.assertRaises(exc.TripleOYumConfigNotFound,
yum_config.update_section,
fakes.FAKE_SECTION1,
updates)
mock_get_configs.assert_called_once_with(fakes.FAKE_SECTION1)
@mock.patch('builtins.open')
def test_add_section(self, open):
yum_config = self._create_yum_config_obj(
valid_options=fakes.FAKE_SUPP_OPTIONS)
config_parser = fakes.FakeConfigParser({fakes.FAKE_SECTION1: {}})
mock_read_config = self.mock_object(
yum_config, '_read_config_file',
mock.Mock(return_value=(config_parser, fakes.FAKE_FILE_PATH)))
updates = {fakes.FAKE_OPTION1: 'new_fake_value'}
yum_config.add_section(fakes.FAKE_SECTION2, updates,
file_path=fakes.FAKE_FILE_PATH)
mock_read_config.assert_called_once_with(
file_path=fakes.FAKE_FILE_PATH)
@mock.patch('builtins.open')
def test_add_section_already_exists(self, open):
yum_config = self._create_yum_config_obj(
valid_options=fakes.FAKE_SUPP_OPTIONS)
config_parser = fakes.FakeConfigParser({fakes.FAKE_SECTION1: {}})
mock_read_config = self.mock_object(
yum_config, '_read_config_file',
mock.Mock(return_value=(config_parser, fakes.FAKE_FILE_PATH)))
updates = {fakes.FAKE_OPTION1: 'new_fake_value'}
self.assertRaises(exc.TripleOYumConfigInvalidSection,
yum_config.add_section,
fakes.FAKE_SECTION1, updates,
file_path=fakes.FAKE_FILE_PATH)
mock_read_config.assert_called_once_with(
file_path=fakes.FAKE_FILE_PATH)
@mock.patch('builtins.open')
def test_add_update_all_sections(self, open):
yum_config = self._create_yum_config_obj(
valid_options=fakes.FAKE_SUPP_OPTIONS)
config_parser = fakes.FakeConfigParser({fakes.FAKE_SECTION1: {}})
mock_read_config = self.mock_object(
yum_config, '_read_config_file',
mock.Mock(return_value=(config_parser, fakes.FAKE_FILE_PATH)))
updates = {fakes.FAKE_OPTION1: 'new_fake_value'}
yum_config.update_all_sections(updates, fakes.FAKE_FILE_PATH)
mock_read_config.assert_called_once_with(fakes.FAKE_FILE_PATH)
def test_source_env_file(self):
p_open_mock = mock.Mock()
mock_open = self.mock_object(subprocess, 'Popen',
mock.Mock(return_value=p_open_mock))
data_mock = mock.Mock()
self.mock_object(data_mock, 'decode',
mock.Mock(return_value=fakes.FAKE_ENV_OUTPUT))
self.mock_object(p_open_mock, 'communicate',
mock.Mock(return_value=[data_mock]))
env_update_mock = self.mock_object(os.environ, 'update')
yum_cfg.source_env_file('fake_source_file', update=True)
exp_env_dict = dict(
line.split("=", 1) for line in fakes.FAKE_ENV_OUTPUT.splitlines()
if len(line.split("=", 1)) > 1)
mock_open.assert_called_once_with(". fake_source_file; env",
stdout=subprocess.PIPE,
shell=True)
env_update_mock.assert_called_once_with(exp_env_dict)
def test_get_config_from_url_invalid_url(self):
yum_config = self._create_yum_config_obj(
valid_options=fakes.FAKE_SUPP_OPTIONS)
fake_context = mock.Mock()
self.mock_object(repos_utils, 'http_get',
mock.Mock(return_value=(fake_context, 404)))
self.assertRaises(exc.TripleOYumConfigUrlError,
yum_config.get_config_from_url,
fakes.FAKE_REPO_DOWN_URL)
def test_get_config_from_url(self):
yum_config = self._create_yum_config_obj(
valid_options=fakes.FAKE_SUPP_OPTIONS)
fake_context = mock.Mock()
self.mock_object(repos_utils, 'http_get',
mock.Mock(return_value=(fake_context, 200)))
parser_mock = mock.Mock()
self.mock_object(configparser, 'ConfigParser',
mock.Mock(return_value=parser_mock))
result = yum_config.get_config_from_url(fakes.FAKE_REPO_DOWN_URL)
self.assertEqual(parser_mock, result)
def test_get_options_from_url_section_not_found(self):
yum_config = self._create_yum_config_obj(
valid_options=fakes.FAKE_SUPP_OPTIONS)
fake_config = mock.Mock()
self.mock_object(fake_config, 'sections',
mock.Mock(return_value=[]))
mock_get_from_url = self.mock_object(
yum_config, 'get_config_from_url',
mock.Mock(return_value=fake_config))
self.assertRaises(exc.TripleOYumConfigInvalidSection,
yum_config.get_options_from_url,
fakes.FAKE_REPO_DOWN_URL,
fakes.FAKE_SECTION1)
mock_get_from_url.assert_called_once_with(fakes.FAKE_REPO_DOWN_URL)
@ddt.ddt
class TestTripleOYumRepoConfig(test_main.TestTripleoYumConfigBase):
"""Tests for TripleOYumRepoConfig class and its methods."""
def setUp(self):
super(TestTripleOYumRepoConfig, self).setUp()
self.config_obj = yum_cfg.TripleOYumRepoConfig(
dir_path='/tmp'
)
@ddt.data(True, False, None)
def test_yum_repo_config_update_section(self, enable):
self.mock_object(os.path, 'isfile')
self.mock_object(os, 'access')
self.mock_object(os.path, 'isdir')
mock_update = self.mock_object(yum_cfg.TripleOYumConfig,
'update_section')
updates = {fakes.FAKE_OPTION1: 'new_fake_value'}
expected_updates = copy.copy(updates)
if enable is not None:
expected_updates['enabled'] = '1' if enable else '0'
self.config_obj.update_section(
fakes.FAKE_SECTION1, set_dict=updates,
file_path=fakes.FAKE_FILE_PATH, enabled=enable)
mock_update.assert_called_once_with(fakes.FAKE_SECTION1,
expected_updates,
file_path=fakes.FAKE_FILE_PATH)
@mock.patch('builtins.open')
@ddt.data(None, fakes.FAKE_REPO_DOWN_URL)
def test_add_or_update_section(self, open, down_url):
mock_update = self.mock_object(
self.config_obj, 'update_section',
mock.Mock(side_effect=exc.TripleOYumConfigNotFound(
error_msg='error')))
mock_add_section = self.mock_object(self.config_obj, 'add_section')
extra_opt = {'key1': 'new value 1'}
mock_get_from_url = self.mock_object(
self.config_obj, 'get_options_from_url',
mock.Mock(return_value=extra_opt))
self.config_obj.add_or_update_section(
fakes.FAKE_SECTION1,
set_dict=fakes.FAKE_SET_DICT,
file_path=fakes.FAKE_FILE_PATH,
enabled=True,
create_if_not_exists=True,
from_url=down_url)
fake_set_dict = copy.deepcopy(fakes.FAKE_SET_DICT)
fake_set_dict['name'] = fakes.FAKE_SECTION1
if down_url:
fake_set_dict.update(extra_opt)
mock_get_from_url.assert_called_once_with(down_url,
fakes.FAKE_SECTION1)
mock_update.assert_called_once_with(fakes.FAKE_SECTION1,
set_dict=fake_set_dict,
file_path=fakes.FAKE_FILE_PATH,
enabled=True)
mock_add_section.assert_called_once_with(
fakes.FAKE_SECTION1,
fake_set_dict,
fakes.FAKE_FILE_PATH,
enabled=True)
@ddt.data((fakes.FAKE_FILE_PATH, False), (None, True))
@ddt.unpack
def test_add_or_update_section_file_not_found(self, fake_path,
create_if_not_exists):
mock_update = self.mock_object(
self.config_obj, 'update_section',
mock.Mock(side_effect=exc.TripleOYumConfigNotFound(
error_msg='error')))
self.assertRaises(
exc.TripleOYumConfigNotFound,
self.config_obj.add_or_update_section,
fakes.FAKE_SECTION1,
set_dict=fakes.FAKE_SET_DICT,
file_path=fake_path,
enabled=True,
create_if_not_exists=create_if_not_exists)
fake_set_dict = copy.deepcopy(fakes.FAKE_SET_DICT)
fake_set_dict['name'] = fakes.FAKE_SECTION1
mock_update.assert_called_once_with(fakes.FAKE_SECTION1,
set_dict=fake_set_dict,
file_path=fake_path,
enabled=True)
@ddt.data(None, False, True)
def test_add_section(self, enabled):
mock_add = self.mock_object(yum_cfg.TripleOYumConfig, 'add_section')
self.config_obj.add_section(
fakes.FAKE_SECTION1, fakes.FAKE_SET_DICT,
fakes.FAKE_FILE_PATH, enabled=enabled)
updated_dict = copy.deepcopy(fakes.FAKE_SET_DICT)
if enabled is not None:
updated_dict['enabled'] = '1' if enabled else '0'
mock_add.assert_called_once_with(fakes.FAKE_SECTION1, updated_dict,
fakes.FAKE_FILE_PATH)
@ddt.data(fakes.FAKE_FILE_PATH, None)
def test_add_or_update_all_sections_from_url(self, file_path):
add_or_update_section = self.mock_object(
self.config_obj, 'add_or_update_section')
fake_config = mock.Mock()
self.mock_object(fake_config, 'sections',
mock.Mock(return_value=[fakes.FAKE_SECTION1]))
options_from_url = {'key3': 'value3'}
self.mock_object(fake_config, 'items',
mock.Mock(return_value=options_from_url))
mock_get_from_url = self.mock_object(
self.config_obj, 'get_config_from_url',
mock.Mock(return_value=fake_config))
exp_file_path = (
file_path or os.path.join(
'/tmp', fakes.FAKE_REPO_DOWN_URL.split('/')[-1])
)
self.config_obj.add_or_update_all_sections_from_url(
fakes.FAKE_REPO_DOWN_URL,
file_path=file_path,
set_dict=fakes.FAKE_SET_DICT,
enabled=True,
create_if_not_exists=True)
mock_get_from_url.assert_called_once_with(fakes.FAKE_REPO_DOWN_URL)
expected_update_dict = copy.deepcopy(fakes.FAKE_SET_DICT)
expected_update_dict.update(options_from_url)
add_or_update_section.assert_called_once_with(
fakes.FAKE_SECTION1, set_dict=expected_update_dict,
file_path=exp_file_path, enabled=True,
create_if_not_exists=True)
@ddt.ddt
class TestTripleOYumGlobalConfig(test_main.TestTripleoYumConfigBase):
"""Tests for TripleOYumGlobalConfig class and its methods."""
@mock.patch('builtins.open')
def test_create_yum_global_config_create_yum_conf(self, open):
self.mock_object(os, 'access')
self.mock_object(os.path, 'isdir')
self.mock_object(os.path, 'isfile',
mock.Mock(side_effect=[False, True]))
fake_cfg_parser = mock.Mock()
mock_read = self.mock_object(fake_cfg_parser, 'read')
mock_add = self.mock_object(fake_cfg_parser, 'add_section')
mock_write = self.mock_object(fake_cfg_parser, 'write')
self.mock_object(configparser, 'ConfigParser',
mock.Mock(return_value=fake_cfg_parser))
cfg_obj = yum_cfg.TripleOYumGlobalConfig()
self.assertIsNotNone(cfg_obj)
mock_read.assert_called_once_with(const.YUM_GLOBAL_CONFIG_FILE_PATH)
mock_add.assert_called_once_with('main')
mock_write.assert_called_once()

108
tox.ini
View File

@ -1,108 +0,0 @@
[tox]
minversion = 3.18.0
skipsdist = True
envlist = py,pep8,packaging,sanity
requires =
tox-ansible>=1.5.3
[testenv]
usedevelop = True
setenv =
ANSIBLE_FORCE_COLOR={tty:1:0}
VIRTUAL_ENV={envdir}
passenv =
HOME
deps =
-c{env:TOX_CONSTRAINTS_FILE:https://releases.openstack.org/constraints/upper/master}
-r{toxinidir}/test-requirements.txt
-r{toxinidir}/requirements.txt
commands =
stestr run --slowest {posargs}
[testenv:venv]
commands = {posargs}
[testenv:docs]
deps =
-c{env:TOX_CONSTRAINTS_FILE:https://releases.openstack.org/constraints/upper/master}
-r{toxinidir}/doc/requirements.txt
allowlist_externals =
rm
commands =
rm -rf doc/build
sphinx-build -W --keep-going -b html doc/source doc/build/html
[testenv:pep8]
deps = flake8
commands = flake8
[testenv:cover]
setenv =
PYTHON=coverage run --source tripleo_repos --parallel-mode
commands =
stestr run {posargs}
coverage combine
coverage html -d cover
coverage xml -o cover/coverage.xml
[testenv:packaging]
description =
Build package, verify metadata, install package and assert basic behavior
deps =
build
twine
ansible-core # used for ansible-galaxy command
skip_install = true
commands =
# build wheel and sdist using PEP-517
{envpython} -c 'import os.path, shutil, sys; \
dist_dir = os.path.join("{toxinidir}", "dist"); \
os.path.isdir(dist_dir) or sys.exit(0); \
print("Removing \{!s\} contents...".format(dist_dir), file=sys.stderr); \
shutil.rmtree(dist_dir)'
{envpython} -m build \
--sdist \
--wheel \
--outdir {toxinidir}/dist/ \
{toxinidir}
# Validate metadata using twine
twine check {toxinidir}/dist/*
# Install the wheel
sh -c "python3 -m pip install {toxinidir}/dist/*.whl"
# Assure that CLIs were installed
tripleo-repos --help
tripleo-get-hash --help
tripleo-yum-config --help
# Validate collection installation
ansible-galaxy collection install --force .
# Ensure that ansible is able to load the modules, as syntax check will fail
# if modules cannot be loaded.
sh -c "ansible-playbook --syntax-check playbooks/*.yaml"
ansible localhost -m tripleo.repos.get_hash -a "release=master os_version=centos8"
allowlist_externals =
sh
# https://github.com/ansible-community/tox-ansible/issues/96
# Override ansible version coming from ansible-test package.
[testenv:sanity]
deps =
ansible-core>=2.11,<2.12
[testenv:molecule]
description = Used by all molecule jobs (tox-ansible)
deps =
ansible-core
molecule>=3.3.0,<3.4.0 # bug with collection install
molecule-docker
usedevelop = False
skip_install = true
commands =
ansible-galaxy collection install -v 'community.docker:>=1.8.0'
ansible-galaxy collection install -v --force .
molecule test
[flake8]
ignore = H803
show-source = True
exclude = .tox,dist,doc,*.egg,build

1
tripleo_repos
View File

@ -1 +0,0 @@
plugins/module_utils/tripleo_repos

40
zuul.d/layout.yaml
View File

@ -1,40 +0,0 @@
- job:
name: tox-sanity-py39
description: Run ansible-test sanity tests on a collection
parent: openstack-tox-py39
vars:
tox_envlist: sanity
- project:
templates:
- check-requirements
- openstack-cover-jobs
- openstack-python3-zed-jobs
check:
jobs:
- openstack-tox-pep8:
vars:
tox_envlist: pep8,packaging
- openstack-tox-py39
- tripleo-tox-molecule
- tripleo-buildimage-overcloud-hardened-uefi-full-centos-9:
dependencies: &deps_unit_lint_cprovider
- openstack-tox-pep8
- openstack-tox-py38
- openstack-tox-py39
- tripleo-ci-centos-9-content-provider:
dependencies: *deps_unit_lint_cprovider
# TODO(marios) re-enable when https://bugs.launchpad.net/tripleo/+bug/2002112
# - tox-sanity-py39
gate:
jobs:
- openstack-tox-pep8:
vars:
tox_envlist: pep8,packaging
- openstack-tox-py39
- tripleo-buildimage-overcloud-hardened-uefi-full-centos-9:
dependencies: *deps_unit_lint_cprovider
- tripleo-ci-centos-9-content-provider:
dependencies: *deps_unit_lint_cprovider
post:
jobs:
- publish-openstack-python-branch-tarball