Commit Graph

542 Commits

Author SHA1 Message Date
Jens Harbott
540bdac7ff Fix wheel release playbook
Somehow an empty role invocation without setting the needed variable
ended up there.

Change-Id: I0a2ab8e4c119f1efcaf7e4231efb1804301637d8
2020-06-23 14:50:49 +02:00
Jeremy Stanley
3f72ab620c Add ensure-tox to pti-python-tarball
The publish-openstack-python-branch-tarball job run by many projects
uses tox (in python-branch-tarball/post.yaml), so needs it installed.

Change-Id: Iba219ad14523ce900d9562ef56c78cc6c15aa01d
2020-06-18 21:43:32 +00:00
Andreas Jaeger
6376aa1341 Fix translation jobs
The translation jobs use scripts that expect tox pre-installed, install
tox globally for them.

Change-Id: I633c868a96721f38f063f8e057b0b3c950d5d3e9
2020-06-18 20:05:02 +02:00
Sean McGinnis
9c91006304 Ensure pip is installed for propose-update-constraints
Pip is used by this task, make sure ensure-pip is used so it is
present.

While here, install git-review to a venv and symlink it globally to
keep things separate.

Change-Id: If6b1abd2545c68f727bddc5317797236f451b67c
Signed-off-by: Sean McGinnis <sean.mcginnis@gmail.com>
Signed-off-by: Ian Wienand <iwienand@redhat.com>
2020-06-17 16:02:46 +10:00
Zuul
6f2b723a9c Merge "Install git-review under python3 for proposed updates" 2020-06-11 23:45:48 +00:00
Ian Wienand
92b378cc9e Don't release centos7 aarch64 volume
We ended up not creating centos7 arm64 wheels and don't have this
volume, don't try and release it.

Change-Id: Ibe4d459ce6c9c6e26330944f30272925b930ba3d
2020-06-11 11:12:05 +10:00
Clark Boylan
8ee0a3a869 Install git-review under python3 for proposed updates
We were installing git-review under python2 in order to propose updates
to Gerrit. git-review happily runs under python3 so update the
installation to avoid python2.

Change-Id: I1671345d71cd60f6508a2c067c4e308680234afe
2020-06-10 15:12:34 -07:00
Andreas Jaeger
b59a59da35 Use python 3 venv for generate-constraints
With change I15d451f6a832b2e698b28e7351a36ef7aea92abe
generate-constraints needs python 3, setup the venv using python 3.

Change-Id: I3dba35267dc6f344ad27910c7dc18c1a33abf4ba
2020-06-07 08:39:19 +02:00
Ian Wienand
6c001b528c Move wheel build role/job outside project-config
We can not speculatively test changes to the wheel build jobs while it
is project config.  These jobs are in the gate for the requirements
project, which run them when bindep.txt changes.

https://review.opendev.org/731394 moved the build-wheel-mirror-base
job/playbook/role into openstack-zuul-jobs, renamed as
build-wheel-cache-* to a) distinguish the name and b) make it a little
clearer that we're building the wheels that goes on the mirror, rather
than passing through to mirroring something existing.

This updates the job names in project-config to reference these new
jobs.

Note the publishing steps are kept here along with the AFS secret.

Depends-On: https://review.opendev.org/732085
Depends-On: https://review.opendev.org/732087
Depends-On: https://review.opendev.org/732083
Depends-On: https://review.opendev.org/732084
Depends-On: https://review.opendev.org/732086

Change-Id: Iac3a906803177e8365f4cfb611800b5ccaed4a6e
2020-06-01 20:10:42 +10:00
Andreas Jaeger
8b09e0cf98 Add root_marker_dir for releasenotes publish
To use the write-root-marker role from opendev/base-jobs, set
the root_marker_dir to the same value that is currently set
by the openstack-zuul-jobs role.

Needed-By: https://review.opendev.org/731460
Change-Id: I169f069aa458bc7d02f9c0295702c1316fff825f
2020-05-28 10:32:58 +02:00
Zuul
463eb880ac Merge "build and publish wheels for aarch64 architecture" 2020-05-27 14:46:11 +00:00
Zuul
8d307eb8ca Merge "build and publish wheels for more distributions (x86-64)" 2020-05-27 14:46:09 +00:00
Marcin Juszkiewicz
541daaa3ff build and publish wheels for aarch64 architecture
- CentOS 7, 8
- Debian 'buster'
- Ubuntu 'focal', 'bionic', 'xenial'

Change-Id: Iebb1062269011664411691ef8fd55219afaccd1e
2020-05-27 14:30:44 +10:00
Ian Wienand
8892414842 Match Debian in AFS slug generation
So we can build/publish Debian wheels

Change-Id: Idac3455109f7d1eddbd50ffa0535e873e6eee2db
2020-05-27 10:20:00 +10:00
Andreas Jaeger
8e607896c4 Fix sphinx playbook: install-if-python was renamed
The new name is ensure-if-python, use it to unbreak the sphinx job.

Change-Id: If03c299eea48aa73e0ef320d0f4903b2effb3ba1
2020-05-26 15:42:45 +02:00
Marcin Juszkiewicz
3445673eea build and publish wheels for more distributions (x86-64)
- CentOS 8
- Debian 'buster'
- Ubuntu 'focal'

Change-Id: I767359d48e04473fad0992b35c804ed511cfaa62
2020-05-22 15:44:17 +02:00
Zuul
2e9c2c6fd0 Merge "install python3.8 now that victoria is in development" 2020-05-17 18:59:57 +00:00
Matthew Thode
dc799bd092
install python3.8 now that victoria is in development
Change-Id: I7758b3879215f8977be82bba273e771d84cd3155
2020-05-17 13:32:40 -05:00
Matthew Thode
5b69bb5edd
drop python2.7 from generate-constraints
Change-Id: I5e51b46afbc26332368525f61a84ff9bd0536a1e
2020-05-15 18:36:42 -05:00
Andreas Jaeger
c6d2d35efc Finish retiring x/pbrx
Depends-On: https://review.opendev.org/726462
Change-Id: I0a5b2a352cdbf443976b18db4e4d210e4c7af581
2020-05-08 21:32:16 +02:00
Andreas Jaeger
cdc4871cae Fix typo in playbooks/yaml2ical/post.yaml
Fix:
  rsync: mkdir "/src/yaml2ical/calendars" failed: No such file or directory (2)

It's /srv, not /src as a few lines above.

Change-Id: I86326fa7de68a0db7e6893e8ab291ba33fe6018d
2020-05-02 16:36:47 +02:00
Andreas Jaeger
2808da5bad Fix irc-meetings publishing
The periodic job fails, example:
https://zuul.opendev.org/t/openstack/build/85f380fb6dde462e85a09bc9b5105ba9/

error is:
rsync: change_dir "/home/zuul//src/opendev.org/openstack/project-config/output" failed: No such file or directory (2)

Problem is that the output directory is in the irc-meetings directory
instead.

Use correct path.

Change-Id: I8e15cbfc9a0c01066de68730709c8c30b234c5f5
2020-05-02 09:24:50 +02:00
Zuul
d2be1e943f Merge "Use zuul deployment keys for yaml2ical" 2020-04-25 15:16:58 +00:00
Sorin Sbarnea
5ea7f35c0a Enable promote to unarchive gz archives in addition to bz2
This should enable us to change archive types.

Change-Id: Id79a4ece4eb9e658afc75cb653bd5e02c3f47d0d
Needed-By: https://review.opendev.org/#/c/715028/
2020-04-22 09:08:07 +01:00
Monty Taylor
295224c41b Use zuul deployment keys for yaml2ical
We're currently adding the private key of a jenkins user. But we
don't use jenkins, so this is lame. Use the zuul deploy key for
project-config instead.

This needs to run as project-config so that per-project deploy
keys work. To do that, shift it from being triggered by irc-meetings
to being triggered hourly by project-config.

As a followup, we should probably convert this to just publish to
AFS and serve the content from there.

Depends-On: https://review.opendev.org/721098
Change-Id: I7874ef46a616e8fd68cf8d95afc3928d3440ba51
2020-04-21 12:40:25 -05:00
Matthew Thode
45ed2e8fa8
add py38 to constraint bot so victoria work can begin
after requirements branches we will remove py27 and py37

Change-Id: I5fde605634108dd628df546a2318731e1832b6b3
Signed-off-by: Matthew Thode <mthode@mthode.org>
2020-04-12 11:24:02 -05:00
Andreas Jaeger
0aedb7733c Use ensure-* roles
The install- roles have been renamed to ensure-*, follow the rename.

Depends-On: https://review.opendev.org/716698/
Change-Id: I2522028e9091efdbe292529474de7cce2cf29039
2020-04-06 19:49:19 +02:00
Ian Wienand
0326363e48 Update ansible-lint and add fake zuul_return
- Use zuul_return fake module to make ansible lint happy, this allows
  to remove Zuul import.
- While ansible-lint 4.2.0 is now able to detect playbooks/roles, this
  is broken, so don't use it
- move its config out of tox.ini so it can be used by any tools, or
  without tox

Change-Id: Ie8935f47db855647e19ae091044e5ac1871f1551
Co-Authored-By: Sorin Sbarnea <ssbarnea@redhat.com>
Co-Authored-By: Andreas Jaeger <aj@suse.com>
2020-03-18 22:28:10 +01:00
Andreas Jaeger
f1acd97b9d Fix translation-source location
The previous change published to
https://tarballs.opendev.org/translation-source/

but we should publish to
https://tarballs.opendev.org/openstack/translation-source/ .

Fix it. We should remove the files from
https://tarballs.opendev.org/translation-source/ afterwards.

Change-Id: I46a857800326db2ad06be276da65279852cc4a90
2020-03-06 07:11:16 +01:00
Clark Boylan
2e12d5df0b Fix afs publishing of translation sources
There are at least two problems here. The first is we can't create an
afs token because we don't have access to the afs secret. The old ssh
based secret was removed by:

  https://review.opendev.org/#/c/706734/2/zuul.d/jobs.yaml

Fix that by adding the new opendev tarballs secret to the job.

Next we need to prefix translation sources with translation-source/ in
the filesystem path. This requires us to write out to a different afs
path than normal. To address that we add a new playbook that is aware of
this different path.

Change-Id: I8b49e4d91600377c98e003b806a13fc89d2614ec
2020-03-05 13:18:29 -08:00
Ian Wienand
cb982781d0 Remove base-ozj job
I do not believe this is used any more.

Story: #2006598
Task: #37735
Change-Id: If8ee6fbbb67c6d1904ba78ce7a9c84fd1e2d7b92
2020-03-04 16:09:25 +11:00
Ian Wienand
391dc9388a Remove base-minimal job
I believe this is now replaced by the base-minimal job from
opendev/base-jobs [1].  This hasn't been updated to upload logs to
swift, so it hasn't been used in quite some time.

[1] 9b235be0c0/zuul.d/jobs.yaml (L96)

Story: #2006598
Task: #37735
Change-Id: I097142dba378b7913e796d06c5a393d86e9e0bbf
2020-03-04 16:09:25 +11:00
Zuul
041bb832bb Merge "Move stackviz publish in-tree" 2020-03-04 04:13:37 +00:00
Andreas Jaeger
fd782318e4 Remove site_specs and playbooks/publish/service-types.yaml
playbooks/publish/service-types.yaml is not used anymore, with AFS
publishing it got obsoleted and can now be removed.

site_specs and site_specs_promote are not used anymore, remove them.

Change-Id: I0c3ca2455b4f8b721d48d44780dbc397ddc9cb20
2020-03-03 21:30:26 +01:00
Andreas Jaeger
15e763dc6a Move stackviz publish in-tree
The new job publish-openstack-stackviz-element is working now,
move the trigger in-tree and change the publish location, we do not need
the "-new" in the name anymore.

Content is published at:
https://tarballs.opendev.org/openstack/stackviz/dist/
The tarballs are equal, only difference is filenames of sources.

Mark the job to run only on master, since it publishes always to the
same location and each invocation overrides the file. This is a
safety measure.

Depends-On: https://review.opendev.org/710754
Change-Id: If9488d7c60b58f51a7a0a4bdf0740d16f61c241e
2020-03-03 16:03:24 +01:00
Andreas Jaeger
8cadacce53 Don't use zuul_work_dir in post-stackviz
The stackviz publish job fails with
"The task includes an option with an undefined variable. The error was:
'zuul_work_dir' is undefined"

Use zuul.project.src.dir instead of zuul_work_dir.

Change-Id: I443d0b4bfb30a06d68321d402802ae1b433bab6b
2020-03-02 10:43:46 +01:00
Andreas Jaeger
14ccbb2e84 Rework post-stackviz to fix uploading
The fetch-javascript-output does not work with the artifact uploading,
it does not create an artifact directory on the server.

Remove use of the role and instead copy the needed bits of
fetch-javascript-content-tarball into the playbook.

Change-Id: I3ed46c1acf023b6d2b323f56ce6bdfe07f70775f
2020-02-29 09:57:45 +01:00
Andreas Jaeger
41e5721d38 Rework stackviz elements publishing
We cannot change the javascript-content job as desired, use
specific post job to rename and fetch the tarball.

Change-Id: I63872db707bd674ebfaa9bd49a1cdf3ae6906a60
2020-02-28 17:56:04 +01:00
Andreas Jaeger
aafb5cbee7 fix stackviz publishing
Move setting of npm_command from playbook to job so that it can
be overriden for stackviz.

Change-Id: I4992584cb88ba715ddc7727bba65b7e853a6c7f5
2020-02-27 12:58:43 +01:00
Andreas Jaeger
d2ed9a0df7 Fix stackviz periodic job
Fix typo in zuul variable.
Use prod, not build as npm command.

Change-Id: I89d408f1ebed8f0051a735883ab40ade5e2c07df
2020-02-27 11:03:17 +01:00
Andreas Jaeger
a0cf998b39 Add publish job for stackviz
stackviz gets published daily to:
https://tarballs.opendev.org/openstack/stackviz/dist/stackviz-latest.tar.gz

The current job periodic-package-stackviz-element is a legacy job,
create a new job publish-openstack-javascript-latest-content that can be
run instead.

The stackviz-element is backed into our images and thus we have to be
extra careful not to break it and test.

The new job is publishing temporarily under a different filename to
check that it's working correctly.

Order of changes:
* Merge this change, so we run both periodic-package-stackviz-element and
  publish-openstack-javascript-latest-content in stackviz.
* Change new job to publish under correct name and remove setup in
  project-config.
* Switch to new job in stackviz repo.

Change-Id: I81ac0975f03756da673ae12e7e11cd5686a0f2d6
2020-02-26 17:52:46 +01:00
Zuul
3d5046247b Merge "Remove obsolete releases publishing" 2020-02-24 18:37:58 +00:00
Zuul
fb8b248533 Merge "Remove unused files in playbooks/publish" 2020-02-24 18:25:33 +00:00
Andreas Jaeger
de9c644dce Fix promote-service-types-authority
This job only publishes part of the directory and needs to keep old content
around. Thus, add a new promote playbook that uses the
upload-afs-synchronize rule to not delete old content like the normal
playbook in the parent does.

Change-Id: I13f9faf698ec459f1d7cd7a950172d836b470a4c
2020-02-21 08:43:01 +01:00
Andreas Jaeger
e46f938f80 Remove obsolete releases publishing
Remove publish-tox-docs-static, the last user has been converted to AFS
publishing.

Remove the parent base-publish-tox-docs-static as well, it's not used
anymore.

Remove playbook files and secrets used by these jobs.

Depends-On: https://review.opendev.org/708918
Change-Id: I68b3ab7e597e230617ec1eaa8217d4f5f4c5fb15
2020-02-20 21:47:34 +01:00
Andreas Jaeger
103c6388fc Remove unused files in playbooks/publish
We changed a lot of jobs to promote jobs and forgot to cleanup,
remove now unused playbook files.

Change-Id: I9e3da882425e6d1aae3d374b681e472492bee79b
2020-02-20 18:13:52 +01:00
Andreas Jaeger
2e6eaa678f Add AFS publish job for releases repo
Add new job publish-tox-docs-releases for publishing of releases repo to
new AFS location to obsolete static.o.o publishing.

Task: #37724
Story: #2006598
Needed-By: https://review.opendev.org/708918
Change-Id: I804f6ddb0b10b4aa3fac6a3cf11a2d23b8b4e730
2020-02-20 18:04:14 +01:00
Zuul
3e0549776b Merge "Remove tarball artifact publishing to static site" 2020-02-20 05:58:53 +00:00
Andreas Jaeger
f783775ea7 Switch specs publishing to AFS
Move specs publishing directly to publish to AFS.

Remove now unused file playbooks/static-docs/promote.yaml.

Depends-On: https://review.opendev.org/708507
Task: #37721
Story: #2006598
Change-Id: I0773fc958f5e2169ba46e9884919470a0214748d
2020-02-18 22:14:40 +01:00
Ian Wienand
e6dc1b343f Remove tarball artifact publishing to static site
Following-on from Ia3a0358249e9ed3d766b1b61535f2f6d67d4eb2d, this
removes the publishing from the static site, which is now happening to
AFS.

The site_tarballs secret is no longer required.  The two jobs still
using it have a parent of publish-openstack-artifacts so should not
need a separate secret.

Depends-On: https://review.opendev.org/706732
Change-Id: I1b1db7d2451d7fb1fdd7921e7c9efd0e020fbce6
2020-02-10 16:39:06 +11:00