Reworks the publish playbooks and the post-run jobs
for the Puppet release job.
Depends-On: https://review.openstack.org/#/c/635941/
Change-Id: Ia9b149d3e1e94331b047d9bc762359d09ce81b20
The site_logs secret moved into opendev/basejobs and we renamed the
site_logs secret here into site_logs_project_config and site_specs.
Unfortauntely many of our playbooks in this repo were not updated to use
the new secret name so they cause some jobs to fail.
Fix this by updated the playbooks to use site_logs_project_config and
site_specs as appropriate.
Change-Id: I852a6362c35dd6aa17b0f767fceea811402dd90c
Changes the hosts field from localhost to all
so that the publish playbook is executed in
the cloud instead of locally on the zuul executor
node.
This is required because when it's executed on the
zuul executor the security context supresses the
facts as a security measure which causes the job to fail.
Change-Id: Iab2252ca647bd6d8eaff6c48f458c566c0393cb2
This is a follow-on to I3761dd9d2ab752e66bcccdd2434405f11535b95f where
I misunderstood what "linear" was doing. What was really intended was
to stop both the python2 and python3 builds copying to AFS at the same
time. "linear" just means the *tasks* start at the same time across
hosts; not that they run serially. To do that you want the (aptly
named) "serial" specifier on the playbook.
Add this so we do the copies sequentially
Change-Id: I4f8a2f2c6c8164e7ea207f8e4b286e06df0b13dd
Publish using tox-docs instead of using sphinx directly.
Change-Id: If409fc54eb6777e63e3998e513ff1120d23cefca
Depends-On: https://review.openstack.org/608442
prepare-for-afs includes already the root-marker wirter, an extra
write-root-marker is not needed and wrong - we currently end with
stable/queens job having two root-markers:
html/.root-marker
html/queens/.root-marker
Remove the extra root-marker.
Change-Id: I9a7de9ec372266b3b6dcd650b69f1ec401ef0b0c
We saw what looks very much like the index generation racing, with
"mv" unable to close the index file ...
Move to a linear strategy so that each host finishes each task before
moving on. Then add run_once to the index generation so only one of
the hosts in a python2/3 situation creates the index file.
Change-Id: I3761dd9d2ab752e66bcccdd2434405f11535b95f
This job is to trigger readthedocs via their new API
The old API was quite simple in that anyone could hit the end-point
and trigger a new build. The v2 API now requires authentication and a
unique id for each project we wish to update.
If projects wish, they can directly use the new "trigger-readthedocs"
role (from the dependent change) and create their own jobs, providing
their username/password or webhook authentication token directly from
a secret in their project playbook.
This job, however, uses the infra secret for the "openstackci" user to
update the docs. Projects that wish to take this route require some
minor setup, but don't have to worry about managing new jobs or their
secrets and gain some benefits in having the openstackci user as a
backup admin.
To use the trigger-readthedocs-webhook job, projects will need to do
the following:
1) add openstackci user as an admin of their RTD project
2) generate an webhook via the "Integrations" dashboard page on the
RTD project
3) provide the id from that webhook URL in job or project variables.
This webhook id is not private; it is useless without
authentication (it appears to be a global monotonically increasing
integer for each webhook generated).
4) include the job. gerrit-dash-creator is populated with the new job
for initial testing. Other projects are currently using the
"docs-on-readthedocs" template. Change
I3b65813671961d07c0a743685c537ad1df4bf68 proposes switching the
template to use this new job.
Change-Id: Ic34ea5ead26a477a1cc5f8b25fd2ce1949c1dc13
Depends-On: https://review.openstack.org/579434
Depends-On: https://review.openstack.org/584230
We duplicate the afs upload in many playbooks, use instead the now
parameterized playbooks/publish/openstack-afs playbook.
Change-Id: I3c6b9b5eb82a2628411571299eaedbf11f0e5d56
Rework the playbook so that it takes an additional parameter and we can
use it in more places (see followup change). Set the current value
as default
Note that we really need to allow a value of "{{ afs.path }}" for the
variable without trailing "/" so that publishing of openstack-manuals
works which publishes to the root folder.
Change-Id: I15b77fc002a6b099ff23e0c07972b49ad743a2c9
The new protected attribute means that only jobs in the same repo can
inherit from it. Use that on base-publish-static and move the secrets
from the inherited job to it now, it's now safe to have them here.
Since the secrets need to be available to the post-job and are not
available to post-jobs of inheried jobs, use a common post job for all
of these. Move static-special.yaml to static.yaml and set the location
for the publish location for the single previous user of static.yaml.
Change-Id: I3d749adf242eb87895ad10b6f9c4d12859882ac9
According to notes in the Ansible documentation for the copy module,
it "does not scale to lots (>hundreds) of files." Several of the 5
sites currently relying on the publish/static-special playbook have
content numbering in the thousands of files. Other sites on the same
server instead rely on the publish/static playbook which already
uses synchronize instead of copy, so this should be entirely
compatible.
Change-Id: Iac85d906ae7c7a4666d7a5eeaa73b5a9f2ac7115
Add "/" to sphinx dir to to remove the final "html" from path to fix
publishing.
Also, add "/" to playbooks/publish/publications.yaml.
Change-Id: Ie90b3204b2530c30468a28f107f74794235a0921
Depends-On: I0eed5ebf5af8d43f6a105a1fc0dcf1b0a9813dbd
This gets the main publish-openstack-sphinx jobs, plus api-ref,
api-guide, deploy-guide, install-guide and placement-api-ref.
The write_root_marker and prepare-docs-for-afs roles have been udpated
to have good defaults and also to honor sphinx_build_dir as a variable,
so remove setting those explicitly.
Depends-On: I7c3d578a00a4aa4c9e175f5f83e3171c6f6a4a61
Change-Id: I9ed0b02d6ca20a6b679507c4d988531abc55b09d
The ocata and earlier guides have been publishing to the service-type,
not the project-name. The Zuul v3 conversion broke this, restore this.
Note that this still results in some different names since we use the
services-type-authority now and formerly had hand-written assignments.
Change-Id: Ib03cc560868db725e151e0e8c9107846137f5d27
This allows us to add the build-wheel-mirror jobs to
openstack/requirements check pipelines, but not release wheels to afs.
Making it easier to properly debug these jobs, otherwise an infra-root
must manually enqueue-ref for periodic pipeline.
Change-Id: Iabedbfeb62f1773d10462e8b07bbe82f1fb76389
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
The build job was modified to not use tox in
I56909152975f731a9d2c21b2825b972195e48ee8. Fix up the argument and
don't try to get the tox output that won't exist any more.
Change-Id: I0b470df7b5e9c772a36f25a549a46f059be3a41b
Add native zuul v3 job to publish index file of
openstack-infra/publications.
Run the post job on all branches, so that a change to the master branch
regenerates the index file.
Add native zuul v3 job to publish a specific publication from
openstack-infra/publications.
Use the job for the project.
Needed-By: I190b34217c87d45e13bca18c7e121189323b698f
Change-Id: I6da66649726a1a96d06d7ca643adeec8ea8649eb
We just lost the docs.o.o index page: The tox run failed and only a
few files were ready. Since we delete existing content when pushing new,
this removed all our static pages.
Add a safe guard for all doc building: Only collect and publish
documents when we have build them with success. Add this to related
roles like creating/destroying afs as well.
Change-Id: Ife2496fe920bd40d52803705b409e6e621532254
If we failed to build a wheel / tarball, we don't actually want to
attempt an upload. Checking the zuul_status results will provide that
information.
Change-Id: If0afb88f5063fe11e664a0393528933911da667b
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
New job to publish the old project install guide. This runs only
on newton and ocata.
Needed-By: Ide665a796bd80173b1e2248dfc8a7ad412c6c8bc
Change-Id: I85c101b83f745196bc6f8673c26f3864d61ff3f5
Create new publish-placement-api-ref job and use the new
build-placement-api-ref job in nova.
Depends-On: Ib50a094db845f63d802c0f76f4a9d44517f401fa
Needed-By: Ia62e5f9f14716d3595b2d14edfe0781d4e7af9ac
Change-Id: I6c57d43b464a26eb396f3c12d5a14b6b21fa2ef8
The Documentation team has previously used /contributor-guide for their
own on-boarding documentation. This publishes the general contributor
guide to /contributors.
Change-Id: I223610d480b7754176dd00366ae093676cf612c9
We publish deploy-guide to /latest or /branchname - like we do with
normal documents. Fix location for this.
We can use the same logic as for normal repos, keeping in mind that the
docs live under deploy-guide, not doc.
Depends-On: I61cd84be201d14d0a1d5daba3a45c8797496bc7f
Change-Id: Ifff767a95e6283c54033bababd3dae84bb19adb6
This fixes the following error:
Destination directory src/git.openstack.org/openstack/neutron/html does not exist
Change-Id: Ifb0cb886498724488b904c578aaf8bc20c2ab350
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
We should be publishing api-ref and api-guide content to a project's
service-type, not to its project name. Use the service-types.json file
that we publish from the service-types-authority since that's the
official mapping.
Use the set-service-type-data-fact role from openstack-zuul-jobs to get
the data.
Depends-On: Iea42ba981cee260cbccd0b1eb2705d3ad419bf20
Change-Id: Ib191c63a6899a8c04955bb9ee9e0ee0c8edbc802
Create new publish-api-ref and publish-api-guide jobs.
Change-Id: I9ca650def0c69111f5e34a90394399e8bc113960
Needed-By: I65384ab90d4639b8def2992ae965e2d037e9408d