The Gatekeeper, or a project gating system
You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

setup.cfg 1.7KB

Use yarn and webpack to manage zuul-web javascript yarn drives package and dependency management. webpack handles bundling, minification and transpiling down to browser-acceptable javascript but allows for more modern javascript like import statements. There are some really neat things in the webpack dev server. CSS changes, for instance, get applied immediately without a refresh. Other things, like the jquery plugin do need a refresh, but it's handled just on a file changing. As a followup, we can also consider turning the majority of the status page into a webpack library that other people can depend on as a mechanism for direct use. Things like that haven't been touched because allowing folks to poke at the existing known status page without too many changes using the tools seems like a good way for people to learn/understand the stack. Move things so that the built content gets put into zuul/web/static so that the built-in static serving from zuul-web will/can serve the files. Update MANIFEST.in so that if npm run build:dist is run before the python setup.py sdist, the built html/javascript content will be included in the source tarball. Add a pbr hook so that if yarn is installed, javascript content will be built before the tarball. Add a zuul job with a success url that contains a source_url pointing to the live v3 data. This adds a framework for verifying that we can serve the web app urls and their dependencies for all of the various ways we want to support folks hosting zuul-web. It includes a very simple reverse proxy server for approximating what we do in openstack to "white label" the Zuul service -- that is, hide the multitenancy aspect and present the single tenant at the site root. We can run similar tests without the proxy to ensure the default, multi-tenant view works as well. Add babel transpiling enabling use of ES6 features ECMAScript6 has a bunch of nice things, like block scoped variables, const, template strings and classes. Babel is a javascript transpiler which webpack can use to allow us to write using modern javascript but the resulting code to still work on older browsers. Use the babel-plugin-angularjs-annotate so that angular's dependency injection doesn't get borked by babel's transpiling things (which causes variables to otherwise be renamed in a way that causes angular to not find them) While we're at it, replace our use of var with let (let is the new block-scoped version of var) and toss in some use of const and template strings for good measure. Add StandardJS eslint config for linting JavaScript Standard Style is a code style similar to pep8/flake8. It's being added here not because of the pep8 part, but because the pyflakes equivalent can catch real errors. This uses the babel-eslint parser since we're using Babel to transpile already. This auto-formats the existing code with: npm run format Rather than using StandardJS directly through the 'standard' package, use the standardjs eslint plugin so that we can ignore the camelCase rule (and any other rule that might emerge in the future) Many of under_score/camelCase were fixed in a previous version of the patch. Since the prevailing zuul style is camelCase methods anyway, those fixes were left. That warning has now been disabled. Other things, such as == vs. === and ensuring template strings are in backticks are fixed. Ignore indentation errors for now - we'll fix them at the end of this stack and then remove the exclusion. Add a 'format' npm run target that will run the eslint command with --fix for ease of fixing reported issues. Add a 'lint' npm run target and a 'lint' environment that runs with linting turned to errors. The next patch makes the lint environment more broadly useful. When we run lint, also run the BundleAnalyzerPlugin and set the success-url to the report. Add an angular controller for status and stream page Wrap the status and stream page construction with an angular controller so that all the javascripts can be bundled in a single file. Building the files locally is wonderful and all, but what we really want is to make a tarball that has the built code so that it can be deployed. Put it in the root source dir so that it can be used with the zuul fetch-javascript-tarball role. Also, replace the custom npm job with the new build-javascript-content job which naturally grabs the content we want. Make a 'main.js' file that imports the other three so that we just have a single bundle. Then, add a 'vendor' entry in the common webpack file and use the CommonsChunkPlugin to extract dependencies into their own bundle. A second CommonsChunkPlugin entry pulls out a little bit of metadata that would otherwise cause the main and vendor chunks to change even with no source change. Then add chunkhash into the filename. This way the files themselves can be aggressively cached. This all follows recommendations from https://webpack.js.org/guides/caching/ https://webpack.js.org/guides/code-splitting/ and https://webpack.js.org/guides/output-management/ Change-Id: I2e1230783fe57f1bc3b7818460463df1e659936b Co-Authored-By: Tristan Cacqueray <tdecacqu@redhat.com> Co-Authored-By: James E. Blair <jeblair@redhat.com>
2 years ago
Use yarn and webpack to manage zuul-web javascript yarn drives package and dependency management. webpack handles bundling, minification and transpiling down to browser-acceptable javascript but allows for more modern javascript like import statements. There are some really neat things in the webpack dev server. CSS changes, for instance, get applied immediately without a refresh. Other things, like the jquery plugin do need a refresh, but it's handled just on a file changing. As a followup, we can also consider turning the majority of the status page into a webpack library that other people can depend on as a mechanism for direct use. Things like that haven't been touched because allowing folks to poke at the existing known status page without too many changes using the tools seems like a good way for people to learn/understand the stack. Move things so that the built content gets put into zuul/web/static so that the built-in static serving from zuul-web will/can serve the files. Update MANIFEST.in so that if npm run build:dist is run before the python setup.py sdist, the built html/javascript content will be included in the source tarball. Add a pbr hook so that if yarn is installed, javascript content will be built before the tarball. Add a zuul job with a success url that contains a source_url pointing to the live v3 data. This adds a framework for verifying that we can serve the web app urls and their dependencies for all of the various ways we want to support folks hosting zuul-web. It includes a very simple reverse proxy server for approximating what we do in openstack to "white label" the Zuul service -- that is, hide the multitenancy aspect and present the single tenant at the site root. We can run similar tests without the proxy to ensure the default, multi-tenant view works as well. Add babel transpiling enabling use of ES6 features ECMAScript6 has a bunch of nice things, like block scoped variables, const, template strings and classes. Babel is a javascript transpiler which webpack can use to allow us to write using modern javascript but the resulting code to still work on older browsers. Use the babel-plugin-angularjs-annotate so that angular's dependency injection doesn't get borked by babel's transpiling things (which causes variables to otherwise be renamed in a way that causes angular to not find them) While we're at it, replace our use of var with let (let is the new block-scoped version of var) and toss in some use of const and template strings for good measure. Add StandardJS eslint config for linting JavaScript Standard Style is a code style similar to pep8/flake8. It's being added here not because of the pep8 part, but because the pyflakes equivalent can catch real errors. This uses the babel-eslint parser since we're using Babel to transpile already. This auto-formats the existing code with: npm run format Rather than using StandardJS directly through the 'standard' package, use the standardjs eslint plugin so that we can ignore the camelCase rule (and any other rule that might emerge in the future) Many of under_score/camelCase were fixed in a previous version of the patch. Since the prevailing zuul style is camelCase methods anyway, those fixes were left. That warning has now been disabled. Other things, such as == vs. === and ensuring template strings are in backticks are fixed. Ignore indentation errors for now - we'll fix them at the end of this stack and then remove the exclusion. Add a 'format' npm run target that will run the eslint command with --fix for ease of fixing reported issues. Add a 'lint' npm run target and a 'lint' environment that runs with linting turned to errors. The next patch makes the lint environment more broadly useful. When we run lint, also run the BundleAnalyzerPlugin and set the success-url to the report. Add an angular controller for status and stream page Wrap the status and stream page construction with an angular controller so that all the javascripts can be bundled in a single file. Building the files locally is wonderful and all, but what we really want is to make a tarball that has the built code so that it can be deployed. Put it in the root source dir so that it can be used with the zuul fetch-javascript-tarball role. Also, replace the custom npm job with the new build-javascript-content job which naturally grabs the content we want. Make a 'main.js' file that imports the other three so that we just have a single bundle. Then, add a 'vendor' entry in the common webpack file and use the CommonsChunkPlugin to extract dependencies into their own bundle. A second CommonsChunkPlugin entry pulls out a little bit of metadata that would otherwise cause the main and vendor chunks to change even with no source change. Then add chunkhash into the filename. This way the files themselves can be aggressively cached. This all follows recommendations from https://webpack.js.org/guides/caching/ https://webpack.js.org/guides/code-splitting/ and https://webpack.js.org/guides/output-management/ Change-Id: I2e1230783fe57f1bc3b7818460463df1e659936b Co-Authored-By: Tristan Cacqueray <tdecacqu@redhat.com> Co-Authored-By: James E. Blair <jeblair@redhat.com>
2 years ago
Add web-based console log streaming zuul now provides socket-based console streaming, which is super cool. In order to have jenkins parity with web streaming, we need to provide a websocket (javascript in browsers can't really connect to random ports on servers) After surveying the existing python websocket options, basically all of them are based around twisted, eventlet, gevent or asyncio. It's not just a thing we can easily deal with from our current webob/paste structure, because it is a change to the fundamental HTTP handling. While we could write our own websocket server implementation that was threaded like the rest of zuul, that's a pretty giant amount of work. Instead, we can run an async-based server that's just for the websockets, so that we're not all of a sudden putting async code into the rest of zuul and winding up frankensteined. Since this is new code, using asyncio and python3 seems like an excellent starting place. aiohttp supports running a websocket server in a thread. It also supports doing other HTTP/REST calls, so by going aiohttp we can set ourselves up for a single answer for the HTTP tier. In order to keep us from being an open socket relay, we'll expect two parameters as the first message on the websocket - what's the zuul build uuid, and what log file do we want to stream. (the second thing, multiple log files, isn't supported yet by the rest of zuul, but one can imagine a future where we'd like to support that too, so it's in the protocol) The websocket server will then ask zuul over gearman for the IP and port associated with the build and logfile and will start streaming it to the socket. Ultimately we'll want the status page to make links of the form: /console.html?uuid=<uuid>&logfile=console.log and we'll want to have apache map the websocket server to something like /console. Co-Authored-By: Monty Taylor <mordred@inaugust.com> Change-Id: Idd0d3f9259e81fa9a60d7540664ce8d5ad2c298f
4 years ago
12345678910111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061
  1. [metadata]
  2. name = zuul
  3. summary = A Project Gating System
  4. description-file =
  5. README.rst
  6. author = Zuul Team
  7. author-email = zuul-discuss@lists.zuul-ci.org
  8. home-page = https://zuul-ci.org/
  9. python-requires = >=3.5
  10. classifier =
  11. Intended Audience :: Information Technology
  12. Intended Audience :: System Administrators
  13. License :: OSI Approved :: Apache Software License
  14. Operating System :: POSIX :: Linux
  15. Programming Language :: Python
  16. Programming Language :: Python :: 3
  17. Programming Language :: Python :: 3.5
  18. Programming Language :: Python :: 3.6
  19. Programming Language :: Python :: 3.7
  20. Programming Language :: Python :: 3.8
  21. [pbr]
  22. warnerrors = True
  23. [global]
  24. setup_hooks =
  25. zuul._setup_hook.setup_hook
  26. [entry_points]
  27. console_scripts =
  28. zuul-scheduler = zuul.cmd.scheduler:main
  29. zuul-merger = zuul.cmd.merger:main
  30. zuul = zuul.cmd.client:main
  31. zuul-executor = zuul.cmd.executor:main
  32. zuul-bwrap = zuul.driver.bubblewrap:main
  33. zuul-web = zuul.cmd.web:main
  34. zuul-migrate = zuul.cmd.migrate:main
  35. zuul-fingergw = zuul.cmd.fingergw:main
  36. zuul-manage-ansible = zuul.cmd.manage_ansible:main
  37. [build_sphinx]
  38. source-dir = doc/source
  39. build-dir = doc/build
  40. all_files = 1
  41. warning-is-error = 1
  42. [extras]
  43. mysql_reporter=
  44. PyMySQL
  45. migrate=
  46. jenkins-job-builder==1.6.2
  47. # The container image build process will look at the python extras and install
  48. # any that match the target container name. It makes a container image for
  49. # every console_script listed above. The extra name must have underscores.
  50. # This is a place to put dependencies that should go into reasonable container
  51. # images but that we do not want to list directly in requirements.txt.
  52. zuul_base =
  53. PyMySQL
  54. psycopg2-binary
  55. yappi
  56. objgraph