From 86f10e29f7f15f69936ac40df5d19d2483c5bfdb Mon Sep 17 00:00:00 2001 From: Donny Davis Date: Wed, 21 Aug 2019 08:36:37 -0400 Subject: [PATCH] Cleanup index / add in driver support matrix Change-Id: Ie20d4b00104ca1fb37ff2429a366d48ce3a44508 --- doc/requirements.txt | 1 + doc/source/{admin => api}/api.rst | 0 doc/source/conf.py | 1 + doc/source/configuration/index.rst | 2 +- doc/source/index.rst | 35 ++--- .../install/{from-source.rst => common.rst} | 133 ++++++++++++------ doc/source/install/common_prerequisites.rst | 105 -------------- ...{installation.rst => install-from-pip.rst} | 8 +- doc/source/install/install-from-source.rst | 54 +++++++ doc/source/reference/support-matrix-fpga.ini | 77 ++++++++++ doc/source/reference/support-matrix-gpu.ini | 56 ++++++++ doc/source/reference/support-matrix.rst | 85 +++++++++++ 12 files changed, 392 insertions(+), 165 deletions(-) rename doc/source/{admin => api}/api.rst (100%) rename doc/source/install/{from-source.rst => common.rst} (50%) delete mode 100644 doc/source/install/common_prerequisites.rst rename doc/source/install/{installation.rst => install-from-pip.rst} (64%) create mode 100644 doc/source/install/install-from-source.rst create mode 100644 doc/source/reference/support-matrix-fpga.ini create mode 100644 doc/source/reference/support-matrix-gpu.ini create mode 100644 doc/source/reference/support-matrix.rst diff --git a/doc/requirements.txt b/doc/requirements.txt index 6a7ab091..02f78f07 100644 --- a/doc/requirements.txt +++ b/doc/requirements.txt @@ -4,6 +4,7 @@ sphinx!=1.6.6,!=1.6.7,>=1.6.2,<2.0.0;python_version=='2.7' # BSD sphinx!=1.6.6,!=1.6.7,>=1.6.2;python_version>='3.4' # BSD +sphinx-feature-classification>=0.1.0 # Apache 2.0 sphinxcontrib-httpdomain>=1.3.0 # BSD sphinxcontrib-pecanwsme>=0.2 # Apache-2.0 sphinxcontrib-seqdiag>=0.8.4 # BSD diff --git a/doc/source/admin/api.rst b/doc/source/api/api.rst similarity index 100% rename from doc/source/admin/api.rst rename to doc/source/api/api.rst diff --git a/doc/source/conf.py b/doc/source/conf.py index 03bacab5..578a3a5e 100755 --- a/doc/source/conf.py +++ b/doc/source/conf.py @@ -22,6 +22,7 @@ sys.path.insert(0, os.path.abspath('../..')) # extensions coming with Sphinx (named 'sphinx.ext.*') or your custom ones. extensions = [ 'sphinx.ext.autodoc', + 'sphinx_feature_classification.support_matrix', #'sphinx.ext.intersphinx', 'openstackdocstheme', 'oslo_config.sphinxconfiggen', diff --git a/doc/source/configuration/index.rst b/doc/source/configuration/index.rst index a8cde7ca..ffc27b9c 100644 --- a/doc/source/configuration/index.rst +++ b/doc/source/configuration/index.rst @@ -3,7 +3,7 @@ Configuration Guide =================== .. toctree:: - :maxdepth: 1 + :maxdepth: 2 config-options sample_config diff --git a/doc/source/index.rst b/doc/source/index.rst index ab0eb54c..4ba4d33c 100644 --- a/doc/source/index.rst +++ b/doc/source/index.rst @@ -14,16 +14,19 @@ Overview user/architecture user/usage -Configuration -------------------- +API +~~~ .. toctree:: :maxdepth: 1 - configuration/index + api/api -User Documentation -------------------- +Documentation for Operators +---------------------------- + +The documentation in this section is aimed at Cloud +Operators needing to install or configure Cyborg. Installation ~~~~~~~~~~~~ @@ -31,20 +34,21 @@ Installation .. toctree:: :maxdepth: 1 - install/installation - install/from-source + install/install-from-pip + install/install-from-source admin/config-wsgi -API -~~~ +Configuration Reference +~~~~~~~~~~~~~~~~~~~~~~~ .. toctree:: - :maxdepth: 1 + :maxdepth: 2 - admin/api + configuration/index + reference/support-matrix -CLI Reference -------------- +Documentation for End Users +---------------------------- Information on the commands available through Cyborg's Command Line Interface (CLI) can be found in this section of documentation. @@ -55,9 +59,8 @@ Interface (CLI) can be found in this section of documentation. cli/index - -Developer Documentation ------------------------ +Documentation for Developers +---------------------------- .. toctree:: :maxdepth: 1 diff --git a/doc/source/install/from-source.rst b/doc/source/install/common.rst similarity index 50% rename from doc/source/install/from-source.rst rename to doc/source/install/common.rst index d35bfc41..07a6178d 100644 --- a/doc/source/install/from-source.rst +++ b/doc/source/install/common.rst @@ -1,49 +1,111 @@ -========================== -Install Cyborg from Source -========================== - -This section describes how to install and configure the Acceleration Service -for Ubuntu 18.04.1 LTS from source code. - -.. include:: common_prerequisites.rst - -Install and Configure +Common Configuration --------------------- -#. Create a folder which will hold all Cyborg components. +Regardless of the package or code source you must do the following +to properly setup the Accelerator Life Cycle Management service. +A database, service credentials, and API endpoints must be created. - .. code-block:: console +#. To create the database, complete these steps: - mkdir ~/cyborg + * Use the database access client to connect to the database + server as the ``root`` user: - .. + .. code-block:: console -#. Clone the cyborg git repository to the management server. + $ mysql -u root -p + .. - .. code-block:: console + * Create the ``cyborg`` database: - cd ~/cyborg - git clone https://opendev.org/openstack/cyborg - .. + .. code-block:: mysql -#. Set up the cyborg config file + CREATE DATABASE cyborg; + .. - First, generate a sample configuration file, using tox + * Grant proper access to the ``cyborg`` database: - .. code-block:: console + .. code-block:: mysql - cd ~/cyborg/cyborg - tox -e genconfig - .. + GRANT ALL PRIVILEGES ON cyborg.* TO 'cyborg'@'localhost' IDENTIFIED BY 'CYBORG_DBPASS'; + .. - And make a copy of it for further modifications + Replace ``CYBORG_DBPASS`` with a suitable password. - .. code-block:: console + * Exit the database access client. - cp -r ~/cyborg/cyborg/etc/cyborg /etc - cd /etc/cyborg - ln -s cyborg.conf.sample cyborg.conf - .. + .. code-block:: mysql + + exit; + .. + +#. Source the ``admin`` credentials to gain access to + admin-only CLI commands: + + .. code-block:: console + + $ . admin-openrc + .. + +#. To create the service credentials, complete these steps: + + * Create the ``cyborg`` user: + + .. code-block:: console + + $ openstack user create --domain default --password-prompt cyborg + .. + + * Add the ``admin`` role to the ``cyborg`` user: + + .. code-block:: console + + $ openstack role add --project service --user cyborg admin + .. + + * Create the cyborg service entities: + + .. code-block:: console + + $ openstack service create --name cyborg --description "Acceleration Service" accelerator + .. + +#. Create the Acceleration service API endpoints: + + * If cyborg-api service is deployed using uwsgi, create the following + endpoints: + + .. code-block:: console + + $ openstack endpoint create --region RegionOne \ + accelerator public http:///accelerator/v1 + $ openstack endpoint create --region RegionOne \ + accelerator internal http:///accelerator/v1 + $ openstack endpoint create --region RegionOne \ + accelerator admin http:///accelerator/v1 + .. + + * Otherwise, if cyborg-api service is running as a python process, create + the following endpoints: + + .. code-block:: console + + $ openstack endpoint create --region RegionOne \ + accelerator public http://:6666/v1 + $ openstack endpoint create --region RegionOne \ + accelerator internal http://:6666/v1 + $ openstack endpoint create --region RegionOne \ + accelerator admin http://:6666/v1 + .. + + .. note:: + + URLs (publicurl, internalurl and adminurl) may be different + depending on your environment. + + .. + +Configure Cyborg +----------------- #. Edit ``cyborg.conf`` with your favorite editor. Below is an example which contains basic settings you likely need to configure. @@ -101,19 +163,10 @@ Install and Configure .. -#. Install Cyborg packages. - - .. code-block:: console - - cd ~/cyborg/cyborg - sudo python setup.py install - .. - #. Create database tables for Cyborg. .. code-block:: console - cd /usr/local/bin cyborg-dbsync --config-file /etc/cyborg/cyborg.conf upgrade .. diff --git a/doc/source/install/common_prerequisites.rst b/doc/source/install/common_prerequisites.rst deleted file mode 100644 index ea5b6ece..00000000 --- a/doc/source/install/common_prerequisites.rst +++ /dev/null @@ -1,105 +0,0 @@ -============= -Prerequisites -============= - -Before you install and configure the Accelerator Life Cycle Management service, -you must create a database, service credentials, and API endpoints. - -#. To create the database, complete these steps: - - * Use the database access client to connect to the database - server as the ``root`` user: - - .. code-block:: console - - $ mysql -u root -p - .. - - * Create the ``cyborg`` database: - - .. code-block:: mysql - - CREATE DATABASE cyborg; - .. - - * Grant proper access to the ``cyborg`` database: - - .. code-block:: mysql - - GRANT ALL PRIVILEGES ON cyborg.* TO 'cyborg'@'localhost' IDENTIFIED BY 'CYBORG_DBPASS'; - .. - - Replace ``CYBORG_DBPASS`` with a suitable password. - - * Exit the database access client. - - .. code-block:: mysql - - exit; - .. - -#. Source the ``admin`` credentials to gain access to - admin-only CLI commands: - - .. code-block:: console - - $ . admin-openrc - .. - -#. To create the service credentials, complete these steps: - - * Create the ``cyborg`` user: - - .. code-block:: console - - $ openstack user create --domain default --password-prompt cyborg - .. - - * Add the ``admin`` role to the ``cyborg`` user: - - .. code-block:: console - - $ openstack role add --project service --user cyborg admin - .. - - * Create the cyborg service entities: - - .. code-block:: console - - $ openstack service create --name cyborg --description "Acceleration Service" accelerator - .. - -#. Create the Acceleration service API endpoints: - - * If cyborg-api service is deployed using uwsgi, create the following - endpoints: - - .. code-block:: console - - $ openstack endpoint create --region RegionOne \ - accelerator public http:///accelerator/v1 - $ openstack endpoint create --region RegionOne \ - accelerator internal http:///accelerator/v1 - $ openstack endpoint create --region RegionOne \ - accelerator admin http:///accelerator/v1 - .. - - * Otherwise, if cyborg-api service is running as a python process, create - the following endpoints: - - .. code-block:: console - - $ openstack endpoint create --region RegionOne \ - accelerator public http://:6666/v1 - $ openstack endpoint create --region RegionOne \ - accelerator internal http://:6666/v1 - $ openstack endpoint create --region RegionOne \ - accelerator admin http://:6666/v1 - .. - - .. note:: - - URLs (publicurl, internalurl and adminurl) may be different - depending on your environment. - - .. diff --git a/doc/source/install/installation.rst b/doc/source/install/install-from-pip.rst similarity index 64% rename from doc/source/install/installation.rst rename to doc/source/install/install-from-pip.rst index 8dfafed1..87f803d8 100644 --- a/doc/source/install/installation.rst +++ b/doc/source/install/install-from-pip.rst @@ -1,6 +1,6 @@ -============ -Installation -============ +===================== +Installation with pip +===================== At the command line:: @@ -10,3 +10,5 @@ Or, if you have virtualenvwrapper installed:: $ mkvirtualenv cyborg $ pip install openstack-cyborg + +.. include:: common.rst diff --git a/doc/source/install/install-from-source.rst b/doc/source/install/install-from-source.rst new file mode 100644 index 00000000..cc7a611f --- /dev/null +++ b/doc/source/install/install-from-source.rst @@ -0,0 +1,54 @@ +========================== +Install Cyborg from Source +========================== + +This section describes how to install and configure the Acceleration Service +for Ubuntu 18.04.1 LTS from source code. + +Install from git repository +---------------------------- + +#. Create a folder which will hold all Cyborg components. + + .. code-block:: console + + mkdir ~/cyborg + + .. + +#. Clone the cyborg git repository to the management server. + + .. code-block:: console + + cd ~/cyborg + git clone https://opendev.org/openstack/cyborg + .. + +#. Set up the cyborg config file + + First, generate a sample configuration file, using tox + + .. code-block:: console + + cd ~/cyborg/cyborg + tox -e genconfig + .. + + And make a copy of it for further modifications + + .. code-block:: console + + cp -r ~/cyborg/cyborg/etc/cyborg /etc + cd /etc/cyborg + ln -s cyborg.conf.sample cyborg.conf + .. + +#. Install Cyborg packages. + + .. code-block:: console + + cd ~/cyborg/cyborg + sudo python setup.py install + .. + +.. include:: common.rst diff --git a/doc/source/reference/support-matrix-fpga.ini b/doc/source/reference/support-matrix-fpga.ini new file mode 100644 index 00000000..584cbb8e --- /dev/null +++ b/doc/source/reference/support-matrix-fpga.ini @@ -0,0 +1,77 @@ +# Copyright (C) 2018 Lenovo, Inc. +# +# Licensed under the Apache License, Version 2.0 (the "License"); you may +# not use this file except in compliance with the License. You may obtain +# a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT +# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the +# License for the specific language governing permissions and limitations +# under the License. + +##################################################################### +# Drivers: + +[driver.fpga.fake] +title=Cyborg Fake Driver +link=https://opendev.org/openstack/cyborg/src/branch/master/cyborg/accelerator/drivers + +[driver.fpga.intel.opae] +title=Intel FPGA OPAE Driver +link=https://opendev.org/openstack/cyborg/src/branch/master/cyborg/accelerator/drivers/fpga/intel + +##################################################################### +# Functions: +[operation.supported] +title=Supported Vendor Driver +status=optional +notes=A vendor driver is considered supported if the vendor is + running a third party CI that regularly runs and reports + accurate results. +driver.fpga.fake=partial +driver.fpga.intel.opae=partial + +[operation.create_accelerator] +title=Create Accelerator +status=mandatory +notes=Supports the ability to create an accelerator. +driver.fpga.fake=complete +driver.fpga.intel.opae=complete + +[operation.delete_accelerator] +title=Delete Accelerator +status=mandatory +notes=Supports the ability to create an accelerator. +driver.fpga.fake=complete +driver.fpga.intel.opae=complete + +[operation.load_bitstream] +title=Load Bitstream +status=optional +notes=Supports the ability to load a bitstream to an FPGA. +driver.fpga.fake=missing +driver.fpga.intel.opae=complete + +[operation.delete_bitstream] +title=Delete Bitstream +status=optional +notes=Supports the ability to load a bitstream to an FPGA. +driver.fpga.fake=missing +driver.fpga.intel.opae=missing + +[operation.preloaded_bitstream] +title=Preloaded Bitstream +status=optional +notes=Supports the ability to use preloaded bitstreams already in FPGA. +driver.fpga.fake=missing +driver.fpga.intel.opae=complete + +[operation.update_shell] +title=Update FPGA shell +status=optional +notes=Supports the ability to update blue bits/shell for FPGA. +driver.fpga.fake=missing +driver.fpga.intel.opae=missing diff --git a/doc/source/reference/support-matrix-gpu.ini b/doc/source/reference/support-matrix-gpu.ini new file mode 100644 index 00000000..12aa1440 --- /dev/null +++ b/doc/source/reference/support-matrix-gpu.ini @@ -0,0 +1,56 @@ +# Copyright (C) 2018 Lenovo, Inc. +# +# Licensed under the Apache License, Version 2.0 (the "License"); you may +# not use this file except in compliance with the License. You may obtain +# a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT +# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the +# License for the specific language governing permissions and limitations +# under the License. + +##################################################################### +# Drivers: + +[driver.gpu.fake] +title=Cyborg Fake GPU Driver +link=https://opendev.org/openstack/cyborg/src/branch/master/cyborg/accelerator/drivers + +[driver.gpu.nvidia] +title=Cyborg NVIDIA GPU Driver +link=https://opendev.org/openstack/cyborg/src/branch/master/cyborg/accelerator/drivers/gpu/nvidia + +##################################################################### +# Functions: +[operation.supported] +title=Supported Vendor Driver +status=optional +notes=A vendor driver is considered supported if the vendor is + running a third party CI that regularly runs and reports + accurate results. +driver.gpu.fake=missing +driver.gpu.nvidia=complete + +[operation.create_accelerator] +title=Create Accelerator +status=mandatory +notes=Cyborg supports the ability to create an accelerator. +driver.gpu.fake=missing +driver.gpu.nvidia=complete + +[operation.delete_accelerator] +title=Delete Accelerator +status=mandatory +notes=Cyborg supports the ability to create an accelerator. +driver.gpu.fake=missing +driver.gpu.nvidia=complete + +[operation.update_firmware] +title=Update GPU Firmware +status=optional +notes=Supports the ability to update firmware for GPU. +driver.gpu.fake=missing +driver.gpu.nvidia=missing \ No newline at end of file diff --git a/doc/source/reference/support-matrix.rst b/doc/source/reference/support-matrix.rst new file mode 100644 index 00000000..2d154d2b --- /dev/null +++ b/doc/source/reference/support-matrix.rst @@ -0,0 +1,85 @@ +===================== +Cyborg Support Matrix +===================== + +Cyborg supports specific operations on VMs with attached accelerator +resources, which are generally a subset of the full set of VM operations +supported by Nova [nova-vm-ops]_. These are the operations that have been +validated to work in this release. + +This document lists the supported operations on VMs with accelerators in this +release, with caveats as needed, and also the list of VM operations known not +to work. Other VM operations outside both lists may or may not work; users try +them at their own risk. + +For the Train release, to enable the following VM operations are supported, +you must install certain Nova patches on top of the standard release +[nova-patches]_. + +.. list-table:: Supported VM Operations + :header-rows: 1 + + * - VM Operation + - Command + * - VM creation + - ``openstack server create`` + * - VM deletion + - ``openstack server delete`` + * - Reboot within VM + - ``ssh to VM and reboot in OS`` + * - Soft reboot + - ``openstack server reboot --soft`` + * - Pause/Unpause + - ``openstack server pause``, ``openstack server unpause`` + * - Lock/Unlock + - ``openstack server lock``, ``openstack server unlock`` + +.. list-table:: VM Operations Known to Fail + :header-rows: 1 + + * - VM Operation + - Command + * - Hard Reboot + - ``openstack server reboot --hard`` + * - Stop/Start + - ``openstack server stop``, ``openstack server start`` + * - Suspend/Resume + - ``openstack server suspend``, ``openstack server resume`` + + +The following support matrix reflects the drivers that are currently +available or are available in +`cyborg.accelerator.driver section of Cyborg's setup.cfg +`_ +at the time of release. + +FPGA Driver Support +~~~~~~~~~~~~~~~~~~~~ + +The following table of drivers lists support status for FPGA accelerators + +.. support_matrix:: support-matrix-fpga.ini + +GPU Driver Support +~~~~~~~~~~~~~~~~~~~ + +The following table of drivers lists support status for GPU accelerators + +.. support_matrix:: support-matrix-gpu.ini + +Driver Removal History +~~~~~~~~~~~~~~~~~~~~~~ + +The section will be used to track driver removal starting from the Train +release. + +* Train + +References +~~~~~~~~~~ + +.. [nova-vm-ops] `Server concepts + `_ + +.. [nova-patches] `Nova patches needed for VMs with accelerators + `_