167 Commits

Author SHA1 Message Date
Yumeng Bao
33b86b8631 Follow the new PTI for document build
In order to follow the new PTI for doc build, this patch:
1)removes unused build_sphinx in setup.cfg
2)move doc dependencies from test-requirements.txt to doc/requirements.txt
3)update tox.ini

For compliance with the Project Testing Interface as described in:
https://governance.openstack.org/tc/reference/project-testing-interface.html

For more details information, please refer to:
http://lists.openstack.org/pipermail/openstack-dev/2017-December/125710.html

Change-Id: Ic700f04b857516ce5b25ce5e0fc431d1f08e24f3
2018-08-11 12:55:49 +08:00
Zuul
b727c1670a Merge "support sub provider" 1.0.0.0rc1 2018-08-10 14:43:11 +00:00
Shaohe Feng
604e5b54fe support sub provider
With this patch, cyborg can report accelerators information into
placement. Then scheduer can find an appropriate host according to
user requirements.

This patch leverages Nova's code in order to avoid reinventing the wheel.

It is batter that nova can split the placement client to a separated lib.
So other projects can use it easily.

This new client can support provider trees, that is needed by cyborg's
different dirvers.
We can create a sub provider tree easily by the follow method.
class SchedulerReportClient(object):
    def get_provider_tree_and_ensure_root(
           self, context, rp_uuid, name=None,
           parent_provider_uuid=None)

The patch include:
1. add get_ksa_adapter for placement client

2. add placement client

3. update placement config

Missing testcase for this patch.

Change-Id: I1ad9d525fa070dfa0f7cbf374003a74c50de17b4
2018-08-10 07:49:37 +00:00
wangxu
e82036d50c Fix deployable get all sort issue and unit test failure.
Change-Id: I6a34c4038a6e0cb15cb77be95003d6b78b29f476
2018-08-09 10:48:42 +08:00
Zuul
29c4d6c17b Merge "Docs: Autogenerate config documentation" 2018-08-08 10:17:51 +00:00
Yumeng Bao
3cb5c38edb Docs: Autogenerate config documentation
Ensure the documentation is kept up to date by automatically
generating it from the source when the docs are built.

This patch:
1.replace oslosphinx with openstackdocstheme since oslosphinx is obsolete
2.add oslo_config.sphinxconfiggen to generate sample conf file
3.add oslo_policy.sphinxpolicygen to generate sample policy file

Change-Id: I39f03023f0a85e4c8943925268ab3e9066975630
2018-08-08 01:26:38 +00:00
wangzh21
458e0b1162 Imply new cyborg deployable api
1. Imply deployable api as our discussion on 19th, Jul.
   https://etherpad.openstack.org/p/cyborg-rocky-development
2. Add filter to deployable.
3. Improve test_accelerator.
4. Add command "rm" to tox whitelist_externals.

Change-Id: I78154ef14a0ab3831a17ba93f44dfedef0d56df1
2018-08-08 09:22:53 +08:00
Zuul
c8ba137be6 Merge "Added rest API for FPGA programming" 2018-08-07 23:41:03 +00:00
Zuul
ca7accd7eb Merge "Switch to stestr" 2018-08-07 02:49:43 +00:00
Li Liu
185169e380 Added rest API for FPGA programming
Here is an example on how to call this api
curl -s -X PATCH -H "X-Auth-Token: $OS_TOKEN" -H "Content-Type: application/json"\
  -d '[{
    	"path": "/program",
    	"op": "replace",
    	"value": [{
    		"image_uuid": "9a17439a-85d0-4c53-a3d3-0f68a2eac896"
    	}]
  }]'\
  http://{api_ip}:6666/v1/accelerators/deployables/{pf_uuid}/program

Change-Id: Iab6150f39be9ccb34f1fc86d6942b1b7c48a4348
2018-08-06 00:47:37 -04:00
Vu Cong Tuan
e76f2a5f90 Switch to stestr
According to Openstack summit session [1],
stestr is maintained project to which all Openstack projects should migrate.
Let's switch to stestr as other projects have already moved to it.

[1] https://etherpad.openstack.org/p/YVR-python-pti

Change-Id: I8b0f96f05799b892acfae03522726c17010f0df3
2018-07-24 15:10:52 +07:00
Coco
06970bcd0b Add "interface_type" field in deployable DB
The added "interface_type" field represents deployables' interface
type when attaching to the hypervisor, e.g pci,mdev etc.
When "interface_type" equals "pci", the "address" field represents
the pci_address, and  when "interface_type" equals "mdev", the
"address" field represents mdev_path.

Change-Id: I9acbecab2aa8741e9aa7f19f7543a5c05defd90b
2018-07-24 03:57:13 -04:00
Zuul
88b95ae654 Merge "Improve .gitignore for project" 2018-07-13 11:44:37 +00:00
Nguyen Van Trung
e4179de2a8 Improve .gitignore for project
This patch will help project ignore some redundant items
during push into gerrit.

Change-Id: I4a9bf4c5b8be64e3327dbf0592f9d858bea6becc
2018-07-06 07:06:50 +00:00
Doug Hellmann
2fb4367614 remove AUTHORS file from git
The AUTHORS file is generated automatically by pbr as part of building a
package, so it should not be checked in to git.

Change-Id: I1c839b79c6ff97f3515c1f61342c5054058727b9
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
2018-07-03 09:46:42 -04:00
Eric Fried
d116a63414 Move specs under doc/source so they build
Specs were previously under doc/specs and where thus not being rendered
via the docs build process [1].  This patch moves the directory to
doc/source/specs so that'll happen now, and makes minor tweaks to the
documents themselves to clear build errors.

More work should be done at some point soon to split out the appropriate
parts of test-requirements.txt into a doc/requirements.txt.

[1] https://governance.openstack.org/tc/reference/project-testing-interface.html#documentation

Change-Id: I1e2eff09114ef2f2fca883adb454536fb39ab64d
2018-06-27 18:08:31 -05:00
Doug Hellmann
0c7b6d725c fix tox python3 overrides
We want to default to running all tox environments under python 3, so
set the basepython value in each environment.

We do not want to specify a minor version number, because we do not
want to have to update the file every time we upgrade python.

We do not want to set the override once in testenv, because that
breaks the more specific versions used in default environments like
py35 and py36.

Change-Id: If27b1960674af44a4de7af412550321d74f05af6
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
2018-06-13 14:29:02 -04:00
zhipengh
c3f97d4b48 Bug fix in setup.cfg for rocky ms2
As Sean and Clark suggested, etc/cyborg/rootwrap.d = etc/cyborg/rootwrap.d/*
has been moved into one line.

Change-Id: I7d51aa158b4918a89fc41b7e931b9ef0465d87ca
Signed-off-by: zhipengh <huangzhipeng@huawei.com>
2018-06-13 08:55:03 +08:00
Xinran WANG
d5088b7474 Introduce Cyborg Resource Quota -- Usage Part
The concept of quota gives the possibility to limit the maximum number of
acceleration resources that one project/tenant can prossess.
It will also  give the admin users the ability to increase the resource count
on-demand, given that the system constraints are not exceeded.

Change-Id: I554b9d4603d5e65f69c2b924fba66565f7f6c3c4
Partially-Implements: blueprint cyborg-resource-quota
2018-06-11 15:09:09 +08:00
Zuul
1f3933d1d4 Merge "use config options from keystoneauth1 under placement" 2018-06-11 07:01:27 +00:00
Zuul
443cbf9f87 Merge "Fix tox -egenconfig" 2018-06-11 06:55:00 +00:00
Zuul
4ec459afb2 Merge "Specification for Cyborg/Nova interaction for scheduling." 2018-06-11 03:45:04 +00:00
wangzh21
ae87683acd Fix tox -egenconfig
1.Imply the namespace cyborg
2.Remoeve unused namespace oslo.reports

Change-Id: I61de79aa3b8a97c4a4dd6d92ef7bce951f6efd55
Story: #2002200
Task: #21685
2018-06-10 16:37:40 +08:00
Zuul
524703c347 Merge "Added bitstream metadata standardization spec" 2018-06-10 06:34:49 +00:00
Zuul
8bbfad7307 Merge "Added cyborg fpga programming spec" 2018-06-10 02:31:38 +00:00
Li Liu
fada056b05 Added bitstream metadata standardization spec
Change-Id: I14cfd85e946be1a8a653f14b3ae3b8421c01544c
2018-06-09 21:01:40 -05:00
Sundar Nadathur
ebb94796bd Specification for Cyborg/Nova interaction for scheduling.
Addresses issues of representing accelerators in Cyborg for
alignment with Nova, and integrating Cyborg into Nova
scheduler flow.

Change-Id: Ied10c0c9515ac498d374ffa36cf41974adf2381f
Proposes: spec cyborg-nova-scheduling
Signed-off-by: Sundar Nadathur <ns1.sundar@gmail.com>
2018-06-09 03:56:40 -07:00
Li Liu
87790c3f54 Added cyborg fpga programming spec
Change-Id: I261bee93a966d73cf489c85006d316179bc0ca45
2018-06-08 05:44:07 -05:00
Sumit Jamgade
6498dd8a0f use config options from keystoneauth1 under placement
this avoid manual syncup of options from keystoneauth plus gives us the
ability to more options as needed by cyborg

Change-Id: I3ed37cf68be985b7ca06f02244534aebc0137b68
Story: 2002169
Task: 20026
2018-06-08 10:05:46 +02:00
Zuul
10c394406f Merge "Load cyborg-api app with paste_deploy" 2018-06-08 04:09:14 +00:00
Zuul
0157373db7 Merge "Specification for Cyborg Agent-Driver API." 2018-06-08 03:06:45 +00:00
Zuul
c962039a11 Merge "Specification for Cyborg/Nova/Glance interaction in the compute node, including os-acc library API." 2018-06-08 02:55:25 +00:00
Sundar Nadathur
cda3ca7b35 Specification for Cyborg Agent-Driver API.
Enhanced the device description model (which is structured
as a Python dictionary) to align it with RCs and traits
that are published by Cyborg in Nova.

Change-Id: I27a4bcab6379652e6893aee7fcc684036db93d61
Proposes: spec cyborg-agent-driver-api
2018-06-07 09:43:49 -07:00
wangzh21
cc9da9a944 Load cyborg-api app with paste_deploy
Change-Id: I3976d05f737e09ceb88c40f65b445bad0b096516
Story: 2002121
Task: 19800
2018-06-07 21:42:52 +08:00
Zuul
07a273e70d Merge "Fix Deployable get_by_host" 2018-06-07 13:07:09 +00:00
wangzh21
f67e3daaf6 Add note to dev_setup doc
1.Add note(PIP_UPGRADE) for local.conf.
2.Move bash to code-block: console. If not, the quotation marks
will display as Chinese character.

Change-Id: I48f899de345b8dc85b992d18b10922db5fa107e1
2018-06-05 18:29:44 +08:00
wangzh21
0ac1227f69 Fix Deployable get_by_host
1.Fix get_by_host.
2.Add missing configs of placement.

Change-Id: I4452371d02fface1adbc322cb75cc39e32062303
2018-06-04 23:50:22 +08:00
Zuul
2f6f71849e Merge "Added attribute object and its unit tests" 2018-05-31 02:32:25 +00:00
Zuul
d351a7c75b Merge "doc fix: devstack setup doc can not display well" 2018-05-30 09:24:11 +00:00
Shaohe Feng
f2bbb73898 doc fix: devstack setup doc can not display well
Let's use utf-8 encode.
Change md syntax to rst.

Change-Id: I417781692e198e4f9b073fcba44b5af3fa22b4ff
2018-05-29 16:27:15 +00:00
Sundar Nadathur
030d8f5151 Specification for Cyborg/Nova/Glance interaction in the
compute node, including os-acc library API.

Updated to note that a flavor may refer to a function
by name, for operators' ease of use.

Change-Id: Iefdfbdeadd5894f8d6e45368433ae951105f5a7e
Proposes: spec cyborg compute node API
2018-05-19 16:22:22 -07:00
Shaohe Feng
2e10d20110 change service type to "accelerator"
As we discussed in irc meeting, we use "accelerator" as service type.

Change-Id: Ib932f9fe27bf7fc08c3f334acc6b3da6cfd0489f
2018-05-16 09:25:12 +00:00
Nguyen Van Trung
38c1618e60 Add doc8 to pep8 check for cyborg project
This patch adds a doc8 check of .rst files to the current pep8 check.
It includes fixes to the .rst files that didn't pass the check.

Change-Id: Ib7a63d1e579f77039172aa4f99d26a3ceeef83d7
2018-05-10 09:32:01 +07:00
Li Liu
c8ad433a98 Added attribute object and its unit tests
Attributes are associated with deployables, each deployable keeps
a attributes_list. Everytime user loads a deployable from DB, all
the associated attibutes will also be retrieved and put into this
list.

To add attributes to a deployable:

Deployable.add_attribute(Attribute)
Deployable.save(context)

To delete an attribute:

Deployable.delete_attribute(context, Attribute)

In addition, Deployable.get_by_filter() also works with attributes
filters as well. For instance:

query = {"attr_key": "attr_val"}
deployable_list = objects.Deployable.get_by_filter(context, query)

Change-Id: Ic2a1874a995ce73124dc33b1c2dbdafa412ecae2
2018-05-08 23:40:17 -05:00
Zuul
3670bbb83a Merge "uncap eventlet" 2018-05-08 06:22:29 +00:00
Doug Hellmann
305c320dcb uncap eventlet
We will manage the eventlet version using constraints now. See the
thread starting at
http://lists.openstack.org/pipermail/openstack-dev/2018-April/129096.html
for more details.

Change-Id: I3c4282f0bc2ddedf816f366cd058823777ecdab9
Co-Authored-By: Michel Peterson <michel@redhat.com>
2018-05-07 17:42:19 +03:00
Zuul
0d2f38c435 Merge "bug fix: endpoint register, import and devstack broken issues." 2018-04-29 03:24:31 +00:00
wangzh21
801aad10a4 Fix remote call conductor error
Change-Id: I130bad097ffa2ef6594c5979f536b76a9350939f
Story: 2001928
Task: 14457
2018-04-28 15:36:39 +08:00
Shaohe Feng
a308525d38 bug fix: endpoint register, import and devstack broken issues.
1. amend keyston endpoint register info
Service type should acceleration instead of cyborg.
For cyborg is just the service name.
We do not support uwsgi at present, so we must indicate service
port explicitly.

2. Fix package import error
Add full path to import module, so that the reference can be
resolved.

Story: 2001887
Task: 14368

3. devstack broken
TypeError: launch_service() got an unexpected keyword argument 'restart_method'
When initialize cyborg api service launcher, we already pass restart_method.
So no need to pass restart_method to launcher.launch_service, and it has not
restart_method argument.

Co-Authored-By: jiapei <jiapei2@lenovo.com>
Co-Authored-By: Zhipeng Huang <huangzhipeng@huawei.com>

Change-Id: Ic69e76b7bf86bbd1382aff07694ea9fbafd0862a
2018-04-28 07:11:33 +00:00
Zuul
ce0b6e5a95 Merge "Implemented the Objects and APIs for vf/pf" 2018-04-19 18:00:34 +00:00