When run:
$ python -m unittest cyborg.tests.unit.accelerator.drivers.modules.test_generic
It will report the follow error:
File "cyborg/tests/unit/accelerator/drivers/modules/test_generic.py", line 20, in <module>
from cyborg.accelerator.drivers.generic_driver import GenericDriver as generic
File "cyborg/accelerator/drivers/generic_driver.py", line 24, in <module>
from cyborg.accelerator import accelerator
File "cyborg/accelerator/accelerator.py", line 23, in <module>
class accelerator(Base):
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/ext/declarative/api.py", line 64, in __init__
_as_declarative(cls, classname, cls.__dict__)
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/ext/declarative/base.py", line 88, in _as_declarative
_MapperConfig.setup_mapping(cls, classname, dict_)
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/ext/declarative/base.py", line 103, in setup_mapping
cfg_cls(cls_, classname, dict_)
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/ext/declarative/base.py", line 125, in __init__
clsregistry.add_class(self.classname, self.cls)
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/ext/declarative/clsregistry.py", line 65, in add_class
module.add_class(classname, cls)
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/ext/declarative/clsregistry.py", line 172, in add_class
existing.add_item(cls)
AttributeError: '_ModuleMarker' object has no attribute 'add_item'
That's because:
_ModuleMarker will add the module name and class name in to same contents dict.
_MultipleClassMarker refers to multiple classes of the same name within _decl_class_registry.
It has add_item attribute.
_ModuleMarker refers to a module name within _decl_class_registry.
It does not have add_item attribute.
So the class name should be different with module name.
REF:
https://github.com/zzzeek/sqlalchemy/blob/master/lib/sqlalchemy/ext/declarative/clsregistry.py
Change-Id: I8d8aa8809f6afd5b9eaace832a06ba7f3770e652
More patches might follow
Update 0(pacth 1):
1.remove unnecessary files
2.reorg doc directory
Update 1(patch 2):
add releasenotes and cleanup the internal-api spec
Update 2(patch 3):
1.added a new releasenote for generic-driver
2.fixed errors in the other four earlier releasenotes
3.add comment in cyborg/object/accelerator.py and deployable.py to
let user better know the meanings of the table entries.
Change-Id: I45a104f9f77a0a163008657c681cf109f941dd11
Signed-off-by: zhipengh <huangzhipeng@huawei.com>
SPDKDRIVER is a virtual interface which provides common methods
for specific drivers (such as: VHOSTDRIVER, NVMFDRIVER, etc.).
For this reason, the Cyborg agent should invoke these drivers
via py-spdk[0] to communicate with the backend SPDK-base app server.
The py-spdk is management lib for SPDK applications which need to
be imported into the cyborg, so we put the pyspdk lib into cyborg/
cyborg/accelerator/drivers/spdk/util/. There are some unit tests
we added in cyborg/cyborg/tests/unit/accelerator/drivers/spdk/.
Now We are first implementing the unit tests of
test_discover_accelerator() and test_accelerator_list(),
and the rest will be added later.
For example:
When the Cyborg agent call the NVMFDRIVER.discover_accelerator(),
the return value we get is:
{
'server': 'nvmf_tgt',
'bdevs': [{
"num_blocks": 131072,
"name": "nvme1",
"block_size": 512,
......
}]
'subsystems': [{
"core": 0,
"nqn": "nqn.2018-01.org.nvmexpress.discovery",
"hosts": [],
......
}]
}
[0] The implementation of py-spdk is subbmitted to
https://review.gerrithub.io/#/c/379741/, please visit it.
Change-Id: I2d0e4dc6b58e725584d22ee85961877a870c68a7
This patch implements the Deployables data model and provides corresponding
CRUD REST/RPC API.
Since the CREATE/UPDATE/DELETE method of Deployables should only be used
by internal component, currently keep these in code is just for convenient
test purpose, which will be removed later.
blueprint cyborg-fpga-modelling
Change-Id: I8a028954bd27d015c0b062f730673ea39701774d
This patch is not a plugin implementation
Cyborg agent can call driver as follow:
from cyborg.accelerator.drivers.fpga import base
fdriver = base.FPGADriver
# There maybe more than one vendor fpga card on a host.
venders = fdriver.discover_vendors()
for v in venders:
dr = fdriver.create(v)
dr.discover()
intel = base.FPGADriver.create("intel")
intel.program("/fpga/device/path", "/local/path/image")
When call IntelFPGADriver.discover(), we can get the result as follow:
[{'assignable': False,
'devices': '0000:5e:00.0',
'function': 'pf',
'parent_devices': '',
'path': '/sys/class/fpga/intel-fpga-dev.0',
'pr_num': '1',
'product_id': '0xbcc0',
'regions': [{'assignable': True,
'devices': '0000:5e:00.1',
'function': 'vf',
'parent_devices': '0000:5e:00.0',
'path': '/sys/class/fpga/intel-fpga-dev.2',
'product_id': '0xbcc1',
'vendor_id': '0x8086'}],
'vendor_id': '0x8086'},
{'assignable': True,
'devices': '0000:be:00.0',
'function': 'pf',
'parent_devices': '',
'path': '/sys/class/fpga/intel-fpga-dev.1',
'pr_num': '0',
'product_id': '0xbcc0',
'vendor_id': '0x8086'}]
Cyborg agent can use it to form it's DB.
Co-Authored-By: Dolpher Du <Dolpher.Du@intel.com>
Change-Id: I132be5ecdb90b385b68b0cdf306d1402ef4bfc04
use PeriodicTasks to track resources.
Info:
https://docs.openstack.org/oslo.service/latest/reference/periodic_task.html
Aggent is a bridge for FPGA driver and conductor.
It gets the lastest informations and update them to conductor.
This patch we call conductor API directly instead of object
remotable method. Will support it in the next version.
This patch depends on the FPGA driver patch and conductor patch.
How to test this patch:
1. apply the fpga driver patch
https://review.openstack.org/#/c/531129
2. generate a sysfs fpga data
$ ./cyborg/tests/unit/accelerator/drivers/fpga/intel/prepare_test_data.py
3. Change the SYS_FPGA in cyborg/accelerator/drivers/fpga/intel/sysinfo.py to
"/tmp/sys/class/fpga"
4. change the SYS_FPGA_PATH in cyborg/accelerator/drivers/fpga/utils.py to
"/tmp/sys/class/fpga"
5. restart agent.
The unittest for this patch will be in a separated patch, and comes out ASAP.
Co-Authored-By: Dolpher Du <Dolpher.Du@intel.com>
Change-Id: I5e487cf939aa65d0fc79399ddd5d1337a8c2fa98
Zuul no longer requires the project-name for in-repo configuration.
Omitting it makes forking or renaming projects easier.
Change-Id: I3066f4fdf016d26f43a34ecd60c64ec6423f31fd
This feature is used for updating the placement DB on NOVA side
Cyborg DB should be kept up to date with the placement DB all
the time.
Here is an example on how to use it:
from cyborg.services import report as placement_report_client
p_client = placement_report_client.SchedulerReportClient()
resource_provider = {'name': 'rp_name', 'uuid': 'uuid'}
p_client.create_resource_provider(resource_provider)
Change-Id: I04f5e3fc283074212841e62dde54df496db535ec
This is my attempt at whipping up some internal API calls
for Crushil, there's some assumptions in here we should talk
about. Hopefully this will also be a good place to keep track
of the internal API in general.
Change-Id: I59f3706190d0abed2997a8302717308ea5974ac5
The related bug was spotted when I tried to setup cyborg environment by devstack but failed to start cyborg.
It is result from the permission change in devstack/lib/cyborg.
Closes-bug: #1742643
Change-Id: I54426aea597119253f07acf243d6616cd27935ac
with this patch, we are able to do CRUD via accelerator api
POST /accelerators
Create a new accelerator
GET /accelerators/{uuid}
Retrieve information about the given accelerator
GET /accelerators
Retrieve a list of accelerators
PATCH /accelerators/{uuid}
Update information about the given accelerator
DELETE /accelerators/{uuid}
Delete the given accelerator
Change-Id: Ibc97ec49f557cac69dfd38f99f28bc7b205732b9
This patch implements first step in zuulv3 migration guide [1]:
Move jobs to Cyborg repo
[1] https://docs.openstack.org/infra/manual/zuulv3.html#legacy-job-migration-details
Change-Id: I31d5d7aed511aca18c1096b9f5eb7572f683d613
Needed-By: If816cfec5ace8f9fd7293c0749948d2c0dcb817a
Needed-By: I40e9380d049f541bf34488a3010d95f4a6835179
According to [1], we can passing a "-t" argument to
oslo_debug_helper to indicate the directory where tests
are located. This will solves ImportError exception.
[1] https://docs.openstack.org/developer/oslotest/features.html
Change-Id: I13b7a11bc1a3e76709352a248d9eaa64aeba8bc3
Should remove oslo.serialization which only use at one test-case.
This change will help project do not depends this package,
and let's use the regular json attribute anyway.
Change-Id: I01af1e1b09fb9866acf54fc7a19aedfaab34a684
Dramatically revised from this
https://review.openstack.org/#/c/448228/3/specs/proposal/cyborg-nova-interaction.rst
My goal with this spec is to have it be only where we interact with nova without
details of other components, otherwise we just get into a monolithic spec for
everything.
I plan to expand this into exact api calls and a detailed workflow, especially
for the new API call we will have to make in nova to register whitelisted
devices live. That being said we may need to reboot the machine to change the
grub config anyways so maybe we should be looking at how to make that work
first.
Change-Id: I22037109b613d7b33d7c620b78493ec7e96e735e
python-subunit is not used directly anywhere
and it is dependency of both testrepository
and os-testr.
Change-Id: I2136b810c2c304064dfd5dbe178a4c53b8860020
In a general, Cyborg, as the hardware (like FPGA) and software
(like DPDK, SPDK) acceleration framework, can help backend to
improve performance. And SPDK driver can help Cyborg to manage
SPDK accelerator through calling the py-spdk[0].
Based upon the above reasoning, this patch accomplish two things:
(1) When Cinder uses Ceph as its backend, the Cyborg SPDK driver
can discover, list the SPDK accelerator backend.
(2) When Cinder directly uses SPDK blobstore as its backend, the
Cyborg SPDK driver can discover, list, attach, detach and update
the SPDK accelerator backend.
[0] https://review.gerrithub.io/#/c/379741/
Change-Id: I76a40327bf5c267888fd6293f0ed918bc8cca7ae
Signed-off-by: heluwei <heluwei@huawei.com>
