This patch implements one of the breaking changes specified in
the API v1.1 spec, which requires clients to encapsulate the
list of messages to enqueue in a JSON object, when posting
those messages to the API.
Partially-Implements: blueprint api-v1.1-request-document-changes
Change-Id: I72ae87e1c479b6be6fbc8ce1fbaee89b2729e41a
When posting messages, the "ttl" field may be omitted, in which
case the server defaults to the default TTL value configured by
the operator.
Likewise, when creating a claim, the "ttl" and "grace" fields
may be omitted to use the defaults configured for each field.
Change-Id: I38c6307377293b26370de28ffa13db22228c208c
Partially-Implements: blueprint api-v1.1-request-document-changes
This patch removes the queue exists check from the transport layer.
Implements: blueprint api-v1.1-remove-check-queue-existence
Change-Id: I5fbd3089d552c22ab59a49a7fadc743cbb884770
This patch adds the ability to benchmark Marconi. The benchmark tool
is a console script, and can be triggered using
$ marconi-bench-pc
The Benchmark tool fires up both a Producer Process and a Consumer
Process, while accepting CLI parameters for the number of processes,
number of workers and duration of test.
The Producer Process publishes messages to a given queue, while the
Consumer consumes the messages by claiming and deleting them.
Setup:
Benchmark dependencies need to be pip installed:
pip install -r bench-requirements.txt
Export an environment variable called MESSAGES_PATH and set it to the
path of messages.json in marconi/bench
Note: This allows benchmarking with different set of messages rather
than those specified in messages.json
Usage:
$ marconi-bench-pc -p {No. Processes} -w {No. Workers} -t {No. Seconds}
Example:
$ marconi-bench-pc -p 2 -w 2 -t 4
Partially Implements: blueprint basic-benchmarking
Change-Id: I57ebe853554199490adba8b2a091423f399b0565
Adding the py-redis dependancy to marconi to support
redis storage backend
Adding the dependencies only to test-requirements-*.txt
as database backend requirements will slowly be phased
out of requirements-*.txt
Change-Id: I25581965b3deea201d33a7b53a284294c15290c7
Partially-Implements: bp/redis-storage-driver
This patch implements the pop functionality for v1.1 of the Marconi
API. The POP functionality is implemented as an optional parameter
in the delete messages using id params endpoint. This includes the
following validations,
1. pop & id params cannot be used together in the request.
2. 0 < pop value <= max_messages_per_claim_or_pop
docImpact
Change-Id: I63b22e57ed5ab398b81cde2d0284767e9704ec11
Implements: blueprint api-v1.1-pop-operation
Given that pylint is raising a lot of false positives
and that we have alternative static code checkers,
we decided to remove pylint from the tox envlist.
This patch also fix some pylint true positives
and cleans up the disabled pylint tests in the codebase.
Change-Id: I6fb4b9b4c80af452796032736e39ef064b9bcd03
Closes-Bug: #1297994
Enable storage unit tests for pooled context. This patch adds check for
pooling in test config in test initialization and creates appropriate
wrappers.
Also changes to sqlalchemy pooling controllers were made: in pooled
context project cannot be None due to constraint in database.
Change-Id: I59a663cef5b761e9470b8870d07cea03f5dac98b
Controllers wrappers for pooled context wasn't fully compatible with the
APIs defined in storage/base. Order of parameters was different and
default values was not specified or specified incorrectly. In
particular in pooled setup default limits were None in message and claim
controllers. This commit changes them to be the same as in non-pooled
setup.
Also tests for testing default limits were added.
Change-Id: I2b83383975986179813df7a897d78852a532564e
Closes-bug: #1324587
oslo.i18n has recently been released. This patch switches marconi over
oslo.i18n. As per oslo.i18n's instructions, a new i18n module has been
added under marconi. This module defines the translation globals and
imports the necessary functions from oslo.i18n.
The patch doesn't change the way Marconi does translation, this means
that a `_` function is still being injected to the builtins.
Note that the gettextutils module is still a required module from
oslo-inc because there are oslo-inc modules that depend on the old
gettextutils module.
Closes-bug: #1314300
Change-Id: Ifb8f3296d1a0e2483ebd1c8d868b7359ecc99fb5
Based on current implement of sqlalchemy driver, we're setting
primary key for pool, but unfortunately, it's incorrect, because
user will definately put many queues for one pool. This fix will
remove the primary key for pool and imporve the unit test to
cover it.
Closes-Bug: 1338907
Change-Id: I20b4176e96760356dfd23211d7170c074db8254b
Some tests weren't cleaning the queues they created. This caused other
tests to fail because of they ran on a no clean database. The patch adds
cleanup calls that will delete those queues once the test ends.
Change-Id: Iac354c34e14783a4b9773c60efbf719c21a9f9d2
Marker's test used a local queue_name instead of the one defined in the
base class. This caused some tests to fail because the ran on a no clean
database.
The patch makes the test use the queue_name and the project defined in
the class.
Change-Id: I5a1682e835a07a452eb1114da8c87369fe2f48b4
We're using limit=0 to query unlimited records. Unfornuately,
it doesn't work when a sql based database is used as pool
management database. In sqlalchemy, it means 0. This patch
will check if the limit is 0 to decide if applying the
limit() query for sqlalchemy.
Closes-Bug: 1337191
Change-Id: Iefc07ac41b17f52f7aa6607b0eb83dbc4cdef088
Now all of the calling for errors.QueueNotMapped are using
the wrong parapemters order, this patch will fix it and
improve the unit test to cover that.
Closes-Bug: 1336141
Change-Id: Id6cc52862465535ef5c1c06649a0ac8bf3485386
This patch adds the base API for queue flavors to the storage layer. The
API follows pretty much the leads of other APIs and supports basic
operations that allow users to create, delete, update and retrieve queue
flavors.
Partially-Implements blueprint: marconi-queue-flavors
Change-Id: I0666331ec983b138e324d42bf832d20e0dbfe877
Added tests for new V1.1 functions like pop and shard
Updated existing tests for V1.1 return calls
Added new json schemas
Edited functional test base to support multiple schemas
Change-Id: I35dfb92af22540609ecb50c1eb5c8fa65e6cadc8
Partially-Implements blueprint: api-v1.1-functional-tests
PoolsControllerTest.test_listing_simple assumes the alphabetical
sorting order which is changed to lexicographical sorting order.
Change-Id: I4e484eb3ecbf785a4f4b3a6cbae6e05d4703b254
Closes-Bug: #1254158
This patch updates a couple caching-related comments that were
inaccurate or ambiguous.
See also: https://review.openstack.org/#/c/67978
Change-Id: I4772568ff91a7a72f2633a0b4fe45d4f11a2c5da
