Commit Graph

12 Commits

Author SHA1 Message Date
Monty Taylor
af7c1bc13d Update pbr and requirements to be current
The usage of pbr and the rest of the requirements need to be update to
be up to date with modern usage.

Change-Id: I7dd4c6d6f6721205a293d427851733a0391ea72e
2013-09-18 07:55:39 -07:00
Flaper Fesp
802a37c568 Migrate to pbr
Change-Id: I0d5888f9d58d819ac65561b222cb2e93f260d627
2013-06-14 14:14:42 +02:00
kgriffs
bfd29252f5 Guarantee FIFO/once-and-only-once delivery when using MongoDB
This patch modifies the way message markers are generated and
used, such that Marconi can guarantee FIFO for a single message
producer posting to a single queue (FIFO for multiple producers
is not guaranteed.) At the same time, these changes guarantee
that observer clients will recieve messages once-and-only once
by removing race conditions inherent in timestamp-based markers.

Along the way, some technical debt was also addressed, particulary
regarding style, as well as some minor optimizations.

A new service, marconi-mongo-gc, was also added, and is required
when using the MongoDB storage driver for Marconi. This service
was necessary due to the special requirement that at least the
most recent message always remain in each queue, which is an
a-priori assumption upon which the FIFO/pagination algorithm
is based, and arises from the particular constraints imposed
by MongoDB's semantics.

Note: While implementing this blueprint, many calculated tradeoffs
were made in an attempt to balance performance, risk, readability,
and maintainability. The goal was to create a resonable baseline
implementation that can be iterated upon pending comprehensive
system and performance testing. Due to the many subtleties
of solving the FIFO/once-and-only-once problem for the MongoDB
driver, future contributors should excercise extreme caution when
modifying the algorithm introduced in this patch.

Changes include:
* Align text in comments
* Add counter to queue, messages
* Markers are now converted over to using monotonic counter
* Handle DuplicateKeyError
* Return resources in body as a response to a message POST
* Added mongo driver claims tests
* Return 503 when no messages were enqueued due to marker conflict
* Added backoff sleep between retries
* Added marconi-mongo-gc service. This is a new required servi

Implements: blueprint message-pagination
Change-Id: Ifa0bb9e1bc393545adc4c804d14c6eb2df01848c
2013-06-13 13:16:44 -04:00
Flaper Fesp
b9c4d1f12a Rename bin package into cmd
Most projects are migrating to entry_points now and the name of the
package is cmd. I renamed bin into cmd to keep consistency with other
projects.

Change-Id: Iab889123fb4836a42ed0ef52986ffa7648a140fa
2013-05-13 15:50:27 +02:00
Flaper Fesp
7563c63c90 Implemented marconi-server entry point
This patch adds the first console_script for marconi server.

In order to use it it's enough to run setup.py either using develop or
install.

Side changes:
    The config now uses sys.argv[1:] if no custom cli args have been
    set.

Implements blueprint transport-base

Change-Id: I15732129d66b32fca2b818e9105b17f541094983
2013-04-05 16:12:17 +02:00
Bryan Davidson
54f21769d7 Suppress some Coverage report output
We are not responsible for marconi.openstack.common code so it
will be excluded from our Coverage report. Also excluding
NotImplementedErrors as there's obviously no implementation in
that case to be tested.

Change-Id: Ie1e2cf4b858afef56ffed7ee225a0f8c29dcaef8
2013-03-29 16:07:20 -04:00
Bryan Davidson
a6e7a51712 Bootstrap now dynamically imports storage and transport modules
When the Bootstrap is instantiated, it is passed a configuration
file. The [drivers] section of the configuration file is read
and the specified storage and transport modules are imported and
instantiated.

Implements: blueprint bootstrap-dynamic

Change-Id: I47cd6f529afdbc93fca53f3298b4bfdbf717184c
2013-03-21 09:20:52 -04:00
Zhihao Yuan
38d72dffd0 Decentralized configuration.
The configuration is based on oslo-config, while this module lets
each module which want to be configured define the options it need
by itself, and get the limited access to those config variables.

Implements blueprint config-module

Change-Id: Iffbb859da0cb5fb8434bc439320bf6c20f9d39b5
2013-03-13 14:22:37 -04:00
Flaper Fesp
420c1e046c Added hacking file
As part of adding the hacking file, this patch adds some files taken
from nova project that will help with the coding standards validation.

Tox now uses run_pep8 for the pep8 test

Change-Id: I0784390a0b13c9694e358563d43e5442592a785e
2013-03-07 18:07:53 +01:00
Flaper Fesp
567abaea5c Added license to setup.py
Change-Id: I43d66f7d35772ae8857fbc0bfdc475299884200e
2013-02-25 16:20:40 +01:00
Flaper Fesp
693582e1c4 Fixed setup, tox and requirements
It is now up-to-date with the current test process, versions and
configurations.

Change-Id: I927e99b390a9e2224dfcb6bcde33cbe9d012c53f
2013-02-25 15:47:39 +01:00
Monty Taylor
e0600c2d02 Initial import.
Change-Id: I00a644dabdd01eb1e09a42a8d326cacdaca76f9f
2012-11-01 09:52:20 +01:00