RETIRED, Monasca Analytics Engine
Go to file
Hisashi Osanai 30c75d34ba Add devstack plugin
This patch provides devstack plugin.

Co-Authored-By: Daisuke Fujita <fuzita.daisuke@jp.fujitsu.com>
Change-Id: Id0de403b05f174176e92cc7d6287d38fcd043d02
2016-06-13 16:16:12 +00:00
config Monanas initial code release. 2016-05-17 01:48:24 +01:00
devstack Add devstack plugin 2016-06-13 16:16:12 +00:00
doc Monanas initial code release. 2016-05-17 01:48:24 +01:00
monasca_analytics H238 support 2016-06-06 09:36:13 +00:00
test H238 support 2016-06-06 09:36:13 +00:00
__init__.py Added long Copyright and licence note at the top of each source file 2016-05-24 20:19:20 +01:00
.gitignore Monanas initial code release. 2016-05-17 01:48:24 +01:00
.gitreview Added .gitreview 2016-04-22 11:40:50 +00:00
config_dsl.py Rename main package to monasca_analytics. 2016-06-02 13:03:56 +01:00
fetch-deps.sh Monanas initial code release. 2016-05-17 01:48:24 +01:00
jenkins.sh Monanas initial code release. 2016-05-17 01:48:24 +01:00
LICENSE Add license 2016-05-05 07:41:10 -06:00
Makefile Monanas initial code release. 2016-05-17 01:48:24 +01:00
README.md Fixed links and removed contributors list from README.md 2016-05-26 01:32:13 +01:00
requirements.txt tox support 2016-06-03 17:46:33 +00:00
run.py Rename main package to monasca_analytics. 2016-06-02 13:03:56 +01:00
setup_property.py Added long Copyright and licence note at the top of each source file 2016-05-24 20:19:20 +01:00
setup.cfg tox support 2016-06-03 17:46:33 +00:00
setup.py tox support 2016-06-03 17:46:33 +00:00
SUMMARY.md Monanas initial code release. 2016-05-17 01:48:24 +01:00
test-requirements.txt tox support 2016-06-03 17:46:33 +00:00
tox.ini Add devstack plugin 2016-06-13 16:16:12 +00:00
Vagrantfile Monanas initial code release. 2016-05-17 01:48:24 +01:00

MoNanas - Monasca Analytics Framework

MoNanas Logo

Overview

Monasca Analytics (MoNanas) is a statistical/machine-learning (SML) flow composition engine. Users can compose a sequence of algorithms to be executed by just providing a description as an input to MoNanas. The data flow is automatically handled by the framework.

Easy flow composition and reusability means that we can speed up the extraction of actionable infrastructure insight.

Advantages

👍 Decouple algorithm design from execution.

👍 Reusable specification of the desired flow.

👍 Language independent flow definition.

👍 Data source and format independent.

👍 Easy to add new SML algorithms and # combine them with pre-existing ones in the flow.

👍 Transparently exploit data parallelism.

Example Use Cases

Below are few use cases that are relevant to OpenStack. However, MoNanas enables you to add your own data ingestors.

Example Alert Fatigue Management Anomaly Detection
Dataset Synthetic, but representative, set of Monasca alerts that are processed in a stream manner. This alert set represents alerts that are seen in a data center consisting of several racks, enclosures and nodes. iptables rules together with the number of times they are fired in a time period.
Parsing Monasca alert parser. Simple parser extracting period and number of fire events per rule.
SML algorithm flow filter(bad_formatted) -> filter(duplicates) -> aggregate() >> aggregator aggregation can utilize conditional independence causality, score-based causality, linear algebra causality. detect_anomaly() >> aggregator anomaly detection could be based on SVM, trend, etc.
Output Directed acyclic alert graph with potential root causes at the top. Rule set with an anomalous number of firing times in a time period.
Note Even though this could be consumed directly by devops, the usage of Vitrage MoNanas Sink is recommended. The output of this module can speed up creation of a Vitrage entity graph to do further analysis on it. None.

-> indicates a sequential operation in the flow.

// indicates beginning of group of operations running in parallel.

- indicates operations running in parallel.

>> indicates end of group of operations running in parallel.

Documentation

Repositories

Core: https://github.com/openstack/monasca-analytics.git

MoNanas Design

See: MoNanas/Design for details on MoNanas's architecture, its functional requirements and core concepts.

Technologies

MoNanas uses a number of third-party technologies:

Feature Release Schedule

  • Basic SML flow.
  • New algorithm "add-on" ability.
  • Example datasets and SML flows.
  • Support end-to-end learning + data processing flows (currently, the latter part does not get updated due to Spark's immutability.)
  • Refactor codes to be consistent with terms used in the documentation.
  • Add a source, ingestor and transformer for Monasca.
  • Model connections as objects rather than references and have driver specifics in one place.
  • Expanded orchestration abilities/expressiveness.
  • Container-enabled testing/deployment for non-production environments.
  • Add Vitrage Sink.
  • Add a ready-to-use virtual machine image (get rid of the fetch-deps.sh).

Contributing

There are multiple ways to contribute to the project. All are equally important to us!

  • You can have a look at the Monasca launchpad for problems that needs to be solved (bugs/issues), and blueprints.
  • You can also help us to add new learning algorithms.
  • Finally, we are very interested in having more data sources to experiment with. The source can either be from an existing data provider or randomly generated. The more, the better! :) If you are interested to work on that aspect, you are welcome as well.

For more information on setting up your development environment, see MoNanas/DevGuide.

For more information about Monanas, please visit the wiki page: Monanas wiki.

And for more information about Monasca, please visit the wiki page: Monasca wiki.

License

Copyright (c) 2016 Hewlett Packard Enterprise Development Company, L.P. Licensed under the Apache License, Version 2.0 (the "License"); you may not used this file except in compliance with the License. You may obtain a copy of the License at:

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.