Migrate ARA blog posts from dmsimard.com to ara.recordsansible.org

There's been quite a bit of blog posts over the years.
It's appropriate to migrate them to the new website.

There's some links in posts remaining on "dmsimard.com", that's
because they are references to other blog posts.

I'll adjust the links once we have them online.

Change-Id: Iaa3f4b40980f449702643ff46fe0de8a320fe65e

website/content/blog/2016-05-21_an-idea-to-store-browse-and-troubleshoot-ansible-playbook-runs/index.md

@@ -0,0 +1,182 @@
+---
+author: "David Moreau Simard"
+categories:
+  - development
+tags:
+  - ansible
+  - openstack
+date: 2016-05-21
+title: "ARA: An idea to store, browse and troubleshoot Ansible Playbook runs"
+slug: an-idea-to-store-browse-and-troubleshoot-ansible-playbook-runs
+type: post
+---
+
+# The context
+[Ansible](https://www.ansible.com/) can be used for a lot of things and it's
+grown pretty popular for managing servers and their configuration.
+
+In the [RDO](https://www.rdoproject.org/) and
+[OpenStack](https://www.openstack.org/) communities, Ansible is heavily used to
+deploy or test OpenStack through Continuous Integration (CI). Projects like
+[TripleO-Quickstart](https://github.com/openstack/tripleo-quickstart),
+[WeIRDO](https://github.com/redhat-openstack/weirdo),
+[OpenStack-Ansible](https://github.com/openstack/openstack-ansible) or
+[Zuul v3](https://specs.openstack.org/openstack-infra/infra-specs/specs/zuulv3.html#ansible)
+are completely driven by Ansible.
+
+In the world of automated continuous integration, it's not uncommon to have
+hundreds, if not thousands of jobs running every day for testing, building,
+compiling, deploying and so on.
+
+Keeping up with a large amount of Ansible runs and their outcome, not
+just in the context of CI, is challenging.
+
+# The idea
+[ARA](https://github.com/dmsimard/ara) is an idea I came up with to try
+and make Ansible runs easier to visualize, understand and troubleshoot.
+
+ARA is three things:
+
+1. An [Ansible callback plugin](https://ara.readthedocs.io/en/latest/configuration.html#ansible) to record playbook runs into a local or remote database
+2. A [CLI client](https://ara.readthedocs.io/en/latest/usage.html#querying-the-database-with-the-cli) to query the database
+3. A [web interface](https://ara.readthedocs.io/en/latest/faq.html#what-does-the-web-interface-look-like) to visualize the database
+
+ARA organizes recorded playbook data in a way to make it intuitive for you to
+search and find what you're interested for as fast and as easily as possible.
+
+It provides summaries of task results per host or per playbook.
+
+It allows you to filter task results by playbook, play, host, task or by the
+status of the task.
+
+With ARA, you're able to easily drill down from the summary view for the results
+you're interested in, whether it's a particular host or a specific task.
+
+Beyond browsing a single ansible-playbook run, ARA supports recording and
+viewing multiple runs in the same database.
+
+This allows you to, for example, recognize patterns (ex: this particular host
+is always failing this particular task) since you have access to data from
+multiple runs.
+
+ARA is an open source project available on [Github](https://github.com/dmsimard/ara) under the Apache v2 license.
+[Documentation](https://ara.readthedocs.io/en/latest/) and
+[frequently asked questions](https://ara.readthedocs.io/en/latest/faq.html) are available on readthedocs.io.
+
+# Why ?
+As I mentioned before, the vast majority of the RDO CI is powered by Ansible.
+When a job build fails, I have to look at one of these Jenkins
+[console logs](https://dmsimard.com/files/ansible-jenkins.txt) that's >8000
+lines long. If it doesn't crash my browser, I get to dig across the large
+amount of output to try and figure out what went wrong in the job build.
+
+When you're testing OpenStack trunk, you're going to be troubleshooting a *lot*
+of those large failed jobs and it's painful.
+Over time, I've (*unfortunately*) gotten used to it and got pretty good, actually.
+However, it still takes me a non negligible amount of time just to find where
+Ansible failed to know where to start searching for in the logs.
+
+It's also definitely a nightmare when someone else wants to look at the jobs 
+to try and understand what happened.
+
+ARA solves that painpoint - and many others - by making it easier to
+browse the results of a playbook.
+ 
+### Other attempts
+To try and help us before ARA was born, we leveraged two callbacks to
+try and help us parse the Ansible Playbook output.
+
+The first is [human_log.py](https://gist.github.com/cliffano/9868180) which
+helps pretty-printing output from tasks like "command" or "yum".
+We also have [profile_tasks](https://github.com/jlafon/ansible-profile/blob/master/callback_plugins/profile_tasks.py)
+that is built-in and helps by showing how much time each task took.
+
+These callbacks are definitely helpful for small playbooks or playbooks that
+contain small or short-running tasks.
+On long-running playbooks with a large amount of output, they almost make matters
+worse by adding even more output into the task results.
+
+# How do I get started with ARA ? 
+I've tried to do simple, yet effective documentation on how to get started
+with ARA.
+
+### 1) Install ARA
+Documentation: [https://ara.readthedocs.io/en/latest/installation.html](https://ara.readthedocs.io/en/latest/installation.html)
+
+First, you'll need to install some packaged dependencies and then you
+can install ARA from source or from pip.
+
+For example on a CentOS server:
+
+    yum -y install gcc python-devel libffi-devel openssl-devel
+    pip install ara
+
+### 2) Configure the callback
+Documentation: [https://ara.readthedocs.io/en/latest/configuration.html#ansible](https://ara.readthedocs.io/en/latest/configuration.html#ansible)
+([What's an Ansible Callback ?](https://ara.readthedocs.io/en/latest/faq.html#what-s-an-ansible-callback))
+
+The configuration of the callback is simple and seemless. You want to
+add the following to your ansible.cfg file:
+
+    [defaults]
+    callback_plugins = /usr/lib/python2.7/site-packages/ara/callback
+    
+    # Or, if using a virtual environment, for example
+    
+    [defaults]
+    callback_plugins = $VIRTUAL_ENV/lib/python2.7/site-packages/ara/callback
+
+### 3) Run a playbook with ansible-playbook
+Run your favorite playbook !
+
+### 4.1) Browse your data through the CLI
+Documentation: [https://ara.readthedocs.io/en/latest/usage.html#querying-the-database-with-the-cli](https://ara.readthedocs.io/en/latest/usage.html#querying-the-database-with-the-cli)
+
+    $ ara result list
+    +--------------------------------------+-------------+----------------+--------+---------------+----------------------------+----------------------------+
+    | ID                                   | Host        | Task           | Status | Ignore Errors | Time Start                 | Time End                   |
+    +--------------------------------------+-------------+----------------+--------+---------------+----------------------------+----------------------------+
+    | a73efa33-0d1e-4a7d-8e28-a76fa93b9377 | localhost   | Debug thing    | ok     | False         | 2016-05-21 14:42:24.794070 | 2016-05-21 14:42:24.837268 |
+    +--------------------------------------+-------------+----------------+--------+---------------+----------------------------+----------------------------+
+                                                                                                                                                                                                                                                                          
+    $ ara result show a73efa33-0d1e-4a7d-8e28-a76fa93b9377
+    +---------------+----------------------------------------------------+
+    | Field         | Value                                              |
+    +---------------+----------------------------------------------------+
+    | ID            | a73efa33-0d1e-4a7d-8e28-a76fa93b9377               |
+    | Host          | localhost                                          |
+    | Task          | Debug thing (d04a5828-d32f-4349-89f1-39d7400b328f) |
+    | Status        | ok                                                 |
+    | Ignore Errors | False                                              |
+    | Time Start    | 2016-05-21 14:42:24.794070                         |
+    | Time End      | 2016-05-21 14:42:24.837268                         |
+    +---------------+----------------------------------------------------+
+
+### 4.2) Browse your data through the web interface
+Documentation: [https://ara.readthedocs.io/en/latest/usage.html#browsing-the-web-interface](https://ara.readthedocs.io/en/latest/usage.html#browsing-the-web-interface)
+([What does the web UI look like ?](https://ara.readthedocs.io/en/latest/faq.html#what-does-the-web-interface-look-like))
+
+Fire off the bundled webserver:
+
+    $ ara-manage runserver
+     * Running on http://127.0.0.1:5000/ (Press CTRL+C to quit)
+
+And use your favorite browser.
+
+# There's no step five !
+We're all done here. That's the gist of it.
+
+A lot of effort was made towards making ARA as simple to install,
+configure and use as possible. It is meant to be able to run from start
+to finish locally but it is also powerful enough if you'd like to
+aggregate runs into a central server.
+
+### Discussing or contributing to ARA
+If you'd like to use ARA or contribute to the project, definitely feel
+free ! Feedback, comments, ideas and suggestions are quite welcomed as
+well.
+
+I hang out in the **#ara** channel on freenode IRC if you want to come chat about ARA !
+
+Special thanks to [Lars Kellogg-Stedman](http://blog.oddbit.com/) for the early feedback on the project, ideas and code contributions.
+He was very helpful in fleshing and maturing the idea into something better.

website/content/blog/2016-06-07_ara-one-month-and-200-commits-later/index.md

@@ -0,0 +1,103 @@
+---
+author: "David Moreau Simard"
+categories:
+  - development
+  - changelog
+tags:
+  - ansible
+  - openstack
+date: 2016-06-07
+title: "One month and 200 commits later"
+slug: one-month-and-200-commits-later
+type: post
+---
+
+On May 21st, I wrote a blog post about [ARA, an idea to store, browse and troubleshoot Ansible playbook runs](https://dmsimard.com/2016/05/21/ara-an-idea-to-store-browse-and-troubleshoot-ansible-playbook-runs/).
+Let's rewind a bit further back in time.
+
+On May 6th, I got tired of trying to make our [human_log](https://github.com/rdo-infra/weirdo/blob/master/playbooks/library/human_log.py) callback write user friendly HTML files.
+I simply wasn't happy with my [attempts](https://review.gerrithub.io/#/q/topic:human_log_html)...
+
+I'm a big fan of the UNIX philosophy: Do one thing and do it well.
+Trying to hack HTML writing into human_log didn't feel like that at all.
+
+I got frustrated and took a completely different direction.
+What if, instead of writing and appending to a html file throughout a playbook run, I'd write to a database and make a dynamic, database-driven interface.
+This would allow to organize playbook run data and also provide supported for aggregated run visualization.
+
+Not a lot of sleep was had that weekend. I [created a repository](https://github.com/dmsimard/ara/commit/ad09488bc291e6006f79110f903be962ab0d0a39) on friday and set out to have a prototype out by the following monday,
+
+And what a wild ride it's been since then.
+I first shared the idea with an [alpha prototype preview](https://www.youtube.com/watch?v=K3jTqgm2YuY).
+An awesome collaborator, [Lars Kellog-Stedman](http://blog.oddbit.com/), started contributing code, ideas and feedback.
+I [announced the project](https://dmsimard.com/2016/05/21/ara-an-idea-to-store-browse-and-troubleshoot-ansible-playbook-runs/) to the greater public with a [beta preview](https://www.youtube.com/watch?v=k3qtgSFzAHI).
+
+ARA is one month old and has more than 200 commits today. Since the prototype, ARA has gained quite a few features. Amongst other things:
+
+- Static generation of the web application, including feature parity with the dynamic web application
+- Host fact recording and browsing
+- A CLI client interface
+- Support for aborted playbook runs and overall improvements around failure and exception handling
+- A lot of unit and integration tests to ensure stability and prevent regressions ([we can even run Ansible's own integration test suite!](https://github.com/dmsimard/ara/commit/c1883013e8352eef68420e9136964547eb6a2e0c))
+
+The OpenStack and the Ansible communities are excited with ARA's potential and opportunities to help them.
+This fuels my motivation even further and I was already passionate about the project to help me in my own job to begin with.
+
+If ARA can help other Ansible users make sense of their playbook runs, saying that I'm very happy about it would be an understatement.
+
+### The next steps for ARA: To infinity and beyond
+#### A proper development, contribution and issue tracking workflow.
+The [OpenStack](http://www.openstack.org/) project has awesome [tooling, hosting, and testing infrastructure](http://docs.openstack.org/infra/system-config/).
+
+Every day, developers send hundreds of commits for review across more than a hundred projects in [Gerrit](https://review.openstack.org/#/).
+These will go through the [Zuul](https://github.com/openstack-infra/zuul) pipeline manager and eventually tested with [Jenkins](https://jenkins.io/) on ephemeral virtual machines. These virtual machines are managed by
+[nodepool](http://docs.openstack.org/infra/system-config/nodepool.html) and hosted on OpenStack cloud resources [donated by different contributing companies](http://docs.openstack.org/infra/system-config/contribute-cloud.html).
+
+OpenStack provides these resources under a program that was previously known as [StackForge](http://docs.openstack.org/infra/system-config/stackforge.html) for projects that are relevant to the OpenStack community and want to use the same development workflow.
+It does not imply governance of the projects by the OpenStack foundation or the
+OpenStack technical committee. The self-appointed core contributor
+team continues to drive the direction of the project and decides what goes in (or
+not).
+
+OpenStack happens to host several projects that are not OpenStack
+specific, a good example of which is [Jenkins Job Builder](https://github.com/openstack-infra/jenkins-job-builder) (JJB).
+While JJB was created to scratch an itch in the OpenStack community, the
+tool's development remained agnostic and today sees development and
+usage from outside the OpenStack community.
+
+There was significant interest to have ARA be part of the Ansible community umbrella as well.
+It was a hard decision, but we ended up choosing the OpenStack community as it's home.
+
+ARA, if all goes well, should be joining the OpenStack ecosystem [soon](https://review.openstack.org/#/c/321226/) and we'll update documentation with the contribution workflow once that is done.
+
+#### A stable release
+The break-neck pace of development in ARA until now has prevented us from declaring any sort of stable release beyond tagged development releases.
+
+This doesn't mean ARA has a lot of bugs, it means the database schema is still changing quite a bit. Because of this, we have chosen to not develop automated database migrations and upgrades.
+This helps us merge new features and fixes faster without database migration overhead while the project is not yet considered stable.
+
+We have started identifying features which might require a database schema change.
+Once we have ironed these out and the schema hasn't changed in a bit, it will be a good time to declare the schema stable and draft the first stable release of ARA.
+When that first stable release is out, we will try to make sure that ARA can be upgraded without having to start your database from scratch.
+
+#### An even better user experience
+I'd love ARA's web application to have it's own distinct identity, look and feel. It's a fairly generic web application with very basic html, javascript and bootstrap CSS right now.
+
+I'm really not a frontend guy. I sincerely hope that, over time, we can have skilled contributors help on that front.
+
+#### More features and improvments based on what users need
+I'm a demanding Ansible user and I'll be using ARA a lot, that's for sure.
+The feature development has been mostly driven by what I've needed so far.
+
+We have already received a lot of valuable feedback and comments about ARA.
+I'd love this trend to continue so that we can improve ARA by adding or changing things we didn't think about.
+
+If ARA doesn't do something you'd need, I want to know about it.
+If ARA could do something better, I want to know about it.
+
+We use [GitHub Issues](https://github.com/dmsimard/ara/issues) to help us keep track of things for the time being.
+
+### See you at the stable release
+The first stable release will be a great milestone for ARA. Let's do another recap once it's out !
+
+Until then, come chat with us on #ara on freenode IRC and thanks for all the interest and support !

website/content/blog/2016-11-09_visualizing-kolla-ansible-playbooks-with-ara/index.md

@@ -0,0 +1,92 @@
+---
+author: "David Moreau Simard"
+categories:
+  - community
+tags:
+  - ansible
+  - continuous integration
+  - openstack
+date: 2016-11-09
+title: Visualizing Kolla's Ansible playbooks with ARA
+slug: visualizing-kolla-ansible-playbooks-with-ara
+type: post
+---
+
+[Kolla](https://github.com/openstack/kolla) is an [OpenStack](http://www.openstack.org/) deployment tool that's growing
+in popularity right now.
+
+An OpenStack installation by Kolla was even showcased by [Chris Hoge](https://twitter.com/hogepodge) and
+[Mark Collier](https://twitter.com/sparkycollier) in one of the main [keynotes](https://www.youtube.com/watch?v=GGxzlLwt6WA&t=1325)
+at the recent [Barcelona OpenStack Summit](https://www.openstack.org/summit/barcelona-2016/).
+
+{{< tweet 790816907746213888 >}}
+
+## Installing OpenStack is complex
+
+Installing and configuring OpenStack is no easy task -- Kolla tackles this challenge with the help of [Docker](https://www.docker.com/)
+containers that are deployed with [Ansible](https://www.ansible.com/).
+
+This translates into quite a few playbooks, lots of plays, many more tasks and especially lots of data to parse through
+when trying to understand what is going on.
+
+I was recently pleased to learn that the Kolla project implemented [ARA](https://github.com/openstack/ara) to help them
+troubleshoot their CI jobs.
+
+## Installing ARA is simple
+
+How hard was it to integrate ARA with Kolla ? Eight lines of code.
+8 lines of code, that's what's what it took and all you'll see in the [code review](https://review.openstack.org/#/c/368345/)
+to implement it.
+
+![setup_gate.sh](kolla-1.png)
+
+![deploy_aio.sh](kolla-2.png)
+
+## I'm happy
+
+I'm very happy about this because this is exactly what I have been targetting with ARA.
+A tool that provides a lot of value while staying very simple, requires little configuration, is seemless and above all,
+doesn't get in the way of your existing workflows.
+
+## What it looks like
+
+So, now that Kolla has integrated ARA, what does it look like ?
+
+Let's look at a typical code review in Gerrit, we can see that one of the gate jobs failed:
+
+![gate job](kolla-gate.png)
+
+Clicking on the job link will send us to the logs where Kolla generated the ARA report inside the playbook_reports folder:
+
+![gate job](kolla-logs.png)
+
+Entering that folder gets us directly to a statically generated version of the ARA web application.
+We can see that two playbooks have been run and one of them has a failed task:
+
+![web app](kolla-webapp.png)
+
+Clicking on that particular playbook brings up the playbook summary page:
+
+![playbook](kolla-playbook.png)
+
+You can then drill down directly to the failed tasks by clicking on the appropriate column:
+
+![failed task](kolla-failed.png)
+
+Drilling down further into the particular task that failed, we see that ARA records everything from it and makes it available:
+
+![task details](kolla-task.png)
+
+If we're interested in seeing what the actual Ansible task looks like, clicking on the file line card gets us to a saved
+copy of the whole file, highlighting the line number where the task took place:
+
+![file details](kolla-file.png)
+
+## This is great, let's make it better
+
+It's great that people are using ARA and finding it useful.
+I like the current state of ARA and the direction it's going but I'm also very excited about the awesome opportunities
+some new features landing in the next release will open up.
+
+If you'd like to come chat with me about ARA, ask questions or would love to get involved, come join us on IRC.
+We hang out on the #ara channel on freenode !

website/content/blog/2016-12-01_0.10-the-biggest-release-yet-is-out/index.md

@@ -0,0 +1,98 @@
+---
+author: "David Moreau Simard"
+categories:
+  - development
+  - changelog
+tags:
+  - ansible
+  - openstack
+date: 2016-12-01
+title: 0.10, the biggest release yet, is out !
+slug: 0.10-the-biggest-release-yet-is-out
+type: post
+---
+
+19 commits, 59 changed files, 2,404 additions and 588 deletions... and more than a month's on and off work.
+
+[0.10 is out](https://github.com/openstack/ara/releases/tag/0.10.0) !
+
+Where to get it ? Get started easily by [installing](http://ara.readthedocs.io/en/latest/installation.html) and [configuring](http://ara.readthedocs.io/en/latest/configuration.html) ARA.
+
+I'm excited to tell you about this new release ! Here's a few highlights !
+
+## An improved web application browsing experience
+
+A lot of work has gone into the browsing experience: less clicks, more information, faster.
+
+I am by no means a professional frontend developer and I would definitely love help on this front but I think things are definitely starting to look good !
+I've added updated previews to the documentation:
+
+- In [screenshots](https://ara.readthedocs.io/en/latest/faq.html#what-does-the-web-interface-look-like)
+- In [video](https://www.youtube.com/watch?v=zT1l-rFne-Q&index=3&list=PLyLLwe4-L1ETFVoAogQqpn6s5prGKL5Ty) where we I showcase some playbook runs by the [OpenStack-Ansible](https://github.com/openstack/openstack-ansible) project.
+
+It's awesome to see how much the interface has improved over time.
+
+Curious ? Look at the [Alpha](https://www.youtube.com/watch?v=K3jTqgm2YuY&index=1&list=PLyLLwe4-L1ETFVoAogQqpn6s5prGKL5Ty) and [Beta](https://www.youtube.com/watch?v=k3qtgSFzAHI&list=PLyLLwe4-L1ETFVoAogQqpn6s5prGKL5Ty&index=2) video previews to see how far the project has come !
+
+## Two new Ansible modules
+
+This new release features two new Ansible modules, [ara_record](http://ara.readthedocs.io/en/latest/usage.html#using-the-ara-record-module) and [ara_read](http://ara.readthedocs.io/en/latest/usage.html#using-the-ara-read-module).
+
+These allow you to respectively write and read persistent data that is made available throughout your playbook run and in the web interface.
+
+For example:
+```
+---
+- name: Test playbook
+  hosts: localhost
+  tasks:
+    - name: Get git version of playbooks
+      command: git rev-parse HEAD
+      register: git_version
+
+    - name: Record git version
+      ara_record:
+        key: "git_version"
+        value: "{{ git_version.stdout }}"
+```
+
+That would make the git_version key/value available in the web interface and on the CLI.
+
+And you can record different kind of data, too, see:
+
+```
+---
+- ara_record:
+    key: "{{ item.key }}"
+    value: "{{ item.value }}"
+    type: "{{ item.type }}"
+  with_items:
+    - { key: "log", value: "error", type: "text" }
+    - { key: "website", value: "http://domain.tld", type: "url" }
+    - { key: "data", value: '{ "key": "value" }', type: "json" }
+```
+
+Setting a type will make it so it's displayed accordingly in the interface.
+More types will eventually be made available !
+
+## Improved unit and integration testing coverage
+
+In order to make sure ARA works well -- and keeps working well -- we need to have a good amount of unit and integration testing.
+
+This coverage was improved since the previous release and additionnally, we now test against different versions of Ansible on both CentOS and Ubuntu.
+
+## Database schema finally declared stable
+
+Was ARA unstable before ? Not really.
+However, at the rapid pace of development we were having with the project, we decided to avoid managing SQL migrations to avoid unnecessary overhead as the database schema was moving a lot.
+
+I feel the database is fairly fleshed out at this point at any new modifications will be handled automatically when running ARA.
+
+Any version of ARA >= 0.9.0 is considered a stable and managed database schema.
+
+## This is great, let's make it better !
+
+The feedback around ARA has been awesome, keep it coming !
+A lot of the improvements that are part of this release is directly from ideas and comments provided by users.
+
+If you want to come chat, learn or discuss about ARA, you'll find us on #ara on IRC in the freenode server !

website/content/blog/2017-02-13_announcing-the-release-of-0.11/index.md

@@ -0,0 +1,113 @@
+---
+author: "David Moreau Simard"
+categories:
+  - development
+  - changelog
+tags:
+  - ansible
+  - openstack
+date: 2017-02-13
+title: Announcing the release of 0.11
+slug: announcing-the-release-of-0.11
+type: post
+---
+
+We're on the road to version 1.0.0 and we're getting closer: introducing the release of version 0.11!
+
+[Four new contributors](https://github.com/openstack/ara/graphs/contributors) (!), [55 commits](https://github.com/openstack/ara/compare/0.10.0...0.11.0) since 0.10 and 112 files changed for a total of 2,247 additions and 939 deletions.
+
+New features, more stability, better documentation and better test coverage.
+
+## The changelog since 0.10.5
+
+- New feature: ARA UI and Ansible version (ARA UI is running with) are now shown at the top right
+- New feature: The Ansible version a playbook was run is now stored and displayed in the playbook reports
+- New feature: New command: "ara generate junit": generates a junit xml stream of all task results
+- New feature: ara_record now supports two new types: "list" and "dict", each rendered appropriately in the UI
+- UI: Add ARA logo and favicon
+- UI: Left navigation bar was removed (top navigation bar will be further improved in future versions)
+- Bugfix: CLI commands could sometimes fail when trying to format as JSON or YAML
+- Bugfix: Database and logs now properly default to ARA_DIR if ARA_DIR is changed
+- Bugfix: When using non-ascii characters (ex: äëö) in playbook files, web application or static generation could fail
+- Bugfix: Trying to use ara_record to record non strings (ex: lists or dicts) could fail
+- Bugfix: Ansible config: 'tmppath' is now a 'type_value' instead of a boolean
+- Deprecation: The "ara generate" command was deprecated and moved to "ara generate html"
+- Deprecation: The deprecated callback location, ara/callback has been removed. Use ara/plugins/callbacks.
+- Misc: Various unit and integration testing coverage improvements and optimization
+- Misc: Slowly started working on full python 3 compatibility
+- Misc: ARA now has a logo
+
+### ARA now has a logo !
+
+Thanks [Jason Rist](https://twitter.com/knowncitizen) for the contribution, really appreciate it !
+
+[With the icon](https://github.com/openstack/ara/blob/master/doc/source/_static/ara-with-icon.png):
+![icon](ara-with-icon.png)
+
+[Without the icon](https://github.com/openstack/ara/blob/master/doc/source/_static/ara.png):
+![full](ara.png)
+
+## Taking the newest version of ARA out for a spin
+
+Want to give this new version a try ? It's out on pypi!
+
+Install [dependencies and ARA](https://ara.readthedocs.io/en/latest/installation.html), configure the [Ansible callback location](https://ara.readthedocs.io/en/latest/configuration.html#ansible) and ansible-playbook your stuff !
+
+Once ARA has recorded your playbook, you'll be able to fire off and browse the [embedded server](https://ara.readthedocs.io/en/latest/usage.html#browsing-the-web-interface) or generate a [static version](https://ara.readthedocs.io/en/latest/usage.html#generating-a-static-html-version-of-the-web-application) of the report.
+
+## The road ahead: version 1.0
+
+What is coming in version 1.0 ? Let me ask you this question: what would *you* like in 1.0 ?
+The development of ARA has mostly been driven by it's user's needs and I'm really excited with what we already have.
+
+I'd like to finish a few things before releasing 1.0... let's take a sneak peek.
+
+### New web user interface
+
+I've been working slowly but surely on a complete UI refactor, you can look at an early prototype preview here:
+
+{{< youtube h3vY87_EWHw >}}
+
+Some ideas and concepts have evolved since then but the general idea is to try and display more information in less pages, while not going overboard and have your browser throw up due to the weight of the pages.
+
+Some ARA users are running playbooks involving hundreds of hosts or thousands of tasks and it makes the static generation very slow, large and heavy.
+While I don't think I'll be able to make the static generation work well at any kind of scale, I think we can make this better.
+
+There will have to be a certain point in terms of scale where users will be encouraged to leverage the dynamic web application instead.
+
+### Python 3 support
+
+ARA isn't gating against python3 right now and is actually failing unit tests when running python3.
+
+As Ansible is working towards python3 support, ARA needs to be there too.
+
+### More complex use case support (stability/maturity)
+
+There are some cases where it's unclear if ARA works well or works at all.
+This is probably a matter of stability and maturity.
+
+For example, ARA currently might not behave well when running concurrent ansible-playbook runs from the same node or if a remote database server happens to be on vacation.
+
+More complex use case support might also mean providing users documentation on how to best leverage all the data that ARA records and provides: elasticsearch implementation, junit reports and so on.
+
+If ARA is useful to you, I'd be happy to learn about your use case. Get in touch and let's chat.
+
+### Implement support for ad-hoc ansible run logging
+
+ARA will by default record anything and everything related to ansible-playbook runs.
+It also needs to support ad-hoc ansible commands as well. I want this before tagging 1.0.
+
+### Other features
+
+There's some other features I'd like to see make the cut for version 1.0:
+
+- [Fully featured Ansible role](https://github.com/openstack/ansible-role-ara) for ARA
+- Store variables and extra variables
+- Provide some level of support for data on a role basis (filter tasks by role, metrics, duration, etc.)
+- Support generating a html or junit report for a specific playbook (rather than the whole thing)
+- Packaging for Debian/Ubuntu and Fedora/CentOS/RHEL
+
+A stretch goal would be to re-write ARA to be properly split between client, server, UI and API -- however I'm okay to let that slip for 2.0!
+
+What else would you like to see in ARA ? Let me know in the comments, on IRC in #ara on freenode or on [twitter](https://twitter.com/dmsimard)!
+

website/content/blog/2017-03-12_an-even-better-ansible-reporting-interface-with-ara-0-12/index.md

@@ -0,0 +1,146 @@
+---
+author: "David Moreau Simard"
+categories:
+  - development
+  - changelog
+tags:
+  - ansible
+  - openstack
+date: 2017-03-12
+title: An even better Ansible reporting interface with ARA 0.12
+slug: an-even-better-ansible-reporting-interface-with-ara-0-12
+type: post
+---
+
+Not even a month ago, I announced the release of [ARA 0.11](https://dmsimard.com/2017/02/13/announcing-the-ara-0.11-release/) with a bunch of new features and improvements.
+
+Today, I'm back with some more great news and an awesome new release, ARA 0.12(.3) !
+
+That's right, 0.12.3!
+
+Due to the nature of this new release, I wanted to be sure to get feedback from the users before getting the word out.
+
+We got a lot of great input! This allowed us to fix some bugs and significantly improve the performance.
+
+0.12 features a completely re-written and re-designed web application user interface. Let's look at some of the highlights !
+
+## A new web application interface
+
+I know what you're most interested in is... WHAT DOES IT LOOK LIKE !?
+
+### What it looks like
+
+Here's some highlights of the new user interface -- it doesn't end here so please
+read on !
+
+The home page now features the data recorded by ARA:
+
+![home](home.png)
+
+The core of the user interface now revolves around one and single page where you'll be able to find all the information about your playbooks:
+
+![reports](reports.png)
+
+Quickly have a glance at your playbook host summary:
+
+![summary](summary.png)
+
+Or dig into the host details to look at all the facts Ansible gathered for you:
+
+![facts](facts.png)
+
+Figure out which tasks took the longest just by sorting the table accordingly:
+
+![sort](sort.png)
+
+Or search to figure out which tasks failed:
+
+![search](search.png)
+
+Click on the action to get context on where this task ran:
+
+![file](file.png)
+
+Or click on the status to take a look at all the details Ansible has to offer:
+
+![result](result.png)
+
+## The logic behind the UI changes
+
+There were three main objectives with this refactor of the web interface.
+
+### Improve UX
+
+A lot of effort was spent on the user experience.
+You need to be able to find what you want: intuitively, quickly and easily.
+
+Data and result tables are now sortable and searchable by default and browsing
+tips were added to the interface to help you make the most of what it has to
+offer.
+
+### Scalability and performance
+
+The interface must be fast, responsive, clutter-free, make sense and behave
+ consistently across your use case scenarios, whether you are looking at
+reports which contains five tasks or ten thousand.
+
+Pagination settings have been introduced in order to customize your browsing
+experience according to your needs.
+
+### Static report generation time and weight
+
+Another objective of this user interface work was to optimize the static report
+generation performance and weight.
+
+Static generation is one of the great features of ARA which is very heavily used
+in the context of continuous integration where the report is generated and
+attached to the artifacts of the job.
+
+Here's a real-life example of the same database being generated on ARA 0.11 and
+ARA 0.12:
+
+ARA integration tests (5 playbooks, 59 tasks, 69 results):
+
+* Before: 5.4 seconds, 1.6MB (gzipped), 217 files
+* After: 2 seconds, 1.2MB (gzipped), 119 files
+
+OpenStack-Ansible (1 playbook, 1547 tasks, 1667 results):
+
+* Before: 6m21 seconds, 31MB (gzipped), 3710 files
+* After: 20 seconds, 8.9MB (gzipped), 1916 files
+
+For larger scale playbooks, we're looking at a generation performance that is
+over 19 times faster. I'm really happy about the results.
+
+## But wait, there's more
+
+If you thought the UI work was enough to warrant it's own release, you're right !
+
+Some other changes also sneaked their way into this release as well.
+
+### First party WSGI support
+
+A lot of ARA users were interested in scaling their centralized deployment.
+This meant helping users deploy the ARA web interface through WSGI with a web server.
+
+To help people get going, we now ship a WSGI script bundled with ARA and documented
+how you can set it up with Apache and mod_wsgi.
+The documentation is available [here](http://ara.readthedocs.io/en/latest/webserver.html).
+
+### Other things
+
+* Fixed syntax highlighting when viewing files
+* Preparations for supporting the upcoming Ansible 2.3 release
+* Started working on full python 3 support
+* Various performance improvements
+
+## Well, that's it for now
+
+That was certainly a lot of stuff in one release !
+
+I hope you're enjoying ARA - if you're not using it yet, it's easy !
+
+Have a look at the documentation to learn how to [install ARA](http://ara.readthedocs.io/en/latest/installation.html)
+and how to [configure Ansible](http://ara.readthedocs.io/en/latest/configuration.html) to use it.
+
+If you have any questions, feel free to drop by on IRC in #ara on the freenode server or hit me up on twitter: [@dmsimard](https://twitter.com/dmsimard).

website/content/blog/2017-05-05_ara-0-13-is-out-and-its-awesome/index.md

@@ -0,0 +1,148 @@
+---
+author: "David Moreau Simard"
+categories:
+  - development
+  - changelog
+tags:
+  - ansible
+  - openstack
+date: 2017-05-05
+title: "ARA 0.13 is out and it's awesome !"
+slug: ara-0-13-is-out-and-its-awesome
+type: post
+---
+
+I'm excited to announce the release of ARA: Ansible Run Analysis 0.13.0!
+
+ARA 0.13.0 is available on [PyPi](https://pypi.python.org/pypi/ara) or from source on [GitHub](https://github.com/openstack/ara).
+I'm also happy to announce that ARA 0.13.0 will be the first version of ARA packaged for Fedora and CentOS EPEL.
+
+Stay tuned in the near future to hear when the packages will be available.
+
+## Wait, what's ARA ?
+
+ARA is an Ansible callback plugin that records your playbook runs, wherever it is.
+
+Whether you're running Ansible from your personal laptop or from a server,
+you basically just need to [install ARA](http://ara.readthedocs.io/en/latest/installation.html),
+configure [Ansible](https://ara.readthedocs.io/en/latest/configuration.html#ansible) to use ARA and you're good to go.
+
+ARA organizes the data in a way to help you visualize, understand and troubleshoot
+what happened throughout your playbook.
+
+So, what does it look like ?
+Here's a video demo of the interface where I explain the different features it
+offers:
+
+{{< youtube k3i8VPCanGo >}}
+
+ARA also provides a [command line interface (CLI)](http://ara.readthedocs.io/en/latest/usage.html#querying-the-database-with-the-cli)
+as well for use in your different scripts or implementations.
+
+## What's new in 0.13
+
+I'm very surprised with the amount of improvements that we managed to land in
+0.13, I hope you'll be as happy as I am !
+
+The full changelog is available on [GitHub](https://github.com/openstack/ara/releases/tag/0.13.0)
+but let's highlight the really cool stuff.
+
+### Permanent links !
+
+If you've been using ARA for a while, you know that before the recent UI re-design,
+almost the entirety of the content had their own links.
+
+This, in fact, was quite problematic as it meant creating thousands and thousands
+of files when generating the static HTML version of the ARA web application.
+
+With the new UI, we're leveraging a lot of fun hacks in order to optimize the
+time it takes to generate a static report, it's size and it's weight.
+
+In any case, permanent links are back and this time, without a significant impact
+on the static generation so I'm very happy with that.
+
+Find the blue chain icon to get your permanent links !
+
+![permalinks](permalinks.png)
+
+### Playbook parameters and task tags
+
+When you run the ``ansible-playbook`` command, you can pass options to it.
+
+Whether that's an inventory file (``-i /path/hosts``), tags (``--tags production``)
+or maybe extra-vars (``--extra-vars var=value``).
+
+These are now all recovered by ARA and attached to your playbook report:
+
+![parameters](parameters.png)
+
+Extra vars are not saved by default as a security precaution since it is often
+used for passing things like passwords. You can make it so ARA saves extra vars
+or does not save another key you feel might contain sensitive data in the
+[configuration](https://ara.readthedocs.io/en/latest/configuration.html#ara-ignore-parameters).
+
+Additionally, ARA now records task tags which allows to highlight when tasks
+are tagged and to search for them in the task list:
+
+![parameters](tags.png)
+
+### Get to your reports faster with more content in less space
+
+We've slimmed down the width requirements of the web interface while also making
+more room inside the panels for content.
+
+I am not interested in restricting the width of the application so that users
+with larger resolutions can fully take advantage of their width.
+
+It's a delicate balance to maintain so that larger resolutions don't feel like
+there's too much whitespace and smaller resolutions are still functional. I think
+we've struck an acceptable middle ground.
+
+To that end, the browsing tips were useful but were taking too much screen
+real estate and have been folded into "?" icons in each panel.
+
+You'll also notice that the "Home" page has been relocated to "About" and that
+the playbook reports page is now the default home page.
+
+The "About" page serves it's purpose at explaining what ARA is and what it is
+doing so it's actual content remains unchanged.
+
+### A better view of your recorded files
+
+After a lot of headaches, we've finally been able to land a proper file
+panel to make your files easier to browse.
+
+This is what is looked like before and after:
+
+![file](new-file-tab.png)
+
+I'm pretty happy with the way it turned out and the headaches were worth it !
+
+### Bug fixes
+
+What good software does not have any bugs ? We fixed a few things here and
+there:
+
+- Include tasks could be recorded twice on Ansible >= 2.2
+- Tasks using loops (ex: ``with_items``) now record each item result
+- Performance improvements
+
+## That's it for now
+
+0.13 was supposed to be a small release in preparation of packaging ARA for
+RHEL-based distributions. Turns out I was wrong, there's a bunch of stuff in
+there and now that it's out, I can sleep better !
+
+For the road to the next version of ARA, I'd really love to target full python 3
+compatibility and I could definitely use some help.
+
+If you'd like to contribute to ARA's development, you can find documentation
+on how to do it [here](https://ara.readthedocs.io/en/latest/contributing.html).
+
+Otherwise, keep the feedback coming !
+
+ARA functionality is in large part driven by users' needs and feedback.
+Come chat with us on IRC on the freenode server in ``#ara``.
+
+In celebration of this new release, I'm also hosting an AMA (Ask me anything)
+on [Reddit](https://www.reddit.com/r/ansible/comments/69gkpz/hi_ransible_a_new_version_of_ara_ansible_run/) -- feel free to join !

website/content/blog/2017-05-08_ara-is-one-year-old-a-look-back-at-the-past-year/index.md

@@ -0,0 +1,255 @@
+---
+author: "David Moreau Simard"
+categories:
+  - community
+tags:
+  - ansible
+  - openstack
+date: 2017-05-08
+title: "ARA is one year old! A look back at the past year."
+slug: ara-is-one-year-old-a-look-back-at-the-past-year
+type: post
+---
+
+ARA is one year old, happy birthday ARA !
+
+![first-commit](first-commit.png)
+
+ARA's come a long way since the early prototypes.
+
+The latest version, [0.13](https://dmsimard.com/2017/05/05/ara-ansible-run-analysis-0-13-is-out-and-its-awesome/), looks pretty awesome.
+It even caught the eye of Michael DeHaan, the author of Ansible !
+
+{{< tweet 861007096803995648 >}}
+
+Let's go back in time to look at the humble beginnings of the project and some
+of the important milestones that marked it's history this past year.
+
+## An idea and a first prototype
+
+I was beyond frustrated from trying to understand and troubleshoot Ansible
+playbook runs at scale.
+
+In the [OpenStack](https://www.openstack.org/) and
+[RDO](https://www.rdoproject.org/) communities, we heavily leverage Ansible for,
+amongst other things, continuous integration and testing jobs.
+
+This meant part of my days were spent looking at tens of thousands of lines of
+Ansible playbook output. The [human_log](https://gist.github.com/cliffano/9868180) callback
+did not really help, it almost made matters worse by making the output even larger.
+
+I got the idea for ARA on Friday **May 6th 2016**.
+I started hacking that evening and after what was probably the most time I've
+spent coding in a weekend, I had a prototype to share with close colleagues
+and friends to validate the idea with the very first demo video.
+
+The family was unfortunately neglected during that weekend !
+
+{{< youtube K3jTqgm2YuY >}}
+
+<br>
+## A very, very important contributor
+
+One of those colleagues that ended up finding out about this prototype was
+Lars Kellogg-Stedman ([@larsks](https://twitter.com/larsks)) and I owe a lot of
+ARA's success to him today.
+
+I don't consider myself a professional programmer by any stretch. Just a system
+administrator passionate with building tools to help him in his job.
+
+Lars came around with [code contributions](https://github.com/openstack/ara/commits?author=larsks)
+for things I admittedly wasn't very good at.
+
+A much better database schema, a better structure and framework for the flask web
+application, a lot of great improvements to the callback plugin, UNIT TESTS
+(of which there was probably none at the time) and I could keep going.
+
+Lars was pretty busy with other projects at the time, which made me appreciate
+so much more the time I spent discussing and developing ARA with him.
+
+## ARA's first public appearance
+
+On **May 21st 2016**, I shared ARA for the first time to the public in a blog post:
+[ARA: An idea to store, browse and troubleshoot Ansible playbook runs](https://dmsimard.com/2016/05/21/ara-an-idea-to-store-browse-and-troubleshoot-ansible-playbook-runs/).
+
+With this post came a second demo video:
+
+{{< youtube k3qtgSFzAHI >}}
+
+<br>
+## ARA becomes an OpenStack community project
+
+We're **June 7th 2016**, [ARA is one month old and we've broken 200 commits already](https://dmsimard.com/2016/06/07/ara-one-month-and-200-commits-later/).
+
+**June 8th 2016**, the [patch](https://review.openstack.org/#/c/321226/) to make ARA
+an OpenStack community project merges and [github.com/openstack/ara](https://github.com/openstack/ara) is born.
+
+To this day, I still keep the [github.com/dmsimard/ara](https://github.com/openstack/ara) repository around for
+the sake of keeping the history of issues and pull requests that were created before ARA
+joined the OpenStack community.
+
+## A first OpenStack project starts using ARA
+
+This came to me as a surprise, **[September 10th 2016](https://review.openstack.org/#/c/368345/)**,
+a Kolla contributor implemented ARA to their integration tests in order to help them
+figure out what's going on.
+
+It was the [first official OpenStack project to use ARA](https://dmsimard.com/2016/11/09/visualizing-kolla-ansible-playbooks-with-ara/).
+It all snowballed from there.
+
+## A second OpenStack project tries ARA
+
+After I [shared that Kolla used ARA](https://dmsimard.com/2016/11/09/visualizing-kolla-ansible-playbooks-with-ara/),
+the OpenStack-Ansible project submitted a [patch](https://review.openstack.org/#/c/396565/)
+to add it to their project on **November 11th 2016**.
+
+[OpenStack-Ansible](https://github.com/openstack/openstack-ansible) was a very important early adopter of ARA, I owe them a lot.
+OpenStack-Ansible simply operated at a much larger scale than we did at RDO.
+
+They had thousands of tasks, many more results and this highlighted a lot of the assumptions
+and flaws that ARA had at the time on operating at a large scale.
+
+The UI and the performance were simply not suited for this kind of scale and we were able
+to improve ARA a lot thanks to their feedback.
+
+## ARA 0.10: A stable database and a first UI redesign
+
+**December 2016**: [ARA 0.10 was the biggest release of ARA at the time](https://dmsimard.com/2016/12/01/ara-0.10-the-biggest-release-yet-is-out/).
+
+It marked the first version where we felt the database schema would not move as much
+and we could manage to not break users by implementing SQL migrations without too much
+development overhead.
+
+ARA 0.10 also shipped a first major UI redesign that would later be the foundation
+of the interface ARA still uses today:
+
+{{< youtube zT1l-rFne-Q >}}
+
+<br>
+## ARA 0.11: ARA gets a logo and a new UI prototype
+
+**[January 2017](https://github.com/openstack/ara/commit/0d5d0939a6b7a319d99acc1fb20d4ca282bd76ab)**,
+Jason Rist ([@knowncitizen](https://twitter.com/knowncitizen)) came up with a logo for ARA !
+
+With the icon:
+![icon](ara-with-icon.png)
+
+Without the icon:
+![full](ara.png)
+
+It looked awesome, I was really happy about it and it first made it's way in
+[version 0.11](https://dmsimard.com/2017/02/13/announcing-the-ara-0.11-release/) of **February 2017**.
+
+This is also around the time where I started experimenting with yet another redesign
+for the user interface:
+
+{{< youtube h3vY87_EWHw >}}
+<br>
+I feel like version 0.11 was the tipping point where users outside of the OpenStack
+community *really* started getting interested in ARA.
+
+I got plenty of interest over social media and started getting a lot of questions from users.
+
+{{< tweet 831617879804088325 >}}
+
+<br>
+## ARA 0.12: A second UI redesign lands
+
+**March 2017**, [ARA 0.12](https://dmsimard.com/2017/03/12/an-even-better-ansible-reporting-interface-with-ara-0-12/) is released
+with a first iteration of the new user interface.
+
+0.12 was the first version I felt confident recommending to users running Ansible at a large scale.
+The UI redesign had those large scale users in mind while staying efficient and intuitive for small scale users.
+
+It made browsing large amount of contents easy and straightforward and a lot of considerations went into making the interface as responsive as possible. It also very significantly improved one of the key features of ARA, the
+[static HTML generation of the web application](https://ara.readthedocs.io/en/latest/usage.html#generating-a-static-html-version-of-the-web-application).
+
+0.12 was also the first time I did a voiced demo of the ARA web interface to highlight its different features:
+
+{{< youtube aQiN5wBXZ4g >}}
+<br>
+The amount of interest in ARA was growing from all over the place.
+It was exciting, motivating but also a bit overwhelming.
+This is about the time where I [reached out](http://lists.openstack.org/pipermail/openstack-dev/2017-March/114509.html)
+to the OpenStack community to ask if they wanted to help developing ARA.
+
+This call for help brought a few contributions but not as much as I would have hoped.
+The fact that ARA is not my full time job means I have to spend a lot my 
+personal time on the project but in the end, the user feedback is so rewarding
+that I don't mind too much.
+
+That said, I have to say I am extremely grateful to my employer, Red Hat, and my
+managers who don't mind me spending time on ARA. Red Hat is awesome.
+
+## An Ansible meetup talk and STICKERS !
+
+**April 2017**, I was invited to speak at the [Ansible Montreal](https://www.meetup.com/Ansible-Montreal/events/238557821/)
+meetup to talk about how Ansible users can extend Ansible with the help of plugins.
+
+I totally had to make ARA stickers to bring to the meetup !
+
+{{< tweet 844173191627116545 >}}
+
+I did not bring enough stickers, though. It was a limited print and I wanted
+to keep some for the next occasion. The stickers were gone in minutes !
+
+The meetup was a great opportunity for me to highlight how ARA was taking advantage
+of the plugin system in Ansible.
+
+{{< tweet 852318379406962688 >}}
+
+You can find the slides for that talk (*in french*) [here](http://redhat.slides.com/dmsimard/comment-personnaliser-et-etendre-ansible-module-actions-et-callbacks).
+
+## ARA 0.13: Where we are today
+
+**May 4th 2017**, there's lightsaber battles at the Red Hat Summit and [ARA 0.13 is released](https://dmsimard.com/2017/05/05/ara-ansible-run-analysis-0-13-is-out-and-its-awesome/).
+
+{{< tweet 860111867552944128 >}}
+
+ARA 0.13 improved on the foundation that 0.12 brought: bug fixes, new features, better performance
+and a better and more streamlined UI.
+
+There were so many improvements in 0.13 that I had to create yet another video demo
+because the demo from 0.12 felt outdated.
+
+{{< youtube k3i8VPCanGo >}}
+<br>
+The amount of interest in ARA with this new release is absolutely nuts.
+
+Lots of comments on [Reddit](https://www.reddit.com/r/ansible/comments/69gkpz/hi_ransible_a_new_version_of_ara_ansible_run/) and
+so many Twitter mentions, likes and retweets that I had to create a filter for
+all the email notifications... My phone wouldn't stop ringing!
+
+## The future: what's next
+
+I could have never predicted what would become of this idea I had a year ago and
+with that in mind, I definitely can't predict what the next year holds for ARA.
+
+On the technical side of things, there are clear things we need to be doing to
+further improve ARA.
+
+ARA needs full python 3 compatibility, we need to have a better split of the
+components (web application, API, client, CLI) and things like that.
+I'm not a frontend developer and I'm convinced a professional UX developer could
+make the UI so much better.
+
+There's other very interesting opportunities to make the life of Ansible users
+easier, not just with Ansible itself but through it's implementation in other
+tools.
+
+There are [questions](https://www.reddit.com/r/ansible/comments/69gkpz/hi_ransible_a_new_version_of_ara_ansible_run/dh6tiyt/)
+with the eventual Open Source release of [Ansible Tower](https://www.ansible.com/tower)
+and how we might see Tower and ARA working together.
+
+ARA might also end up integrated in [some shape or form](https://review.openstack.org/#/c/444088/)
+with [Zuul](http://status.openstack.org/zuul/), OpenStack's famous gate and pipelining
+system.
+
+Some people have also been requesting ARA to be a native plugin in [Molecule](https://github.com/metacloud/molecule/issues/728).
+
+What's next ? Only time will tell !
+
+Thanks to everyone that was involved with ARA so far, even as a user, your feedback
+and questions have contributed to shaping ARA into what it is today.
+
+See you next year :)

website/content/blog/2017-08-16_whats-coming-in-ara-1.0/index.md

@@ -0,0 +1,240 @@
+---
+author: "David Moreau Simard"
+categories:
+  - development
+tags:
+  - ansible
+  - openstack
+date: 2017-08-16
+title: "What's coming in ARA 1.0 ?"
+slug: whats-coming-in-ara-1.0
+type: post
+---
+
+Not long ago, I wrote that ARA: Ansible Run Analysis had it's first [birthday](https://dmsimard.com/2017/05/08/ara-is-one-year-old-a-look-back-at-the-past-year/).
+
+It was an important milestone and it was a great opportunity to reflect back on where
+the project was coming from and think about what we needed to do in the future.
+
+Just for fun, let's look at what I had written back in May to summarize what
+was probably coming:
+
+- ~~Python 3 compatibility~~
+  - This is done and was shipped in [ARA 0.14](https://github.com/openstack/ara/releases/tag/0.14.0)!
+    It was a lot of work but ARA is now working and is fully tested against both python2.7 and python3.5.
+- ~~An implementation in Zuul~~
+  - This is in [progress](https://review.openstack.org/#/c/487853/) ! It's no longer a question of "if" but rather "how".
+    A lot of improvements in ARA 1.0 is geared towards making this integration easier (amongst other things) and we'll
+    have a lot of discussions about it at the upcoming [OpenStack project team gathering](https://www.openstack.org/ptg/).
+- ARA and [Ansible Tower](https://www.ansible.com/tower) working together ?
+  - I'm excited to see the opportunities with Tower but it's not open source yet !
+    We know it's coming soon and [latest estimates](https://www.reddit.com/r/ansible/comments/6jat04/my_review_of_ansiblefest_london_2017/djdrjgk/)
+    put us at a release within the next year.
+
+## ARA 1.0: An important version
+
+ARA 1.0 will be the largest and most significant release ever since the project
+was created.
+
+For me, 1.0 is not just any number. I attach a lot of weight and importance to
+it.
+
+First, it signals a very clear message that I feel ARA is battle tested and stable
+enough for production use.
+Users and deployments, both small and large, have proven me that ARA can be relied on.
+
+Second, it means that we are making backwards incompatible changes, especially
+around the database schema: more on that later.
+
+And finally, we are completely re-writing the backend and it's logic, both of which
+had been implemented in the very early days when prototyping ARA.
+This new foundation will help ARA move forward faster and make integration
+easier for users and systems.
+
+Development of ARA 1.0 is not finished yet.
+You can follow the progress and the code reviews [here](https://review.openstack.org/#/q/project:openstack/ara+branch:feature/1.0)
+or by hanging out in IRC in #ara on the freenode server.
+
+Here's some highlights about things that have already landed or will land in this next release.
+
+## What's coming: decoupling components and a revamped backend
+
+ARA will have lived until 1.0 on a backend implementation that was born out of the
+prototype and proof of concept more than a year ago. It has served us well but it was
+no longer appropriate for the scope of the project.
+
+One of the biggest flaws of the previous backend implementation was that the
+different components were coupled and also imported and queried the database model directly.
+
+For example, when using ARA in a centralized fashion, this meant:
+
+- Installing the callback, module, CLI and Ansible on the web server(s) hosting the web application
+- Installing the web application (and all it's dependencies) when only requiring the CLI, callback or modules
+- Putting/sharing a username/password for a database connection string to send data back to a central database
+
+From a developer perspective, it also meant that we had no abstraction to the database and needed
+to know and understand how the different components worked to introduce new features.
+It also meant understanding how to query the database directly and code duplication
+because every feature had their own database queries.
+
+The backend is being essentially re-written from scratch in order to leverage a
+new python and REST API.
+This layer of abstraction will make things easier to maintain and develop against.
+
+## What's coming: a python and REST API
+
+What is probably the biggest endeavour of the 1.0 release is the introduction of python and REST APIs.
+
+We are doing this for the project's own consumption in the first place to decouple the
+components and re-write the backend.
+However, I'm curious to see what users and developers might come up with !
+
+Something that was important for me when designing the API was to keep ARA simple.
+The introduction of an API should not mean that users suddenly have to run a web server in order to collect data.
+It shouldn't also be a requirement when running the web application because the reports can be
+generated to static HTML.
+
+Before 1.0, all you had to do to get started was to [install ARA](http://ara.readthedocs.io/en/latest/installation.html) and configure the [Ansible callback path](http://ara.readthedocs.io/en/latest/configuration.html).
+This will not change and the default use case remains local and offline data collection with no additional
+set up or configuration steps.
+
+The approach I am taking with the two APIs is to make them identical with the exact
+same interface. In fact, the python API is simply an internal passthrough to the REST API
+with no HTTP calls involved. This will make it easier to use, develop, test and maintain because of the
+feature parity that is available by default.
+
+Here's some examples to show the APIs are the same -- these examples do and return the exact same thing.
+
+**For retrieving data**:
+
+```
+# Get the playbook with 'id' 1 over http
+curl -H "Content-Type: application/json" -X GET <ara>/api/v1/playbooks/1
+
+# Get the playbook with 'id' 1 with the python API
+from ara.api.playbooks import PlaybookApi
+playbook = PlaybookApi().get(id=1)
+```
+
+**For creating data**:
+
+```
+# Create a play in a playbook over http
+curl -H "Content-Type: application/json" -X POST -d '{ "playbook_id": 1, "name": "Play name", "started": "1970-08-14T00:52:49.570031" }' <ara>/api/v1/plays/
+
+# Create a play in a playbook with the python API
+from ara.api.plays import PlayApi
+
+data = {
+    "playbook_id": 1,
+    "name": "Play name",
+    "started": "1970-08-14T00:52:49.570031"
+}
+play = PlayApi().post(data)
+```
+
+**For updating data**:
+
+```
+# Update a play in a playbook over http
+curl -H "Content-Type: application/json" -X PATCH -d '{ "play_id": 1, "ended": "1970-08-14T00:52:49.570031" }' <ara>/api/v1/plays
+
+# Update a play in a playbook with the python API
+from ara.api.plays import PlayApi
+
+data = {
+    "play_id": 1,
+    "ended": "1970-08-14T00:52:49.570031"
+}
+play = PlayApi().patch(data)
+```
+
+There might eventually be some "wrapper" methods for the Python API (i.e, "create_playbook")
+but in the end, it will use the same API behind the scene.
+
+## What's coming: input and output drivers
+
+Currently, there is only one supported way to send data to ARA -- it's through the
+Ansible callback.
+However, some users have also requested different means of sending data to ARA.
+
+An example that was given is that events could be dropped on a message bus (ex: mqtt, rabbitmq)
+instead of being sent directly to the API or database and then ARA could pick up and
+process the events asynchronously.
+
+The input will be made modular and generic to allow the notion of "input drivers" so that
+we can implement these new methods easily.
+
+For output, in a similar fashion, it's currently possible to export data from ARA to
+static HTML, junit or subunit.
+
+There's also been interest in sending data to other backends such as graphite, elasticsearch or influxdb.
+
+The appeal in using ARA to export data to these backends instead of using different Ansible callbacks
+that already do that comes from having the data aggregated, organized and indexed in one place and
+then being able to export the data asynchronously.
+
+The current export methods will be folded back as "output drivers" and will use a common
+interface to export data. We will be able to leverage this to make it easier to implement
+other output drivers.
+
+## What's coming: database improvements and backwards incompatibility
+
+A significant amount of work has been done on the database model.
+The magnitude of these changes have made me consider backwards incompatibility
+for the first time since ARA introduced database upgrades in [ARA 0.10 from December 2016](https://github.com/openstack/ara/releases/tag/0.10.0).
+
+The decision to move forward without supporting migrations and upgrades was to make this work possible and drastically easier.
+
+I think the database layout is almost ready to go for 1.0, here's some examples of what has been done so far:
+
+- Primary keys and identifiers are now integers instead of UUIDs
+  (Thanks [Monty Taylor](http://lists.openstack.org/pipermail/openstack-dev/2017-April/115013.html) for the tip)
+- The ``Stats`` and ``HostFacts`` tables were merged with the ``Hosts`` table
+- The ``TaskResults`` table was renamed to ``Results`` because there's no other kind of results
+- The ``Data`` table was renamed to ``Records`` because "Data" was an awful name and it better matches it's intended use case with the ``ara_record`` [module](http://ara.readthedocs.io/en/latest/usage.html#using-the-ara-record-module).
+- Different fields have been removed or renamed to make their names more accurate and their meaning consistent throughout the application
+- Some relationships between components were added, removed or changed
+
+## What's coming: new features
+
+All of the previous changes do not have much of an impact on end users. They are
+all mostly under the hood and does not add much immediate value for users.
+
+Here's some features I would like to land in 1.0:
+
+- Add support for recording ad-hoc command execution (ex: ``ansible -m service -a "name=foo state=stopped``)
+- Add support check/dry mode (``ansible-playbook --check``)
+- Add support diff mode (``ansible-playbook --diff``)
+- Add support for grouping or searching playbooks
+
+**Edit**: The following features will also be landing in 1.0!
+
+- Inventory, vars, host_vars and group_vars files will be saved with the report
+- Role handlers, meta and defaults files will also be saved with the report
+- Hosts will now display which groups they are a member of
+- Playbook reports will be able to be named for displaying in the UI. For example
+  you will be able to name a playbook "deploy mysql" and it will display that instead
+  of the playbook file name "<playbook>.yml".
+
+## When is all of that coming ?
+
+So, when is all of that coming, you ask ?
+
+Work is progressing very well on the API implementation and there should not be
+too many changes remaining to do in the database model.
+
+{{< tweet 897641159681593344 >}}
+
+There has not been any work so far to refactor the backend to use the API and there
+are also some features I'd like to land.
+
+With all things considered, I would like to be able to release an alpha or a beta
+release sometime during September. I'll reach out to ask users to test it out :)
+
+If you'd like to follow the progress, I sometime post updates on Twitter, you can
+follow me there as [@dmsimard](https://twitter.com/dmsimard).
+
+Feel free to come hang around on IRC to chat about ARA as well !
+You'll find users and myself in #ara on the Freenode server.
+

website/content/blog/2017-08-27_new-ways-of-reaching-the-ara-community/index.md

@@ -0,0 +1,63 @@
+---
+author: "David Moreau Simard"
+categories:
+  - community
+tags:
+  - ansible
+  - openstack
+  - irc
+  - slack
+  - twitter
+date: 2017-08-27
+title: "New ways of reaching the ARA community"
+slug: new-ways-of-reaching-the-ara-community
+type: post
+---
+
+More and more users requested other ways of reaching the ARA community and I've
+finally given in !
+
+Until now, the only way of getting in touch was through IRC and I understand
+that, in 2017, IRC is not for everyone.
+
+## Seamless communication across IRC, Slack ~~and Discord~~
+
+That's right, you can now reach us through Slack and Discord.
+Both are linked to IRC so messages sent to one will be relayed automatically
+to the others.
+
+- To join us on Slack, use this [invitation](https://join.slack.com/t/ara-community/shared_invite/MjMxNzI4ODAxMDQxLTE1MDM4MDEzMTEtNzU1NTUwMTcyOQ)
+- ~~To join us on Discord, use this [invitation](https://discord.gg/z2SGdc9)~~ (With over 30 people in the ARA slack and none in the Discord, we discontinued it.)
+
+To join us on **IRC**, you'll find us in the **#ara** channel of the **freenode** server.
+A web chat application to connect to IRC is available [here](https://webchat.freenode.net/).
+
+### On Reddit
+
+When I'm not discussing Ansible or answering questions about it in
+[/r/ansible](https://www.reddit.com/r/ansible/), you might come across noteworthy
+announcements about ARA that I post there.
+
+There's a lot of discussion around ARA on /r/ansible and I will never get used
+to users recommending other users that they use ARA, it's pretty awesome.
+
+### On Twitter (new!)
+
+You might have been following me on Twitter ([@dmsimard](https://twitter.com/dmsimard))
+to keep up with updates I post about ARA's developments or new announcements.
+
+That's cool but in case you'd like a curated feed that contains only worthwhile
+updates about things relevant to Ansible and ARA, the new account to follow
+is [@ara_community](https://twitter.com/ara_community).
+
+{{< tweet 901820543539798016 >}}
+
+## Other things
+
+No, I'm not going to stand up a WhatsApp, a Snapchat, an Instagram and a Facebook
+page :)
+
+This is the best you've got for now !
+
+Thanks for reading and see you online !
+

website/content/blog/2017-11-22_status-update-ara-1.0/index.md

@@ -0,0 +1,394 @@
+---
+author: "David Moreau Simard"
+categories:
+  - development
+tags:
+  - ansible
+  - openstack
+date: 2017-11-22
+title: "Status update: ARA 1.0"
+slug: status-update-ara-1.0
+type: post
+---
+
+Back in [August](https://dmsimard.com/2017/08/16/whats-coming-in-ara-1.0/), I
+posted about what was the roadmap for ARA 1.0 and why it was a very important
+milestone for the project.
+
+We're now almost in December and I said there would likely be a beta version
+out by September. "What's going on ?", one might ask... A fair question.
+
+There's definitely been progress and I could've been doing a better job at
+communicating updates other than the [tweet](https://twitter.com/ara_community)
+from time to time. It's time for a lengthy status update !
+
+If you don't know what's ARA yet, you should be excited and read about what's
+coming. Looking at a [video demo](https://www.youtube.com/watch?v=k3i8VPCanGo&list=PLyLLwe4-L1ETFVoAogQqpn6s5prGKL5Ty&t=12s&index=6)
+will give you an idea of what the project can already do today.
+
+This is a companion blog post to a talk given at the
+[Ansible Montreal](https://www.meetup.com/Ansible-Montreal/) meetup, you can see
+the slides (in french) [here](https://redhat.slides.com/dmsimard/ara-10-update).
+
+If you'd like skip ahead and read the technical details about what's done
+and what's not, feel free to scroll down a bit !
+
+How is 1.0 being developed ?
+============================
+
+Time goes quickly when we're having fun, doesn't it ? 1.0 has been in progress
+for a good four months already.
+
+1.0 is being developed 100% in the open. If you're on [Slack or IRC](https://dmsimard.com/2017/08/27/new-ways-of-reaching-the-ara-community/)
+you might have seen one of the notifications that is sent every time there is
+a new patch submitted for code review.
+
+![slack notification](slack-notification.png)
+
+You can also see all the patches as well
+as the results for their unit and integration tests live on
+[Gerrit](https://review.openstack.org/#/q/project:openstack/ara+branch:feature/1.0).
+
+There's currently two git branches,
+[master](https://github.com/openstack/ara/tree/master) and
+[feature/1.0](https://github.com/openstack/ara/tree/feature/1.0). This allows
+the development of version 1.0 to move forward without impacting the master
+branch from which the 0.15.x series of releases are currently tagged from.
+
+These branches have diverged a LONG time ago and are VERY different by now.
+
+![git network](git-network.png)
+
+Keeping these two separate branches is great, but it also means that bugfixes
+done in a branch that is applicable to the other branch should be backported.
+
+Sometimes, this is easy, other times... it is not. Ever heard of merge
+conflicts ? Maybe some code is gone, perhaps it has moved elsewhere or has
+changed dramatically.
+
+While I am trying to keep the master branch frozen for new features in order to
+reduce the maintenance cost associated with this model, there are some things
+that absolutely need to go through.. for example Ansible 2.4 support.
+
+It was unrealistic to delay the support for Ansible 2.4 until ARA 1.0.
+
+Ansible 2.4 changed (and [broke](https://github.com/ansible/ansible/pull/31200))
+some internal methods ARA was relying on and there was no way to support it
+properly unless we adapted to it.
+
+This is compounded by the fact that ARA supports all stable releases of Ansible
+in both Python 2 and Python 3 which have not reached their
+[end of life](http://docs.ansible.com/ansible/latest/release_and_maintenance.html)
+yet.
+
+Hopefully we can ship this awesome 1.0 release soon so we no longer have to
+maintain these two branches in parallel !
+
+Why is this taking so long ?
+============================
+
+I'll be honest here: like probably many open source software maintainers, ARA
+isn't my full time job. I'm not sure I would enjoy working on ARA full time
+anyway. I enjoy system administration and architecture way too much.
+
+Don't get me wrong, I'm quite passionate about ARA but it's ultimately a project
+I created to make my job easier. I'm not making money with ARA, the fact that
+it is useful to other people is super rewarding for me and that's enough.
+
+I consider myself very privileged to be working at Red Hat which values
+innovation driven by open source projects. While Red Hat grants me a certain
+amount of freedom to work on ARA at my discretion, I recognize that my role is
+very important for the [RDO](https://www.rdoproject.org/) and
+[OpenStack](https://www.openstack.org/) communities and that's why I'll always
+priorize spending the vast majority of my time there.
+
+So far, ARA has been largely developed on my own time and I had to take a step
+back for a bit.
+
+I like to say that work isn't work when you're having fun -- I still believe
+that, but I was just "working" too much for a while there.
+
+I'll always be happy to receive and review patches from contributors in ARA.
+If you'd like to help, you can get started by looking at the
+[contribution documentation](http://ara.readthedocs.io/en/latest/contributing.html).
+
+So when is it shipping ?
+========================
+
+I'd rather not try to plan for a date again like I did when I mentioned
+September for a beta. It sucks because I failed to deliver and it sucks because
+I am not managing the expectations of the users.
+
+1.0 will ship when it is ready.
+However, the primary purpose of 1.0 is to be the sandbox where we allow
+ourselves to break all backwards compatibility.
+
+If we're done merging all the features that we have an expectation would have
+an impact on backwards (in)compatibility, it means we're nearing release.
+
+I'm not going to delay the release indefinitely, things without impact that
+aren't ready for 1.0 will ship later, that's all.
+
+I'll stay vague on purpose: in the first half of 2018 ? If you end up having it
+sooner, you can celebrate !
+
+Let's move on to the status update of things.
+
+The API is done (kind of)
+========================
+
+I took an iterative approach and there was definitely some trial and error
+involved.
+
+I decided to write a first version of the API by itself, only being validated
+through purpose-built functional unit tests. When I started integrating it in
+the different components of ARA.. there was some drawbacks. Me, as a user of
+the API, did not feel comfortable leaving things as is.
+
+The experiment was still worthwhile, though. It allowed me to "bootstrap" the
+API from end to end and have an endpoint represent each component. The big
+picture was there and we only needed to tweak things.
+
+Some problems were [performance](https://github.com/flask-restful/flask-restful/issues/612)
+related, others were around needing to do too many API calls to get one
+particular bit of information and so on.
+
+{{< tweet 927199009147678721 >}}
+
+I drew some inspiration from the [Ansible AWX](https://github.com/ansible/awx)
+API structure as well as from a Dropbox project which also uses Flask and
+Flask-Restful, [Changes](https://github.com/dropbox/changes).. and went back to
+the drawing board.
+
+Today, the API itself is mostly finished.
+
+The default callback, the [ara_record and ara_read modules](http://ara.readthedocs.io/en/latest/usage.html#using-the-ara-record-module)
+have been re-written to consume the API rather than doing direct SQL queries.
+
+The next step is to refactor the [command line interface](http://ara.readthedocs.io/en/latest/usage.html#querying-the-database-with-the-cli)
+as well as the web application interface do to the same thing.
+
+Using an API without compromising the user experience
+=====================================================
+
+Creating an API for an application that wasn't designed "API first" and then
+re-writing it to consume that API was fun and challenging.
+
+It was a learning experience and I'm pretty happy I was able to keep the result in line with the 
+[project core values](http://ara.readthedocs.io/en/latest/manifesto.html):
+
+- Simplicity is fundamental
+- Do one thing and do it well
+- Empower users to get their work done
+- Don’t require users to change their workflows
+- De-centralized, offline and standalone by default
+
+**De-centralized, offline and standalone by default** ? With an API ? Yes.
+
+I might end up explaining this implementation in it's own blog post eventually
+but through a clever use of [Flask](http://flask.pocoo.org/), the built-in API
+client in ARA can either communicate with an API endpoint over HTTP (REST) or
+offline, locally through internal Flask calls.
+
+**Don't require users to change their workflows**: This means ARA continues
+to ship without a requirement to run a daemon, a web server or an API server.
+
+``pip install ara``, set up the [Ansible callback](http://ara.readthedocs.io/en/latest/configuration.html#ansible),
+run your Ansible playbook as you are used to and that's it. That's what it has
+been since the project was created and I promise it will stay that way.
+
+**Simplicity is fundamental** is a very important one for me. Adding an API
+shouldn't make things complicated for the users... or the developers.
+
+In fact, most of the work in ARA 1.0 isn't user-facing. It's a re-write of the
+backend to move from the foundation the prototype had been built on more than
+a year ago. The API will allow for faster development, make maintenance
+easier and allow people to programmatically query ARA.
+
+Again, simplicity is a core theme here, even in how the built-in API client
+was designed to work.
+
+Here's some quick examples that leverages the client:
+
+```
+# Import the "Playbook" API client
+from ara.api.v1.playbooks import PlaybookApi
+
+# Get a list of all playbooks
+response, playbooks = PlaybookApi().get()
+
+# Get the details of a specific playbook
+response, playbook = PlaybookApi().get(id=1)
+
+# Create a new playbook in the database
+response, playbook = PlaybookApi().post(
+  ansible_version="2.4.1.0",
+  path="/home/user/ansible/playbooks/playbook.yml",
+  parameters={
+    "inventory": "/etc/ansible/inventory",
+    "become": True,
+    # ...
+  }
+
+# Import the "Result" API client
+from ara.api.v1.results import ResultApi
+
+# Get all failed results for a specific playbook
+response, results = ResultApi().get(playbook_id=1,
+                                    failed=True)
+```
+
+I mentioned earlier that the API had two ways of being exposed, one that was
+offline (internal, no network or http involved) and one online (HTTP REST).
+
+What you're not seeing in the background for the examples above is that no
+matter if we're relying on the internal API or the HTTP REST API, the code
+*and* the return values don't change.
+A new configuration parameter, ``ARA_API_CLIENT`` which defaults to ``offline``,
+can be set by users to ``online`` with a configurable API endpoint.
+
+When using ``offline`` as the API client, you're limited to posting and querying
+data on the host you are running Ansible from. This is perfect for most use
+cases while keeping latency and overhead to a minimum.
+
+When using ``online`` as the API client, you can host the web application and
+the API endpoint elsewhere and aggregate data from multiple machines running
+Ansible to a single location without having to share SQL database credentials.
+
+Saving more files in ARA
+========================
+
+ARA currently saves playbook and role task files. That's pretty cool, but
+what if we could also save group_vars, host_vars, defaults, meta, handlers
+and other files ?
+
+Consider it done in 1.0:
+
+![files](1.0-files.png)
+
+There will be a feature in order to selectively exclude files from being saved
+if you happen to store sensitive information in them and would rather leave
+those out.
+
+An improved database schema
+===========================
+
+The main reason for breaking backwards compatibility and not providing an
+upgrade path to 1.0 is the database schema.
+
+**Before 1.0**
+
+![database before](db-layout-0.x.png)
+
+**1.0 (currently)**
+
+![database after](db-layout-1.x.png)
+
+It would have been quite a burden to handle SQL migrations for every change
+that landed. Tables have been deleted, some fields have been renamed or even
+moved to other tables.
+
+Another issue is that the database model did not provide a
+[metadata description](http://docs.sqlalchemy.org/en/latest/core/metadata.html).
+
+This made it outright impossible to properly upgrade models on certain engines
+such as [sqlite](https://github.com/miguelgrinberg/Flask-Migrate/issues/61)
+because indexes and foreign keys may have been created without a name that we
+can refer back to.
+
+All in all, 1.0 is meant to be a clean slate with hopefully no "unfixable"
+mistakes from a database perspective. If you're an expert at database models
+and sqlalchemy, feel free to reach out, I'd love your input before this
+version ships !
+
+Still on the to-do list
+=======================
+
+There's been a lot of work on 1.0 already, but that doesn't mean we're done
+yet.
+
+![git network](1.0-diff.png)
+
+Here's a high level overview of the big items still on the to-do list:
+
+- **Input drivers**: The default callback will be folded back as an
+  input driver in a structure that will easily allow us to implement other means
+  of sending data to ARA. For example, we might want to send and process data
+  asynchronously through a message bus such as RabbitMQ, MQTT or a buffer like
+  Redis.
+
+- **Output drivers**: The current ways of exporting data from ARA like HTML,
+  junitXML and Subunit will be converted to output drivers in a generic
+  structure in order to make it straightforward to implement new ways of
+  exporting data. For example, this would allow us to create new drivers to
+  send task data from ARA to InfluxDB, Graphite or Logstash which could be a
+  nice addition to your Grafana or Kibana dashboards.
+
+- **Decoupling components**: Splitting ARA components into subpackages would
+  make it easier to install only what you need on the machines you want. It's
+  silly to install the web application dependencies if you only want to use
+  the callback. ``pip install ara`` should still reel everything in, but
+  if you need just the callback, for example, there should be a package that
+  does just that.
+
+- **Playbook labels**: These would allow users to label their playbooks in
+  order to display custom names in the web interface. For instance, you might
+  want your playbook to be named "Install MySQL" instead of
+  "/home/user/playbooks/playbook.yml". That's fair, let's do that.
+
+- **Playbook logical groups**: It would be useful to be able to group playbooks
+  together somehow. Pretend your deployment is really in fact 9 different
+  playbooks, you might want to group them together under a name like
+  "production deployment" so that you know you ran these together and look at
+  them as one logical report instead of 9 different reports. Or maybe those 9
+  playbooks are actually 3 for the development environment, 3 for the test
+  environment and 3 for the production -- you will be able to group them as
+  such.
+
+On the wishlist
+===============
+
+I don't want to delay the release of ARA 1.0 for too long, but there are some
+features I know would be very cool to add but are either complicated or would
+delay the release further.
+
+What's probably on the top of that list are persistent hosts. Ironically, the
+prototype of ARA more than a year ago shipped with persistent hosts. What are
+they ?
+
+Right now, Ansible does not provide a way to uniquely identify each host and
+this is a problem.
+
+In puppet, for example, there is the concept of certificate signature between
+a puppet agent and a puppet master. If a puppet agent shows up with the same
+hostname as an existing machine, the certificate won't match and there will be
+an error.
+
+In the early days of ARA, we ran with the assumption that hosts were unique
+across playbook runs. This means that if you had a server called "webserver"
+and you ran 10 playbooks with that host, we could tally the results for that
+particular host. You can see what it looked like in a very old video
+[demo](https://www.youtube.com/watch?v=k3qtgSFzAHI&index=2&list=PLyLLwe4-L1ETFVoAogQqpn6s5prGKL5Ty).
+
+The reason why we took that out is because your host, "webserver", can be
+one server today and a completely different server tomorrow: Ansible won't
+know the difference.
+This kind of assumption ended up causing problems and we decided it was wise
+to revisit this later.
+
+I have some ideas around how we could implement this for ARA but the
+implementation of persistent hosts is a lot of work that would have repercussions
+on the database schema, the backend, the API, the CLI, the web interface..
+everything.
+
+What's on your wishlist ?
+=========================
+
+Do you have any items on your wishlist for ARA ? Let me know !
+
+You can get in touch through [Twitter](https://twitter.com/dmsimard), [IRC
+and Slack](https://dmsimard.com/2017/08/27/new-ways-of-reaching-the-ara-community/)
+and you'll also find me lurking in [/r/ansible](https://www.reddit.com/r/ansible/).
+
+Thanks for reading and see you around !
+

website/content/blog/2018-02-23_rebranding-ansible-run-analysis-to-ara-records-ansible/index.md

@@ -0,0 +1,100 @@
+---
+author: "David Moreau Simard"
+categories:
+  - development
+tags:
+  - ansible
+  - openstack
+date: 2018-02-23
+title: "Rebranding Ansible Run Analysis to ARA Records Ansible"
+slug: rebranding-ansible-run-analysis-to-ara-records-ansible
+type: post
+---
+
+So I got an idea recently... Let's rebrand Ansible Run Analysis to ARA records
+Ansible.
+
+![review](gnu-is-not-unix.png)
+
+[![asciicast](https://asciinema.org/a/164917.png)](https://asciinema.org/a/164917)
+
+If you'd like to review and comment on the code change, you can do
+so here: [https://review.openstack.org/#/c/547245/](https://review.openstack.org/#/c/547245/).
+
+# Why ?
+
+I watched the last season of [Sillicon Valley](https://www.hbo.com/silicon-valley) recently.
+The series, while exaggerated, provides a humorous look at the world of startups.
+
+I don't have any plans on creating a startup but I love that it makes you think
+about things like needing a clever name or how you would do a proper "elevator"
+pitch to get funding.
+
+I'll take this opportunity to practice for fun.
+
+ARA Records Ansible is Another Recursive Acronym.
+
+{{< tweet 966896714274631685 >}}
+
+## The "elevator" pitch
+
+ARA Records Ansible is a project from the OpenStack community that makes it
+easier to understand and troubleshoot your Ansible roles and playbooks.
+
+- Do you run ``ansible-playbook -vv`` by default ?
+- 50 000 lines of console output to look at ?
+- How do you tell what changed ? Where ? Which task failed ?
+- What parameters did you use in that ansible-playbook command yesterday ?
+- What was the value of your host facts last week ?
+- What code or Ansible version did your playbook run with a month ago ?
+
+With ARA, you don't need to look at your 50 000 lines.
+ARA tells you everything about your entire playbook execution history through an
+intuitive self-hosted web interface, command line interface and soon, a full REST API.
+
+![ara reports](ara-reports.png)
+
+## The "we're stuck in the elevator" pitch
+
+ARA is a native [Ansible callback plugin](http://docs.ansible.com/ansible/devel/plugins/callback.html)
+that transparently save everything about your playbook executions.
+
+No matter if you're running Ansible from your laptop or from a server, this data
+ends up in a database -- offline to sqlite by default.
+If you prefer MySQL or PostgreSQL, that's cool too.
+
+ARA makes your playbook execution history database available through:
+
+- A web dashboard: Python [flask](http://flask.pocoo.org/) and [Patternfly](http://www.patternfly.org/) CSS (pretty!)
+- A CLI client: Python [cliff](https://github.com/openstack/cliff) -- same client interface as the "[openstack](https://docs.openstack.org/python-openstackclient/latest/)" command (json or yaml output, etc.!)
+- An API: Available in version 1.0 ([work in progress](https://dmsimard.com/categories/ara/)!)
+
+{{< tweet 920666123674075137 >}}
+
+# Learn more about the project
+
+One of ARA's core features and values is its simplicity.
+You can read about the other core values of ARA in the [project manifesto](https://ara.readthedocs.io/en/latest/manifesto.html).
+
+To learn more about ARA, you can find the [open source code on GitHub](https://github.com/openstack/ara) or
+watch this short video demo on YouTube:
+
+{{< youtube k3i8VPCanGo >}}
+
+# Chat with us
+
+If you want to chat with us, we're on IRC in the #ara channel on [Freenode](https://webchat.freenode.net/)
+and on [Slack](https://join.slack.com/t/ara-community/shared_invite/MjMxNzI4ODAxMDQxLTE1MDM4MDEzMTEtNzU1NTUwMTcyOQ).
+
+The two are bridged with [slack-irc](https://github.com/ekmartin/slack-irc) so
+everyone can talk together. It's pretty awesome.
+
+# Stay up to date
+
+If you'd like to stay up to date on what's coming in ARA 1.0, you can follow
+the project on [Twitter: @ARecordsAnsible](https://twitter.com/ARecordsAnsible)
+and [this blog](https://dmsimard.com/categories/ara/).
+
+Thanks for reading, feel free to let me know if you have any questions !
+
+- [@dmsimard](https://twitter.com/dmsimard)

website/content/blog/2018-04-09_scaling-ara-to-a-million-ansible-playbooks-a-month/index.md

@@ -0,0 +1,118 @@
+---
+author: "David Moreau Simard"
+categories:
+  - community
+tags:
+  - ansible
+  - openstack
+  - zuul
+date: 2018-04-09
+title: "Scaling ARA to a million Ansible playbooks a month"
+slug: scaling-ara-to-a-million-ansible-playbooks-a-month
+type: post
+---
+
+The [OpenStack](https://www.openstack.org/) community runs over 300 000 CI jobs
+with [Ansible](https://www.ansible.com/) every month with the help of the
+awesome [Zuul](https://zuul-ci.org/).
+
+![Zuul Pipelines](zuul-pipelines.png)
+
+It even provides ARA reports for ARA's [integration test jobs](https://github.com/openstack/ara#contributing-testing-issues-and-bugs)
+in a sort-of nested way. Zuul's Ansible ends up installing Ansible and ARA.
+It makes my brain hurt sometimes... but in an awesome way.
+
+![Zuul ARA Report](zuul-ci.png)
+
+As a core contributor of the infrastructure team there, I get to witness issues
+and get a lot of feedback directly from the users.
+
+[Static HTML report generation](https://ara.readthedocs.io/en/latest/usage.html#generating-a-static-html-version-of-the-web-application)
+in ARA is simple but didn't scale very well for us. One day, I was randomly
+chatting with Ian Wienand and he pointed out an
+[attempt](https://review.openstack.org/#/c/120317/) at a WSGI middleware that
+would serve extracted logs.
+
+That inspired me to write something similar but for dynamically loading ARA
+sqlite databases instead... This resulted in an awesome feature that I had not
+yet taken the time to explain very well... until now.
+
+*Excerpt from the [documentation](https://ara.readthedocs.io/en/latest/advanced.html#serving-ara-sqlite-databases-over-http)*
+
+> To put this use case into perspective, it was “benchmarked” against a single job from the [OpenStack-Ansible](https://github.com/openstack/openstack-ansible) project:
+> 
+> - 4 playbooks
+> - 4647 tasks
+> - 4760 results
+> - 53 hosts, of which 39 had gathered host facts
+> - 416 saved files
+>
+> Generating a static report from that database takes ~1min30s on an average machine.
+> The result contains 5321 files and 5243 directories for an aggregate size of 63MB (or 27MB recursively gzipped).
+>
+> This middleware allows you to host the exact same report on your web server just by storing the sqlite database which is just one file and weighs 5.6MB.
+> 
+
+This middleware can be useful if you're not interested in aggregating data in
+a central database server like MySQL or PostgreSQL.
+
+The OpenStack CI use case is decentralized: each of the >300 000 Zuul CI jobs
+have their own sqlite database uploaded as part of the log and artifact collection.
+
+There's a lot of benefits of doing things this way:
+
+- There's no network latency to a remote database server: the first bottleneck is your local disk speed.
+  - Even if it's a 5ms road trip, this adds up over hundreds of hosts and thousands of tasks.
+  - Oh, and contrary to popular belief, [sqlite is pretty damn fast](https://sqlite.org/speed.html).
+- There's no risk of a network interruption or central database server crash which would make ARA (and your sysadmins) panic.
+- Instead of one large database with lots of rows, you have more databases ("shards") with fewer rows.
+- Instead of generating thousands of files and directories, you're dealing with one small sqlite file.
+- There's no database cluster to maintain, just standard file servers with a web server in front.
+
+Another benefit is that you can easily have as many individual reports as
+you'd like, all you have to do is to configure ARA to use a custom database
+location.
+
+When I announced that we'd be switching to the sqlite middleware on
+[openstack-dev](http://lists.openstack.org/pipermail/openstack-dev/2018-March/128902.html),
+I mentioned that projects could leverage this within their jobs and
+OpenStack-Ansible was the first to take a stab at it:
+[https://review.openstack.org/#/c/557921/](https://review.openstack.org/#/c/557921/).
+
+Their job's logs now look like this:
+
+```
+ara-report/ansible.sqlite   # ARA report for this Zuul job
+logs/                       # Job's logs
+└── ara-report/             # ARA report for this OpenStack-Ansible deployment
+    └── ansible.sqlite      # Database for this OpenStack-Ansible deployment
+```
+
+The performance improvements for the OpenStack community at large are
+significant.
+
+Even if we're spending 1 minute generating and transferring thousands of HTML
+files... That's >300 000 minutes worth of compute that could be spent running
+other jobs.
+
+How expensive are 300 000 minutes (or 208 days!) of compute ?
+What about bandwidth and storage ?
+
+## Unfreezing ARA's stable release for development
+
+The latest version of ARA is currently 0.14.6 and ARA was more or less in
+feature-freeze mode while all the work was focused on the next major release,
+"[1.0](https://dmsimard.com/2017/11/22/status-update-ara-1.0/)".
+
+However, there is a growing amount of large scale users (me included!) that are
+really pushing the current limitations of ARA and 1.0 (or 2.0!) won't be ready
+for a while still.
+
+I couldn't afford to leave performance issues and memory leaks ruin the
+experience of a tool that would otherwise be very useful to them.
+
+These improvement opportunities have convinced me that there will be a 0.15.0
+release for ARA.
+
+Stay tuned for the 0.15.0 release notes and another update about 2.0 in the
+near future :)

website/content/blog/2018-05-03_ara-records-ansible-0.15-has-been-released/index.md

@@ -0,0 +1,90 @@
+---
+author: "David Moreau Simard"
+categories:
+  - development
+  - changelog
+tags:
+  - ansible
+  - openstack
+date: 2018-05-03
+title: "ARA Records Ansible 0.15 has been released"
+slug: ara-records-ansible-0.15-has-been-released
+type: post
+---
+
+I was recently writing that ARA was open to limited development for the stable
+release in order to improve the performance for larger scale users.
+
+This limited development is the result of this 0.15.0 release.
+
+{{< tweet 986675784478752770 >}}
+
+# Changelog for ARA Records Ansible 0.15.0
+
+The following are highlights from the 0.15.0 release.
+The full list of changes between 0.14.6 and 0.15.0 are available on [GitHub](https://github.com/openstack/ara/compare/0.14.6...0.15.0).
+
+- ARA: Ansible Run Analysis has been [rebranded](https://dmsimard.com/2018/02/23/rebranding-ansible-run-analysis-to-ara-records-ansible/)
+  to ARA Records Ansible (Another Recursive Acronym)
+- Significant improvements to memory usage and performance when running ARA as
+  a WSGI application with ``ara-wsgi`` or ``ara-wsgi-sqlite``.
+- Resolved an issue where the ``ara-wsgi-sqlite`` middleware could serve a
+  cached report instead of the requested one
+- Added support for configuring the ``SQLALCHEMY_POOL_SIZE``,
+  ``SQLALCHEMY_POOL_TIMEOUT`` and ``SQLALCHEMY_POOL_RECYCLE`` parameters.
+  See [configuration documentation](https://ara.readthedocs.io/en/latest/configuration.html#parameters-and-their-defaults) for more information.
+- Logging was fixed and improved to provide better insight when in DEBUG level.
+- Vastly improved the default [logging configuration](https://github.com/openstack/ara/blob/3c91da40871e50fa2854231d54f298ed996d1da6/ara/config/logger.py#L27-L78).
+  ARA will create a default logging configuration file in ``~/.ara/logging.yml`` that you can customize, if need be.
+  Deleting this file will make ARA create a new one with updated defaults.
+- Added python modules to help configure Ansible to use ARA, for example,
+  ``python -m ara.setup.callback_plugins`` will print the path to ARA's callback plugins.
+  You can find more examples in the [configuration documentation](https://ara.readthedocs.io/en/latest/configuration.html).
+- Implemented a workaround for fixing a race condition where an ``ansible-playbook`` command
+  may be interrupted after the playbook was recorded in the database but before playbook file was saved.
+- Flask 0.12.3 was [blacklisted](https://github.com/openstack/ara/commit/87272840bfc8b4c5db10593e47884e33a0f4af40)
+  from ARA's requirements, this was a broken release.
+- The ARA CLI can now be called with "python -m ara". This can be useful if you
+  need to specify a specific python interpreter, for example.
+- Updated and improved integration tests across different operating systems,
+  python2 and python3 with different versions of Ansible. The full test matrix
+  is available in the [README](https://github.com/openstack/ara#contributing-testing-issues-and-bugs).
+
+# Known issue: memory usage
+
+Please note that there is a known issue regarding high memory usage when
+browsing reports in the web application for **very large** reports.
+
+For the time being, we have put mitigations in place in order to prevent very
+high memory usage but this might be insufficient for large scale users.
+
+{{< tweet 984265830052651008 >}}
+
+If you'd like to help and contribute to resolve this, please reach out.
+
+# A disclaimer
+
+As with previous releases of the 0.x series, this new version comes with a 
+warning that we are not currently planning to provide backwards/forwards
+compatibility with the next major release of ARA, 1.0.
+
+In practice, this means that we are trying to focus all of our current
+contribution efforts towards making 1.0 as great as possible without burdening
+our limited development resources with keeping backwards and forwards
+compatibility.
+This means your existing databases will not be upgradable to the new version so
+you will need to start from a new database.
+
+One thing you don't have to worry about is that we are serious about keeping
+the project as simple as possible, in line with the project's [manifesto and core values](https://ara.readthedocs.io/en/latest/manifesto.html).
+ARA 1.0 will work the same way as it did before: ``pip install ara``, ``export ANSIBLE_CALLBACK_PLUGINS=$ara_location/plugins/callbacks`` and you're good to go.
+
+1.0 will provide additional features such as a revamped database, a backend
+rewritten from scratch, an improved frontend and a full REST API.
+
+You can look at some of the work we have been doing for this new version [here](https://dmsimard.com/categories/ara/).
+
+There are a lot exciting opportunities where you can contribute to the development
+of 1.0, please reach out to me (dmsimard) if you're interested in helping !
+
+You can come chat with us on [IRC or on Slack](https://github.com/openstack/ara#contributing-testing-issues-and-bugs).

website/content/blog/2018-10-08_ansiblefest-2018-community-project-highlights/index.md

@@ -0,0 +1,120 @@
+---
+author: "David Moreau Simard"
+categories:
+  - community
+tags:
+  - ansible
+  - ansiblefest
+  - openstack
+  - zuul
+  - molecule
+  - ansible-lint
+date: 2018-10-08
+title: "AnsibleFest 2018: Community project highlights"
+slug: ansiblefest-2018-community-project-highlights
+type: post
+---
+
+With two days of [AnsibleFest](https://www.ansible.com/ansiblefest) instead of
+one this time around, we had 100% more time to talk about Ansible things !
+
+I got to attend great sessions, learn a bunch of things, chat and exchange
+war stories about Ansible, ARA, Zuul, Tower and many other things.
+
+It was awesome and I wanted to take the time to share a bit about some of the
+great Ansible community projects that were featured during the event.
+
+At one point during the keynotes, the Ansible community lead, [Robyn Bergeron](https://twitter.com/robynbergeron),
+talked about the community around Ansible with this slide:
+
+![Ansible Community Projects](community-projects-ansiblefest.jpg)
+
+It's possible this was the first time you were hearing about these projects.
+Let me tell you about how awesome they are.
+
+## ansible-lint and molecule
+
+Before AnsibleFest, Ansible announced their intention to [adopt the ansible-lint and Molecule projects](https://www.reddit.com/r/ansible/comments/9j4de4/ansible_to_adopt_molecule_and_ansiblelint_projects/)
+as part of a commitment to improve [Ansible Galaxy](https://galaxy.ansible.com/).
+
+{{< tweet 1047535123254665216 >}}
+
+[ansible-lint](https://github.com/willthames/ansible-lint) helps you write consistent Ansible code for your roles and playbooks while adhering to best practices.
+I think the first project I really started using ansible-lint on was on WeIRDO, a CI framework, as far back as [2015](https://github.com/rdo-infra/weirdo/commit/839b8a2e022b6a6f675dc9dcd40b6cf334a60741#diff-b91f3d5bd63fcd17221b267e851608e8) !
+
+It's simple and it just works:
+
+```
+$ ansible-lint examples/example.yml
+[ANSIBLE0004] Git checkouts must contain explicit version
+examples/example.yml:15
+Task/Handler: git check
+
+[ANSIBLE0002] Trailing whitespace
+examples/example.yml:35
+    with_items: 
+
+[ANSIBLE0006] git used in place of git module
+examples/example.yml:24
+Task/Handler: executing git through command
+```
+
+[Molecule](https://github.com/metacloud/molecule) provides a simple framework for easily and repetedly testing your roles and playbooks against different environments and operating systems.
+There was a great talk last year about it by Elana Hashman from Rackspace: [Infrastructure Testing with Molecule](https://www.ansible.com/infrastructure-testing-with-molecule).
+
+I haven't had the chance to try Molecule yet but here's what it looks like in practice on [Asciinema](https://asciinema.org/a/161977?speed=2&autoplay=1&loop=1):
+
+<script src="https://asciinema.org/a/161977.js" id="asciicast-161977" async></script>
+
+## Zuul
+
+[Zuul](https://zuul-ci.org/) drives continuous integration, delivery, and deployment systems with a focus on project gating and interrelated projects.
+
+The recordings aren't available yet but there were two sessions about Zuul at AnsibleFest last week:
+
+- [Using Zuul CI/CD system to test Ansible Networking content](https://agenda.fest.ansible.com/SessionDetail.aspx?id=482140) by [Paul Belanger](https://twitter.com/pabelanger) and [Ricardo Carrillo Cruz](https://twitter.com/rcarrillocruz) from Red Hat
+- [The Build is Never Broken](https://agenda.fest.ansible.com/SessionDetail.aspx?id=482019) by [Clint Byrum](https://twitter.com/spamaps) from GoDaddy
+
+Zuul eventually replaced Jenkins for the [OpenStack](https://www.openstack.org/) community and if you're curious why, you can read more about that story in a short interview on the [SuperUser blog](http://superuser.openstack.org/articles/zuul-case-study-the-openstack-foundation/).
+
+Zuul is awesome and let me try to give you a (long) elevator pitch of some of my favorite Zuul features:
+
+- By design, Zuul is meant to gate patches against your projects. Zuul will not merge a patch to your project until all the required jobs are green.
+- Your jobs in Zuul are written as Ansible roles and playbooks, letting you re-use your existing deployment code and leverage thousands of built-in Ansible modules and plugins natively out of the box.
+- Zuul takes care of providing you with an Ansible inventory for your jobs by abstracting the logic of creating and destroying single-use virtualized or containerized environments.
+- Every project managed by Zuul is independant and can manage their own jobs, inventories and pipeline configuration in tree.
+- When evaluating a change or a pull request, Zuul will speculatively load and run the new configuration coming from the change. This means you can create a new job and iterate on it until it works without even merging any code or commits.
+- Horizontally scalable for the vast majority of it's workloads, Zuul is highly available and is designed to consume multiple clouds and regions concurrently with the expectation that they will eventually fail.
+- Zuul provides an implementation of [ARA Records Ansible](https://github.com/openstack/ara) for providing host and task granular results and metrics through an intuitive reporting web interface.
+
+{{< tweet 1032799030131142657 >}}
+
+If you'd like to learn more about how [Zuul](https://zuul-ci.org/) works and what it does, I like to recommend the following two presentations by core developers of the project:
+
+- [Zuul: Testing the future](https://www.youtube.com/watch?v=KXh0sh3ETkQ) by James Blair
+- [Zuul v3 for gating](https://www.youtube.com/watch?v=6177329H4Tg) by Monty Taylor
+
+You can chat with the Zuul users and developers on [IRC or on mailing lists](https://zuul-ci.org/community.html).
+
+## ARA Records Ansible
+
+[ARA Records Ansible](https://getara.org/) playbook runs and makes the recorded data available and intuitive for users and systems.
+
+I created ARA a little over two years ago to make it easy for Ansible users to understand and troubleshoot what occurred during their playbook runs.
+
+In a nutshell, ARA provides a native [Ansible callback plugin](https://docs.ansible.com/ansible/latest/plugins/callback.html) which saves everything to a local or remote database.
+It then provides a command line interface as well as a web interface to query that database and provide detailed metrics and reporting for your Ansible playbooks.
+
+ARA is notably implemented in Zuul and is used for reporting on more than [a million playbooks every month](http://superuser.openstack.org/articles/scaling-ara-ansible/) for the OpenStack community.
+You can find the ARA source code mirrored on [GitHub](https://github.com/openstack?q=ara).
+
+If you've never seen an ARA report before, I've recorded a live demonstration of the web interface a while back which is still accurate:
+
+{{< youtube k3i8VPCanGo >}}
+<br>
+
+The upcoming major release, ARA 1.0, is all but a complete rewrite and has been more than a year in the making.
+1.0 will feature a brand new REST API which will let users query and integrate their Ansible execution data in their tools and workflows.
+
+The to-do list for 1.0 is shrinking and I'm really excited that we're getting to a point where people can start using it for testing purposes.
+If you'd like to test 1.0 or contribute to ARA, please reach out to me on [IRC, Slack](https://github.com/openstack/ara#community-and-getting-help) or [Twitter](https://twitter.com/dmsimard)!