diff --git a/ansible/check/README.md b/ansible/check/README.md
deleted file mode 100644
index 537e2e194..000000000
--- a/ansible/check/README.md
+++ /dev/null
@@ -1,21 +0,0 @@
-# Browbeat Ansible Checks
-The Browbeat Ansible checks playbook will
-scan a given deployment to check for known
-configuration issues.
-
-## Output from Checks
-Once the Checks playbook completes there will
-be a report generated in the checks directory.
-
-```
-bug_report.log
-```
-
-This will contain each of the hosts that were
-scanned, and their known issues. Some of the
-issues will have BZ's associated with them, use
-the BZ link to learn more about the bug, and how
-it can impact your deployment. There will be
-Checks that do not have a BZ associated, these
-are simply suggestions.
-
diff --git a/ansible/check/README.rst b/ansible/check/README.rst
new file mode 100644
index 000000000..1f3a73ab1
--- /dev/null
+++ b/ansible/check/README.rst
@@ -0,0 +1,21 @@
+Browbeat Ansible Checks
+=======================
+
+The Browbeat Ansible checks playbook will scan a given deployment to
+check for known configuration issues.
+
+Output from Checks
+------------------
+
+Once the Checks playbook completes there will be a report generated in
+the checks directory.
+
+::
+
+    bug_report.log
+
+This will contain each of the hosts that were scanned, and their known
+issues. Some of the issues will have BZ's associated with them, use the
+BZ link to learn more about the bug, and how it can impact your
+deployment. There will be Checks that do not have a BZ associated, these
+are simply suggestions.
diff --git a/ansible/tune/README.md b/ansible/tune/README.rst
similarity index 58%
rename from ansible/tune/README.md
rename to ansible/tune/README.rst
index ba0f10518..13415d00b 100644
--- a/ansible/tune/README.md
+++ b/ansible/tune/README.rst
@@ -1,4 +1,6 @@
-# Browbeat OSP Performance Tuning Playbook
+Browbeat OSP Performance Tuning Playbook
+========================================
+
 This playbook aims to tune OSP deployed on Red Hat Enterprise Linux.
 
 The playbook in here is currently experimental.
diff --git a/rally/authenticate/README.md b/rally/authenticate/README.md
deleted file mode 100644
index 9f0f962f7..000000000
--- a/rally/authenticate/README.md
+++ /dev/null
@@ -1 +0,0 @@
-# Authenticate scenarios
diff --git a/rally/authenticate/README.rst b/rally/authenticate/README.rst
new file mode 100644
index 000000000..a7ce163c5
--- /dev/null
+++ b/rally/authenticate/README.rst
@@ -0,0 +1,2 @@
+Authenticate scenarios
+======================
diff --git a/rally/cinder/README.md b/rally/cinder/README.md
deleted file mode 100644
index 6f73d9b0d..000000000
--- a/rally/cinder/README.md
+++ /dev/null
@@ -1 +0,0 @@
-# Cinder scenarios
diff --git a/rally/cinder/README.rst b/rally/cinder/README.rst
new file mode 100644
index 000000000..a0f8a994d
--- /dev/null
+++ b/rally/cinder/README.rst
@@ -0,0 +1,2 @@
+Cinder scenarios
+================
diff --git a/rally/keystonebasic/README.md b/rally/keystonebasic/README.md
deleted file mode 100644
index 0d250a01f..000000000
--- a/rally/keystonebasic/README.md
+++ /dev/null
@@ -1 +0,0 @@
-# KeystoneBasic scenarios
diff --git a/rally/keystonebasic/README.rst b/rally/keystonebasic/README.rst
new file mode 100644
index 000000000..d5d9d1c8c
--- /dev/null
+++ b/rally/keystonebasic/README.rst
@@ -0,0 +1,2 @@
+KeystoneBasic scenarios
+=======================
diff --git a/rally/neutron/README.md b/rally/neutron/README.md
deleted file mode 100644
index 1224fd7a5..000000000
--- a/rally/neutron/README.md
+++ /dev/null
@@ -1 +0,0 @@
-# Neutron scenarios
diff --git a/rally/neutron/README.rst b/rally/neutron/README.rst
new file mode 100644
index 000000000..499155ab7
--- /dev/null
+++ b/rally/neutron/README.rst
@@ -0,0 +1,2 @@
+Neutron scenarios
+=================
diff --git a/rally/nova/README.md b/rally/nova/README.md
deleted file mode 100644
index 63ef51c46..000000000
--- a/rally/nova/README.md
+++ /dev/null
@@ -1 +0,0 @@
-# Nova scenarios
diff --git a/rally/nova/README.rst b/rally/nova/README.rst
new file mode 100644
index 000000000..bb52f3e10
--- /dev/null
+++ b/rally/nova/README.rst
@@ -0,0 +1,2 @@
+Nova scenarios
+==============
diff --git a/rally/rally-plugins/README.md b/rally/rally-plugins/README.md
deleted file mode 100644
index c209d52b7..000000000
--- a/rally/rally-plugins/README.md
+++ /dev/null
@@ -1,5 +0,0 @@
-# Rally Plugins Browbeat can use
-## Current plugins
-- netcreate-boot-ping ( not complete )
-- netcreate-boot
-- subnet-router-create
diff --git a/rally/rally-plugins/README.rst b/rally/rally-plugins/README.rst
new file mode 100644
index 000000000..a6337d170
--- /dev/null
+++ b/rally/rally-plugins/README.rst
@@ -0,0 +1,9 @@
+Rally Plugins Browbeat can use
+==============================
+
+Current plugins
+---------------
+
+-  netcreate-boot-ping ( not complete )
+-  netcreate-boot
+-  subnet-router-create