Add "daily" option to bugdaystats.py

Recently, we are triaging bugs of Tempest on the launchpad and that
work was fine to clean up old bugs and fixed bugs with different
patches.
Current bugdaystats shows it in last several days statically, and it
would be nice to add more long terms like in 30 days or a release to see
bug triage trends.

This patch adds a "daily" option to bugdaystats.py for making daily data
json files. The file can be used to show long term data in other graphs.

Change-Id: Ieac396389ce13468b717d03264164c77f7c36104

README.rst

@@ -19,7 +19,11 @@ python bugdaystats.py output
 'output' is the name of the directory you will generate data
 and HTML files to (if they don't exist yet). It should contain a
 'js' subdirectory containing JavaScript include files, but
-otherwise be empty.
+otherwise be empty. In addition, it should contain two types of
+data files. One file contains data in each time when running the
+script, and another one is created daily. That means the creation
+is skipped if the previous data is in the same day. The data file
+is useful for showing long-term bug situation.
 
 You'll need to run the script at least twice to generate enough
 stats to get a graph.

bugdaystats.py

@@ -41,20 +41,29 @@ def create_files(templatepath, outputpath, projects):
             template.stream(project=project).dump(projectfile)
 
 
-def update_stats(outputpath, project_name, rotation):
+def update_stats(outputpath, project_name, rotation, daily=False):
 
     now = int(time.time())
     records = []
     counts = {}
     project = launchpad.projects[project_name]
-    project_stats_filename = os.path.join(outputpath,
-                                          "%s-bug-stats.json" % (project_name))
+    if daily:
+        filename = "%s-bug-stats-daily.json" % project_name
+    else:
+        filename = "%s-bug-stats.json" % project_name
+    project_stats_filename = os.path.join(outputpath, filename)
 
     try:
         data_file = open(project_stats_filename, 'r')
         json_data = json.load(data_file)
         data_file.close()
         for record in json_data['records']:
+            if daily:
+                if (now - record['date']) < (24 * 60 * 60):
+                    # Skip to update stats if the recode contains the same
+                    # day data.
+                    return
+
             if rotation:
                 if (now - record['date']) > (rotation * 24 * 60 * 60):
                     continue
@@ -159,6 +168,8 @@ if __name__ == '__main__':
         config = json.load(configfile)
     projects = config['projects']
     rotation = config.get('rotation')
+    daily = config.get('daily')
+    daily_rotation = config.get('daily_rotation')
     openstack_status = config.get('openstack_status')
 
     # Create files in output directory, if needed
@@ -170,3 +181,6 @@ if __name__ == '__main__':
 
     for p in projects:
         update_stats(outputpath, p['project'], rotation)
+
+        if (daily):
+            update_stats(outputpath, p['project'], daily_rotation, daily=True)

config.js.sample

@@ -10,5 +10,7 @@
         { "project": "openstack-manuals", "title": "Manuals" },
         { "project": "tempest" }
     ],
-    "rotation": 2
+    "rotation": 2,
+    "daily": true,
+    "daily_rotation": 183
 }