Python optimizations for CentOs

Pros:
 we are seeing a 10% performance boost for all python processes.

Cons:
 The time to build python increases.
 This is because pgo (profile guided optimizations) need the unit tests
for python to be invoked and then the entire package compiled a second
time using the timing information from the unit test profiling data
that is gathered.

This commit sets up the src rpm environment for building python 2.7.5

- Needed to disable the tox unit test check phase since a couple tests
 fail due to build dependencies
- Turn off dtrace and systemtap in the spec file
- Use pgo for build in modules: https://bugs.python.org/issue26307
- Rollup cpython patches and add --enable-optimizations and
  --with-computed-gotos for optimized build

These are the cpython commits pulled in:
   4ff151a72a3331c514924a5e0d33747a91413fbd  (Sept 18, 2015 pgo)
   c543a0fc3b2052c35fced09e1f96613625723508 (Jun 2 2016) (lto)
   6d8fdfc84b8617476f3f497e93e9e7b6cecc5474  (Sept 8 2016)
   794b2912acc1995afc6fe9501cc0ca8e56565325 (Sept 8 2016)
   b9999158545f3bc089b926086daa666b8e4c7caa (Nov 20) 2016
   6a04ef7ceddae0930eba6cc57ba2ebfcef00abab  Mar 28 2017

This still does not yield the same level of performance as a vanilla
 python 2.7.14 source tree which suggests there are other commits
 that would provide a performance boost.

Change-Id: Iaddf7ead672412ad1878217ddcfaf88c9b0e2443
Signed-off-by: Al Bailey <Al.Bailey@windriver.com>
This commit is contained in:
Al Bailey 2018-04-19 08:45:12 -05:00
parent 5a9fcd20e1
commit 45d4657f3d
9 changed files with 3675 additions and 0 deletions

View File

@ -0,0 +1,3 @@
The src rpm version indicates that we are patching python 2.7.5, however there are at least 58 patches
added by Centos to fix issues fixed in the later python 2.7 releases.

View File

@ -0,0 +1 @@
TIS_PATCH_VER=3

View File

@ -0,0 +1,25 @@
From e9bc5d92bb0ee327015626072fee83f12018712c Mon Sep 17 00:00:00 2001
From: Al Bailey <Al.Bailey@windriver.com>
Date: Tue, 27 Mar 2018 12:38:50 -0500
Subject: [PATCH] Update package versioning for TIS format
---
SPECS/python.spec | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/SPECS/python.spec b/SPECS/python.spec
index a9fd32f..62f21d4 100644
--- a/SPECS/python.spec
+++ b/SPECS/python.spec
@@ -106,7 +106,7 @@ Summary: An interpreted, interactive, object-oriented programming language
Name: %{python}
# Remember to also rebase python-docs when changing this:
Version: 2.7.5
-Release: 58%{?dist}
+Release: 58.el7%{?_tis_dist}.%{tis_patch_ver}
License: Python
Group: Development/Languages
Requires: %{python}-libs%{?_isa} = %{version}-%{release}
--
1.8.3.1

View File

@ -0,0 +1,25 @@
From 2f23c2c6194d828de7bede593926007802cca610 Mon Sep 17 00:00:00 2001
From: Al Bailey <Al.Bailey@windriver.com>
Date: Tue, 6 Mar 2018 18:21:48 -0600
Subject: [PATCH] Disable unittest check phase
---
SPECS/python.spec | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/SPECS/python.spec b/SPECS/python.spec
index a9fd32f..1030c57 100644
--- a/SPECS/python.spec
+++ b/SPECS/python.spec
@@ -54,7 +54,7 @@
%global with_gdbm 1
# Turn this to 0 to turn off the "check" phase:
-%global run_selftest_suite 1
+%global run_selftest_suite 0
# Some of the files below /usr/lib/pythonMAJOR.MINOR/test (e.g. bad_coding.py)
# are deliberately invalid, leading to SyntaxError exceptions if they get
--
1.8.3.1

View File

@ -0,0 +1,42 @@
From ff1efc0b077c78bedead5fdc6792240806f5a97c Mon Sep 17 00:00:00 2001
From: Al Bailey <Al.Bailey@windriver.com>
Date: Wed, 28 Mar 2018 08:26:56 -0500
Subject: [PATCH] Make Python Faster
---
SPECS/python.spec | 5 ++++-
1 file changed, 4 insertions(+), 1 deletion(-)
diff --git a/SPECS/python.spec b/SPECS/python.spec
index adc6815..fadf838 100644
--- a/SPECS/python.spec
+++ b/SPECS/python.spec
@@ -1195,6 +1195,8 @@ Patch268: 00268-set-stream-name-to-None.patch
# above:
Patch5000: 05000-autotool-intermediates.patch
+Patch9000: Rollup-optimization-commits.patch
+
# ======================================================
# Additional metadata, and subpackages
# ======================================================
@@ -1591,6 +1593,7 @@ find -name "*~" |xargs rm -f
%patch5000 -p0 -b .autotool-intermediates
%endif
+%patch9000 -p1
# ======================================================
# Configuring and building the code:
@@ -1721,7 +1724,7 @@ BuildPython debug \
BuildPython optimized \
python \
python%{pybasever} \
- "" \
+ "--enable-optimizations --with-computed-gotos" \
true
--
1.8.3.1

View File

@ -0,0 +1,25 @@
From 83b8eb17d13fd5afdb926698082e47cba029c531 Mon Sep 17 00:00:00 2001
From: Al Bailey <Al.Bailey@windriver.com>
Date: Wed, 28 Mar 2018 15:23:35 -0500
Subject: [PATCH] Turn off with_systemtap to disable dtrace
---
SPECS/python.spec | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/SPECS/python.spec b/SPECS/python.spec
index 2441951..c2edce8 100644
--- a/SPECS/python.spec
+++ b/SPECS/python.spec
@@ -42,7 +42,7 @@
%global with_gdb_hooks 1
-%global with_systemtap 1
+%global with_systemtap 0
# some arches don't have valgrind so we need to disable its support on them
%ifarch %{ix86} x86_64 ppc %{power64} s390x aarch64
--
1.8.3.1

View File

@ -0,0 +1,4 @@
0001-Update-package-versioning-for-TIS-format.patch
0002-Disable-unittest-check-phase.patch
0003-Make-Python-Faster.patch
0004-Turn-off-with_systemtap-to-disable-dtrace.patch

File diff suppressed because it is too large Load Diff

View File

@ -0,0 +1,2 @@
mirror:Source/python-2.7.5-58.el7.src.rpm