BFS V0.441 vs CFS in Linux 3.11.0

Purpose: Compare CPU schedulers in linux 3.11.0.
Test machine: Intel Ivybridge (i7-3770K @ 4.50 GHz).
Software:

• Arch Linux x86_64.
• Official gcc and gcc-libs packages (4.8.1-3) used to both compile the kernel packages and to run the benchmarks.
• Kernel packages differ only in the ck1 patch used; all other config files held constant
• The config file is the stock Arch Linux config.x86_64 for the 3.11-1 package.

Benchmark: This shell script was used to drive each benchmark which iterated over a compile job a total of 18 times. Benchmark endpoints are simply the time to compile `make -j9 bzImage` on a preconfigured linux kernel and `make -j9` on a preconfigued samba v3 source. Times are reported in seconds and analyzed in an ANOVA plot used to establish differences in median values and whether or not the distribution achieved statistically significance.

Results: BFS was superior to CFS on this machine; results true for both make endpoints.

Median speed gain for building bzImage with bfs v0.441: 1,452 ms
Median speed gain for building samba with bfs v0.441: 600 ms



Data/Analysis:
Results for bzImage

  

Results for samba

  

Note for those of you really paying attention: Both benchmarks used `make -j9 ...` even though the 3770k is a quad (4 physical + 4 virtual=8).  I am aware that it is recommended NOT to use the x+1 formula for kernels running the BFS but felt that in order to fairly compare both schedulers, this needed to be held constant.  That said, I have done some experiments where I varied the make flags (8, 9, 10) and found that there was no statistically significant difference on the BFS.

BFS V0.440 vs CFS in Linux 3.10.0

Purpose: Compare CPU schedulers in linux 3.10.0.
Test machine: Intel Ivybridge (i7-3770K @ 4.50 GHz).
Software:

• Arch Linux x86_64.
• Official gcc and gcc-libs packages (4.8.1-1) used to both compile the kernel packages and to run the benchmarks.
• Kernel packages differ only in the ck1 patch used; all other config files held constant
• The config file is the stock Arch Linux config.x86_64 for the 3.10-1 package.

Benchmark: This shell script was used to drive each benchmark which iterated over a compile job a total of 9 times. Benchmark endpoints are simply the time to compile `make -j9 bzImage` on a preconfigured linux kernel. Times are reported in seconds and analyzed in an ANOVA plot used to establish differences in median values and whether or not the distribution achieved statistically significance.

Result: BFS was superior to CFS on this machine.

Data/Analysis:

Note for those of you really paying attention: Both benchmarks used `make -j9 ...` even though the 3770k is a quad (4 physical + 4 virtual=8).  I am aware that it is recommended NOT to use the x+1 formula for kernels running the BFS but felt that in order to fairly compare both schedulers, this needed to be held constant.  That said, I have done some experiments where I varied the make flags (8, 9, 10) and found that there was no statistically significant difference on the BFS.

BFS v0.430 vs CFS in Linux 3.9.0

Purpose: Compare CPU schedulers in linux 3.9.0.
Test machine: Intel Ivybridge (i7-3770K @ 4.50 GHz).
Software:

• Arch Linux x86_64.
• Official gcc and gcc-libs packages (4.8.0-3) used to both compile the kernel packages and to run the benchmarks.
• Kernel packages differ only in the ck1 patch used; all other config files held constant
• The config file is the stock Arch Linux config.x86_64 for the 3.9-2 package.

Benchmark: This shell script was used to drive each benchmark which iterated over a compile job a total of 18 times. Benchmark endpoints are simply the time to compile `make -j9 bzImage` on a preconfigured linux kernel and `make -j9` on a preconfigued samba v3 source. Times are reported in seconds and analyzed in an ANOVA plot used to establish differences in median values and whether or not the distribution achieved statistically significance.

Result: BFS was superior to CFS on this machine; results true for both make endpoints.

Data:
Pink = CFS and blue = ck1 patched kernel (BFS v0.430):

Results for bzImage:

Results for samba:

--EDITED on 08-May-2013 @ 3:43 PM --

Note for those of you really paying attention: Both benchmarks used `make -j9 ...` even though the 3770k is a quad (4 physical + 4 virtual=8).  I am aware that it is recommended NOT to use the x+1 formula for kernels running the BFS but felt that in order to fairly compare both schedulers, this needed to be held constant.  That said, I have done some experiments where I varied the make flags (8, 9, 10) and found that there was no statistically significant difference on the BFS.