Result Overview
- Number of Run Configurations
- 95
- Change of Run time
- median 1% (min. -15%, max. 179%)
Changes in Benchmark Set
| Commit | Executor | Suite | Benchmark | Variable Values | Cores | Input Size | Extra Arguments |
|---|---|---|---|---|---|---|---|
| affb91 | SOM-RS-bc-int | somrs-interp | CD | 1 | 200 | ||
| affb91 | SOM-RS-bc-int | somrs-interp | DeltaBlue | 1 | 12000 | ||
| affb91 | SOM-RS-bc-int | somrs-interp | Havlak | 1 | 150 | ||
| show more | |||||||
Performance Changes between Versions
som-interp
Executor: PySOM-ast-int
| #M | median time in ms |
time diff % | |||
|---|---|---|---|---|---|
| Bounce |  |
15 | 4699.11 | 2 | |
| CD |  |
15 | 7128.99 | -1 | |
| DeltaBlue |  |
15 | 1360.18 | 1 | |
| Havlak |  |
15 | 7522.28 | 2 | |
| Json |  |
15 | 2014.62 | -1 | |
| List |  |
15 | 4439.97 | -3 | |
| Mandelbrot |  |
15 | 4395.70 | 0 | |
| NBody |  |
15 | 5805.45 | -1 | |
| Permute |  |
15 | 4259.12 | -1 | |
| Queens |  |
15 | 4411.87 | -1 | |
| Richards |  |
15 | 4432.20 | 3 | |
| Sieve |  |
15 | 7577.52 | 4 | |
| Storage |  |
15 | 1251.48 | -1 | |
| Towers |  |
15 | 4717.53 | 1 |
som-interp
Executor: PySOM-bc-int
| #M | median time in ms |
time diff % | |||
|---|---|---|---|---|---|
| Bounce |  |
15 | 4520.38 | 2 | |
| CD |  |
15 | 6785.85 | -3 | |
| DeltaBlue |  |
15 | 1417.55 | 0 | |
| Havlak |  |
15 | 7611.18 | 2 | |
| Json |  |
15 | 2001.78 | -1 | |
| List |  |
15 | 3965.85 | 0 | |
| Mandelbrot |  |
15 | 3549.89 | -1 | |
| NBody |  |
15 | 6807.49 | 1 | |
| Permute |  |
15 | 4110.39 | 0 | |
| Queens |  |
15 | 5440.97 | -2 | |
| Richards |  |
15 | 4517.85 | 0 | |
| Sieve |  |
15 | 7165.55 | -1 | |
| Storage |  |
15 | 1314.73 | 0 | |
| Towers |  |
15 | 4993.91 | -1 |
somrs-interp
Executor: SOM-RS-ast-int
| #M | median time in ms |
time diff % | |||
|---|---|---|---|---|---|
| Bounce |  |
1 | 27568.00 | 77 | |
| CD |  |
1 | 28153.27 | 81 | |
| DeltaBlue |  |
1 | 4746.00 | 71 | |
| Havlak |  |
1 | 27394.82 | 86 | |
| Json |  |
1 | 15800.33 | 27 | |
| List |  |
1 | 18942.44 | 104 | |
| Mandelbrot |  |
1 | 16278.45 | 72 | |
| NBody |  |
1 | 18276.06 | 93 | |
| Permute |  |
1 | 16855.98 | 88 | |
| Queens |  |
1 | 22392.19 | 77 | |
| Richards |  |
1 | 16688.75 | 91 | |
| Sieve |  |
1 | 40689.69 | 86 | |
| Storage |  |
1 | 6728.21 | 70 | |
| Towers |  |
1 | 17359.71 | 99 |
somrs-interp
Executor: SOM-RS-bc-int
| #M | median time in ms |
time diff % | ||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Bounce |  |
1 | 27195.07 | 120 | ||||||||||||||||
| CD | No matching configuration for affb91df35f834dbc67c6e35fb71ce11ee41c965: total |
|||||||||||||||||||
| DeltaBlue | No matching configuration for affb91df35f834dbc67c6e35fb71ce11ee41c965: total |
|||||||||||||||||||
| Havlak | No matching configuration for affb91df35f834dbc67c6e35fb71ce11ee41c965: total |
|||||||||||||||||||
| Json |  |
1 | 14871.54 | 32 | ||||||||||||||||
| List |  |
1 | 28681.32 | 73 | ||||||||||||||||
| Mandelbrot |  |
1 | 15226.31 | 144 | ||||||||||||||||
| NBody |  |
1 | 27927.94 | 157 | ||||||||||||||||
| Permute |  |
1 | 16220.76 | 49 | ||||||||||||||||
| Queens |  |
1 | 22650.37 | 135 | ||||||||||||||||
| Richards |  |
1 | 19518.76 | 119 | ||||||||||||||||
| Sieve |  |
1 | 35480.44 | 179 | ||||||||||||||||
| Storage |  |
1 | 5355.76 | 100 | ||||||||||||||||
| Towers |  |
1 | 20161.11 | 122 | ||||||||||||||||
sompp-interp
Executor: SOMpp-int
| #M | median time in ms |
time diff % | |||
|---|---|---|---|---|---|
| Bounce |  |
15 | 3571.28 | 0 | |
| CD |  |
15 | 5055.16 | 1 | |
| DeltaBlue |  |
15 | 894.51 | -4 | |
| Havlak |  |
15 | 4792.85 | -2 | |
| Json |  |
15 | 1893.97 | 0 | |
| List |  |
15 | 3538.25 | 1 | |
| Mandelbrot |  |
15 | 3668.38 | 1 | |
| NBody |  |
15 | 4612.41 | -3 | |
| Permute |  |
15 | 3632.96 | 1 | |
| Queens |  |
15 | 4632.49 | -5 | |
| Richards |  |
15 | 3018.24 | -1 | |
| Sieve |  |
15 | 5858.51 | -3 | |
| Storage |  |
15 | 1305.21 | -1 | |
| Towers |  |
15 | 3313.99 | 0 |
som-interp
Executor: TruffleSOM-ast-NativeEE-int-main
| #M | median time in ms |
time diff % | |||
|---|---|---|---|---|---|
| Bounce |  |
15 | 4279.90 | -4 | |
| CD |  |
15 | 9144.78 | 2 | |
| DeltaBlue |  |
15 | 1921.08 | 1 | |
| Havlak |  |
15 | 9928.47 | 0 | |
| Json |  |
15 | 2625.36 | -2 | |
| List |  |
15 | 6575.65 | -1 | |
| Mandelbrot |  |
15 | 1844.75 | 0 | |
| NBody |  |
15 | 4872.43 | -6 | |
| Permute |  |
15 | 5392.13 | 4 | |
| Queens |  |
15 | 3933.83 | 0 | |
| Richards |  |
15 | 6232.40 | -1 | |
| Sieve |  |
15 | 3242.46 | -11 | |
| Storage |  |
15 | 1596.00 | 3 | |
| Towers |  |
15 | 6442.92 | -5 |
som-interp
Executor: TruffleSOM-bc-NativeEE-int-main
| #M | median time in ms |
time diff % | |||
|---|---|---|---|---|---|
| Bounce |  |
15 | 6201.95 | -7 | |
| CD |  |
15 | 10069.86 | 9 | |
| DeltaBlue |  |
15 | 2413.84 | 1 | |
| Havlak |  |
15 | 11977.70 | 3 | |
| Json |  |
15 | 3545.32 | -2 | |
| List |  |
15 | 6261.13 | 3 | |
| Mandelbrot |  |
15 | 4743.41 | 19 | |
| NBody |  |
15 | 8798.60 | -7 | |
| Permute |  |
15 | 6757.10 | 0 | |
| Queens |  |
15 | 7030.66 | 12 | |
| Richards |  |
15 | 7491.07 | 5 | |
| Sieve |  |
15 | 7590.75 | -15 | |
| Storage |  |
15 | 1927.75 | -3 | |
| Towers |  |
15 | 7615.24 | -4 |
Executor Comparisons
som-interp
Baseline: PySOM-ast-int
| Exe | #M | median time in ms |
time diff % | |||
|---|---|---|---|---|---|---|
| Bounce |  |
PySOM-ast-int
PySOM-bc-int TruffleSOM-ast-NativeEE-int-main TruffleSOM-bc-NativeEE-int-main |
15
15 15 15 |
4699.11
4520.38 4279.90 6201.95 |
0
-4 -9 32 |
|
| CD |  |
PySOM-ast-int
PySOM-bc-int TruffleSOM-ast-NativeEE-int-main TruffleSOM-bc-NativeEE-int-main |
15
15 15 15 |
7128.99
6785.85 9144.78 10069.86 |
0
-5 28 41 |
|
| DeltaBlue |  |
PySOM-ast-int
PySOM-bc-int TruffleSOM-ast-NativeEE-int-main TruffleSOM-bc-NativeEE-int-main |
15
15 15 15 |
1360.18
1417.55 1921.08 2413.84 |
0
4 41 77 |
|
| Havlak |  |
PySOM-ast-int
PySOM-bc-int TruffleSOM-ast-NativeEE-int-main TruffleSOM-bc-NativeEE-int-main |
15
15 15 15 |
7522.28
7611.18 9928.47 11977.70 |
0
1 32 59 |
|
| Json |  |
PySOM-ast-int
PySOM-bc-int TruffleSOM-ast-NativeEE-int-main TruffleSOM-bc-NativeEE-int-main |
15
15 15 15 |
2014.62
2001.78 2625.36 3545.32 |
0
-1 30 76 |
|
| List |  |
PySOM-ast-int
PySOM-bc-int TruffleSOM-ast-NativeEE-int-main TruffleSOM-bc-NativeEE-int-main |
15
15 15 15 |
4439.97
3965.85 6575.65 6261.13 |
0
-11 48 41 |
|
| Mandelbrot |  |
PySOM-ast-int
PySOM-bc-int TruffleSOM-ast-NativeEE-int-main TruffleSOM-bc-NativeEE-int-main |
15
15 15 15 |
4395.70
3549.89 1844.75 4743.41 |
0
-19 -58 8 |
|
| NBody |  |
PySOM-ast-int
PySOM-bc-int TruffleSOM-ast-NativeEE-int-main TruffleSOM-bc-NativeEE-int-main |
15
15 15 15 |
5805.45
6807.49 4872.43 8798.60 |
0
17 -16 52 |
|
| Permute |  |
PySOM-ast-int
PySOM-bc-int TruffleSOM-ast-NativeEE-int-main TruffleSOM-bc-NativeEE-int-main |
15
15 15 15 |
4259.12
4110.39 5392.13 6757.10 |
0
-3 27 59 |
|
| Queens |  |
PySOM-ast-int
PySOM-bc-int TruffleSOM-ast-NativeEE-int-main TruffleSOM-bc-NativeEE-int-main |
15
15 15 15 |
4411.87
5440.97 3933.83 7030.66 |
0
23 -11 59 |
|
| Richards |  |
PySOM-ast-int
PySOM-bc-int TruffleSOM-ast-NativeEE-int-main TruffleSOM-bc-NativeEE-int-main |
15
15 15 15 |
4432.20
4517.85 6232.40 7491.07 |
0
2 41 69 |
|
| Sieve |  |
PySOM-ast-int
PySOM-bc-int TruffleSOM-ast-NativeEE-int-main TruffleSOM-bc-NativeEE-int-main |
15
15 15 15 |
7577.52
7165.55 3242.46 7590.75 |
0
-5 -57 0 |
|
| Storage |  |
PySOM-ast-int
PySOM-bc-int TruffleSOM-ast-NativeEE-int-main TruffleSOM-bc-NativeEE-int-main |
15
15 15 15 |
1251.48
1314.73 1596.00 1927.75 |
0
5 28 54 |
|
| Towers |  |
PySOM-ast-int
PySOM-bc-int TruffleSOM-ast-NativeEE-int-main TruffleSOM-bc-NativeEE-int-main |
15
15 15 15 |
4717.53
4993.91 6442.92 7615.24 |
0
6 37 61 |
somrs-interp
Baseline: SOM-RS-ast-int
| Exe | #M | median time in ms |
time diff % | |||
|---|---|---|---|---|---|---|
| Bounce |  |
ast-int
bc-int |
1
1 |
27568.00
27195.07 |
0
-1 |
|
| CD |  |
ast-int
bc-int |
1
1 |
28153.27
0.00 |
0
-100 |
|
| DeltaBlue |  |
ast-int
bc-int |
1
1 |
4746.00
0.00 |
0
-100 |
|
| Havlak |  |
ast-int
bc-int |
1
1 |
27394.82
0.00 |
0
-100 |
|
| Json |  |
ast-int
bc-int |
1
1 |
15800.33
14871.54 |
0
-6 |
|
| List |  |
ast-int
bc-int |
1
1 |
18942.44
28681.32 |
0
51 |
|
| Mandelbrot |  |
ast-int
bc-int |
1
1 |
16278.45
15226.31 |
0
-6 |
|
| NBody |  |
ast-int
bc-int |
1
1 |
18276.06
27927.94 |
0
53 |
|
| Permute |  |
ast-int
bc-int |
1
1 |
16855.98
16220.76 |
0
-4 |
|
| Queens |  |
ast-int
bc-int |
1
1 |
22392.19
22650.37 |
0
1 |
|
| Richards |  |
ast-int
bc-int |
1
1 |
16688.75
19518.76 |
0
17 |
|
| Sieve |  |
ast-int
bc-int |
1
1 |
40689.69
35480.44 |
0
-13 |
|
| Storage |  |
ast-int
bc-int |
1
1 |
6728.21
5355.76 |
0
-20 |
|
| Towers |  |
ast-int
bc-int |
1
1 |
17359.71
20161.11 |
0
16 |