[WIP] Add benchmarking project: Botan

[webmaster128]

This is a benchmark compiling a huge project. I'll use it to test #217.

Closes #83.

[codecov-io]

Current coverage is 88.02% (diff: 100%)

Merging #226 into master will decrease coverage by 1.39%

@@             master       #226   diff @@
==========================================
  Files             1          1          
  Lines          1040        969    -71   
  Methods           0          0          
  Messages          0          0          
  Branches        166        160     -6   
==========================================
- Hits            930        853    -77   
- Misses           82         86     +4   
- Partials         28         30     +2   

Powered by Codecov. Last update 1e7b28a...b6601b9

[frerich]

Good stuff! Just tried to give it a shot, but got

C:\Users\Frerich\src\clcache>python benchmark\benchmark_project.py
Running 2 parallel jobs
Traceback (most recent call last):
  File "benchmark\benchmark_project.py", line 91, in <module>
    reset(logfile)
  File "benchmark\benchmark_project.py", line 68, in reset
    shutil.rmtree("build")
  File "C:\ProgramData\chocolatey\lib\python3\tools\lib\shutil.py", line 488, in rmtree
    return _rmtree_unsafe(path, onerror)
  File "C:\ProgramData\chocolatey\lib\python3\tools\lib\shutil.py", line 370, in _rmtree_unsafe
    onerror(os.listdir, path, sys.exc_info())
  File "C:\ProgramData\chocolatey\lib\python3\tools\lib\shutil.py", line 368, in _rmtree_unsafe
    names = os.listdir(path)
FileNotFoundError: [WinError 3] The system cannot find the path specified: 'build'

I added ignore_errors=True to the shutil.rmdir call, but then it bailed out because jom is needed. Is that going to be a strict requirement?

[webmaster128]

Yes, jom is going to be a requirement, because it adds multicore supprt, that nmake does not have.

I did a few untested modifications: in the first build, the directory cannot exists obviously ...

[webmaster128]

Closing as it is unlikely I will finish this PR. If somebody wants to pick it up, go ahead.