Performance
Benchmark Tests were conducted on Amazon Web Services EC2 servers with the following results :
- Compilation times were an average from at least 5 results
- Turbo Agents on 3 m6a.xlarge EC2 instances (18 Slots) gave 30.4% improvement
- Turbo Agents on 6 m6a.xlarge EC2 instances (36 Slots) gave 51.6% improvement
- Turbo Agents on 8 m6a.xlarge EC2 instances (48 Slots) gave 55.7% improvement
The details :
- The EC2 instances used were m6a.xlarge (4 vCPU, 16GB RAM) using Amazons AMI for Windows 2019 Server
- Compilation time was measured for the Foundation solution from within the Poco Framework v1.13.2
- Compilation was on Visual Studio 2019, under "Release" mode and in x64 architecture using msbuild.exe
- Each EC2 instance had a Turbo Agent installed with 6 CPU Slots allocated
- The Turbo Scheduler was installed on a separate m6a.large EC2 instance (no Agent installed on this machine)
- Some folders used by Turbo were "excluded from realtime scanning" from Microsoft Windows Defender to prevent it affecting results (c:\ProgramData\Turbo\logs, c:\ProgramData\Turbo\work-temp, and the Poco Foundation source code folder)
- Only a single build was running on the Build Cluster
- Consistent, repeatable testing was achieved using a Packer built EC2 image and deployed using Terraform (files available on request)
Some background on factors that affect compilation times :
- The number of c/c++ files within each VS Project. Fewer and larger Projects will see more improvement.
- Compilation mode. Release mode will see more improvement than Debug
- Size of the Build Cluster
- The current workload on the Build Cluster
- No caching of compiled object files was used (this facility will be coming in a future release)