The perfcompare tool requires Python 3.
There are two ways to run the perfcompare tool with the required dependencies (currently just the
scipy Python library):
This will automatically download prebuilt, hermetic versions of dependencies.
vpython is used for running perfcompare on the Infra builders.
On Linux, when using Debian/Ubuntu, the dependencies can be installed using APT:
sudo apt-get install python3-scipy
The perfcompare tool can be used to run perf tests locally and to compare the results.
As an example, suppose you want to compare the results from
rust_inspect_benchmarks_test on two Git commits,
This test case is tested in the
terminal product and may not work in a different product.
The following commands would gather a dataset of perf test results for
BEFORE_VERSION and save them in the directory
git checkout BEFORE_VERSION fx set terminal.x64 # covers dependencies of rust_inspect_benchmarks_test fx build fx update python garnet/bin/perfcompare/perfcompare.py run_local \ --boots=5 \ --iter_cmd='fx test --e2e rust_inspect_benchmarks_test' \ --iter_file='out/test_out/*/*.fuchsiaperf.json' \ --dest=perf_results_before
These commands would do the same, but for
AFTER_VERSION, saving the results dataset in a different directory,
git checkout AFTER_VERSION fx build fx update python garnet/bin/perfcompare/perfcompare.py run_local \ --boots=5 \ --iter_cmd='fx test --e2e rust_inspect_benchmarks_test' \ --iter_file='out/test_out/*/*.fuchsiaperf.json' \ --dest=perf_results_after
Note that the
run_local commands will reboot Fuchsia.
The two datasets can then be compared with the following command, which prints a table showing the “before” and “after” results side by side:
python garnet/bin/perfcompare/perfcompare.py compare_perf perf_results_before perf_results_after