tree: ed4e96bc6cf6b14aba1e8028832d60649f9922c8 [path history] [tgz]
  1. cmd/
  2. BUILD.gn
  3. mass_test_failure_check.go
  4. mass_test_failure_check_test.go
  5. no_tests_ran_check.go
  6. no_tests_ran_check_test.go
  7. README.md
  8. run_checks.go
  9. run_checks_test.go
  10. string_in_log_check.go
  11. string_in_log_check_test.go
  12. swarming.go
  13. task_state_check.go
  14. task_state_check_test.go
  15. types.go
tools/testing/tefmocheck/README.md

Tefmocheck

Tefmocheck (Testing Failure Mode Checker) analyzes the outputs of a testing Swarming task and determines whether various failures modes occured. It produces a testing summary file (summmary.json) that contains all of the tests in the input summary, as well as a synthetic test for each failure mode starting with “testing_failure_mode/”.

At most a single check (the most specific one possible) will fail on a single task. The least specific ones start with “testing_failure_mode/task_status/”. This just surfaces the Swarming task status. To see further details, see the infra_and_test_std_and_klog.txt, which includes the output of the Swarming task.

This tool is invoked by the infrastructure recipes, so any changes to its interface must be soft transitions.

Test names

The tests results produced by this tool are analyzed for flakiness. Currently our flake analysis relies on seeing a test fail and then pass, which means that the test names must appear in the output summary even if they pass. This means that we cannot parse a string out of an error message and use that in a test name. https://fxbug.dev./62307 tracks improving this.