[update_test_durations] Garbage-collect old files
Previously we would never delete old test duration files when the
corresponding builder got deleted, so we still have files sitting around
in the CIPD packages for builders that got deleted many years ago.
Also, we'd delete a test from a file as soon as the test hadn't run in 3
days, so if a test got disabled for, say, 10 days, when it got
re-enabled we would no longer have duration data for it.
This solve both of those problems, leaving test entries around for 30
days after they are last seen, and deleting builder files when no test
on that builder has run in the last 30 days. We do this by storing a
`last_seen` field in the JSON files that records the last date on which
the entry was recorded. For backwards compatibility, I'm having the tool
set the `last_seen` field to the current date if it's empty, so the
field will automatically be populated for existing old builder files,
and they will eventually be deleted 30 days after this lands.
Fixed: 42138833
Fixed: 42139134
Change-Id: Ib21439378f8b16690736084c2c2511d0917c3920
Reviewed-on: https://fuchsia-review.googlesource.com/c/infra/infra/+/1052652
Reviewed-by: Danielle Kay <danikay@google.com>
Commit-Queue: Auto-Submit <auto-submit@fuchsia-infra.iam.gserviceaccount.com>
Fuchsia-Auto-Submit: Oliver Newman <olivernewman@google.com>
8 files changed
