commit 1c8a07cfd64fa93a8ecb6310c52710c5a4427a38
author Chayim Refael Friedman <chayimfr@gmail.com> Sun Aug 24 16:44:28 2025 +0300
committer Chayim Refael Friedman <chayimfr@gmail.com> Mon Aug 25 17:17:31 2025 +0300
tree 6d7dbc3ec87a1a66bbc14187a46c29e0e4b66f89
parent e6cd085099f497ccba11ddd398bdc85eaef4fcdc

Cache trait solving across queries in the same revision

Caching trait solving can do a lot to speed. Unfortunately it also consume a huge amount of memory. Therefore, as part of the migration to the new solver Jack Huey disabled caching of trait solving (he made the query transparent).

The PR proposes a middle ground: do cache trait solving, but only for the same revision. This allows us to be safe because during a revision the inputs cannot change.

The result is hopefully much better performance to features that tend to do a bulk of trait solving, and also repeat the same query (e.g. inference then IDE features).

There is another limitation: results are only cached in the same thread, to remove the need for synchronization which will be expensive. More measurements are required to check whether it's better to use a synchronized global cache, or maybe stay with a thread-local cache but batch multiple feature requests (highlighting, inlay hints etc.) of the same file to the same thread.

Alongside the actual cache we store the revision, because we need to verify it (we can't eagerly clear caches when incrementing the revision), and also the address of the db to prevent multiple dbs from interleaving (this is mostly relevant in tests, although injected highlighting also uses a new db, therefore maybe it's better to move it to a separate thread).

This "games" analysis-stats to both be way faster and use way more memory; the former is because analysis-stats doesn't increment revisions, therefore all queries share the cache and hit ratio is way too good, the latter is because analysis-stats doesn't increment revisions and therefore the cache isn't cleared. Both are not representative of a typical IDE scenario.