rustc_metadata: Preprocess search paths for better performance

Over in Zed we've noticed that loading crates for a large-ish workspace can take almost 200ms. We've pinned it down to how rustc searches for paths, as it performs a linear search over the list of candidate paths. In our case the candidate list had about 20k entries which we had to iterate over for each dependency being loaded.

This commit introduces a simple FilesIndex that's just a sorted Vec under the hood. Since crates are looked up by both prefix and suffix, we perform a range search on said Vec (which constraints the search space based on prefix) and follow up with a linear scan of entries with matching suffixes.
FilesIndex is also pre-filtered before any queries are performed using available target information; query prefixes/sufixes are based on the target we are compiling for, so we can remove entries that can never match up front.

Overall, this commit brings down build time for us in dev scenarios by about 6%.
100ms might not seem like much, but this is a constant cost that each of our workspace crates has to pay, even when said crate is miniscule.
8 files changed
tree: cb19d29feee87d355e1a7646d4a7a279e6c9d5c9
  1. .github/
  2. compiler/
  3. library/
  4. LICENSES/
  5. src/
  6. tests/
  7. .clang-format
  8. .editorconfig
  9. .git-blame-ignore-revs
  10. .gitattributes
  11. .gitignore
  12. .gitmodules
  13. .ignore
  14. .mailmap
  15. Cargo.lock
  16. Cargo.toml
  17. CODE_OF_CONDUCT.md
  18. config.example.toml
  19. configure
  20. CONTRIBUTING.md
  21. COPYRIGHT
  22. INSTALL.md
  23. LICENSE-APACHE
  24. LICENSE-MIT
  25. README.md
  26. RELEASES.md
  27. REUSE.toml
  28. rust-bors.toml
  29. rustfmt.toml
  30. triagebot.toml
  31. x
  32. x.ps1
  33. x.py
README.md

Website | Getting started | Learn | Documentation | Contributing

This is the main source code repository for Rust. It contains the compiler, standard library, and documentation.

Why Rust?

  • Performance: Fast and memory-efficient, suitable for critical services, embedded devices, and easily integrate with other languages.

  • Reliability: Our rich type system and ownership model ensure memory and thread safety, reducing bugs at compile-time.

  • Productivity: Comprehensive documentation, a compiler committed to providing great diagnostics, and advanced tooling including package manager and build tool (Cargo), auto-formatter (rustfmt), linter (Clippy) and editor support (rust-analyzer).

Quick Start

Read “Installation” from The Book.

Installing from Source

If you really want to install from source (though this is not recommended), see INSTALL.md.

Getting Help

See https://www.rust-lang.org/community for a list of chat platforms and forums.

Contributing

See CONTRIBUTING.md.

License

Rust is primarily distributed under the terms of both the MIT license and the Apache License (Version 2.0), with portions covered by various BSD-like licenses.

See LICENSE-APACHE, LICENSE-MIT, and COPYRIGHT for details.

Trademark

The Rust Foundation owns and protects the Rust and Cargo trademarks and logos (the “Rust Trademarks”).

If you want to use these names or brands, please read the media guide.

Third-party logos may be subject to third-party copyrights and trademarks. See Licenses for details.