Merge #10995

10995: internal: switch from trait-based TokenSource to simple struct of arrays r=matklad a=matklad

cc #10765 

The idea here is to try to simplify the interface as best as we can. The original trait-based approach is a bit over-engineered and hard to debug. Here, we replace callback with just data. The next PR in series will replace the output `TreeSink` trait with data as well. 


The biggest drawback here is that we now require to materialize all parser's input up-front. This is a bad fit for macro by example: when you parse `$e:expr`, you might consume only part of the input. However, today's trait-based solution doesn't really help -- we were already materializing the whole thing! So, let's keep it simple!

Co-authored-by: Aleksey Kladov <aleksey.kladov@gmail.com>
tree: dc2644c96f3cea0e29539a236112211d3b82ee6c
  1. .cargo/
  2. .github/
  3. .vscode/
  4. assets/
  5. bench_data/
  6. crates/
  7. docs/
  8. editors/
  9. lib/
  10. xtask/
  11. .gitattributes
  12. .gitignore
  13. bors.toml
  14. Cargo.lock
  15. Cargo.toml
  16. LICENSE-APACHE
  17. LICENSE-MIT
  18. PRIVACY.md
  19. README.md
  20. rustfmt.toml
README.md

rust-analyzer is a modular compiler frontend for the Rust language. It is a part of a larger rls-2.0 effort to create excellent IDE support for Rust.

Work on rust-analyzer is sponsored by

Quick Start

https://rust-analyzer.github.io/manual.html#installation

Documentation

If you want to contribute to rust-analyzer or are just curious about how things work under the hood, check the ./docs/dev folder.

If you want to use rust-analyzer's language server with your editor of choice, check the manual folder. It also contains some tips & tricks to help you be more productive when using rust-analyzer.

Security and Privacy

See the corresponding sections of the manual.

Communication

For usage and troubleshooting requests, please use “IDEs and Editors” category of the Rust forum:

https://users.rust-lang.org/c/ide/14

For questions about development and implementation, join rust-analyzer working group on Zulip:

https://rust-lang.zulipchat.com/#narrow/stream/185405-t-compiler.2Frust-analyzer

Quick Links

License

Rust analyzer is primarily distributed under the terms of both the MIT license and the Apache License (Version 2.0).

See LICENSE-APACHE and LICENSE-MIT for details.