Rollup merge of #143070 - joshtriplett:macro-rules-parse, r=petrochenkov

Rewrite `macro_rules!` parser to not use the MBE engine itself

The `macro_rules!` parser was written to match the series of rules using the macros-by-example (MBE) engine and a hand-written equivalent of the left-hand side of a MBE macro. This was complex to read, difficult to extend, and produced confusing error messages. Because it was using the MBE engine, any parse failure would be reported as if some macro was being applied to the `macro_rules!` invocation itself; for instance, errors would talk about "macro invocation", "macro arguments", and "macro call", when they were actually about the macro *definition*.

And in practice, the `macro_rules!` parser only used the MBE engine to extract the left-hand side and right-hand side of each rule as a token tree, and then parsed the rest using a separate parser.

Rewrite it to parse the series of rules using a simple loop, instead. This makes it more extensible in the future, and improves error messages. For instance, omitting a semicolon between rules will result in "expected `;`" and "unexpected token", rather than the confusing "no rules expected this token in macro call".

This work was greatly aided by pair programming with Vincenzo Palazzo (`@vincenzopalazzo)` and Eric Holk (`@eholk).`

For review, I recommend reading the two commits separately.
tree: 22d5450d5d7030b9578f7e3449854d5c35143064
  1. .github/
  2. compiler/
  3. library/
  4. LICENSES/
  5. src/
  6. tests/
  7. .clang-format
  8. .editorconfig
  9. .git-blame-ignore-revs
  10. .gitattributes
  11. .gitignore
  12. .gitmodules
  13. .ignore
  14. .mailmap
  15. bootstrap.example.toml
  16. Cargo.lock
  17. Cargo.toml
  18. CODE_OF_CONDUCT.md
  19. configure
  20. CONTRIBUTING.md
  21. COPYRIGHT
  22. INSTALL.md
  23. LICENSE-APACHE
  24. license-metadata.json
  25. LICENSE-MIT
  26. README.md
  27. RELEASES.md
  28. REUSE.toml
  29. rust-bors.toml
  30. rustfmt.toml
  31. triagebot.toml
  32. x
  33. x.ps1
  34. x.py
README.md

Website | Getting started | Learn | Documentation | Contributing

This is the main source code repository for Rust. It contains the compiler, standard library, and documentation.

Why Rust?

  • Performance: Fast and memory-efficient, suitable for critical services, embedded devices, and easily integrated with other languages.

  • Reliability: Our rich type system and ownership model ensure memory and thread safety, reducing bugs at compile-time.

  • Productivity: Comprehensive documentation, a compiler committed to providing great diagnostics, and advanced tooling including package manager and build tool (Cargo), auto-formatter (rustfmt), linter (Clippy) and editor support (rust-analyzer).

Quick Start

Read “Installation” from The Book.

Installing from Source

If you really want to install from source (though this is not recommended), see INSTALL.md.

Getting Help

See https://www.rust-lang.org/community for a list of chat platforms and forums.

Contributing

See CONTRIBUTING.md.

License

Rust is primarily distributed under the terms of both the MIT license and the Apache License (Version 2.0), with portions covered by various BSD-like licenses.

See LICENSE-APACHE, LICENSE-MIT, and COPYRIGHT for details.

Trademark

The Rust Foundation owns and protects the Rust and Cargo trademarks and logos (the “Rust Trademarks”).

If you want to use these names or brands, please read the Rust language trademark policy.

Third-party logos may be subject to third-party copyrights and trademarks. See Licenses for details.