Auto merge of #128320 - saethlin:link-me-maybe, r=compiler-errors Avoid no-op unlink+link dances in incr comp Incremental compilation scales quite poorly with the number of CGUs. This PR improves one reason for that. The incr comp process hard-links all the files from an old session into a new one, then it runs the backend, which may just hard-link the new session files into the output directory. Then codegen hard-links all the output files back to the new session directory. This PR (perhaps unimaginatively) fixes the silliness that ensues in the last step. The old `link_or_copy` implementation would be passed pairs of paths which are already the same inode, then it would blindly delete the destination and re-create the hard-link that it just deleted. This PR lets us skip both those operations. We don't skip the other two hard-links. `cargo +stage1 b && touch crates/core/main.rs && strace -cfw -elink,linkat,unlink,unlinkat cargo +stage1 b` before and then after on `ripgrep-13.0.0`: ``` % time seconds usecs/call calls errors syscall ------ ----------- ----------- --------- --------- ---------------- 52.56 0.024950 25 978 485 unlink 34.38 0.016318 22 727 linkat 13.06 0.006200 24 249 unlinkat ------ ----------- ----------- --------- --------- ---------------- 100.00 0.047467 24 1954 485 total ``` ``` % time seconds usecs/call calls errors syscall ------ ----------- ----------- --------- --------- ---------------- 42.83 0.014521 57 252 unlink 38.41 0.013021 26 486 linkat 18.77 0.006362 25 249 unlinkat ------ ----------- ----------- --------- --------- ---------------- 100.00 0.033904 34 987 total ``` This reduces the number of hard-links that are causing perf troubles, noted in https://github.com/rust-lang/rust/issues/64291 and https://github.com/rust-lang/rust/issues/137560
The goal of this project is to create an alternative codegen backend for the rust compiler based on Cranelift. This has the potential to improve compilation times in debug mode. If your project doesn't use any of the things listed under “Not yet supported”, it should work fine. If not please open an issue.
The Cranelift codegen backend is distributed in nightly builds on Linux and x86_64 macOS. If you want to install it using Rustup, you can do that by running:
$ rustup component add rustc-codegen-cranelift-preview --toolchain nightly
Once it is installed, you can enable it with one of the following approaches:
CARGO_PROFILE_DEV_CODEGEN_BACKEND=cranelift cargo +nightly build -Zcodegen-backend.cargo/config.toml:[unstable] codegen-backend = true [profile.dev] codegen-backend = "cranelift"
Cargo.toml:# This line needs to come before anything else in Cargo.toml cargo-features = ["codegen-backend"] [profile.dev] codegen-backend = "cranelift"
You can also download a pre-built version from the releases page. Extract the dist directory in the archive anywhere you want. If you want to use cargo clif build instead of having to specify the full path to the cargo-clif executable, you can add the bin subdirectory of the extracted dist directory to your PATH. (tutorial for Windows, and for Linux/MacOS).
If you want to build the backend manually, you can download it from GitHub and build it yourself:
$ git clone https://github.com/rust-lang/rustc_codegen_cranelift $ cd rustc_codegen_cranelift $ ./y.sh prepare $ ./y.sh build
To run the test suite replace the last command with:
$ ./test.sh
For more docs on how to build and test see build_system/usage.txt or the help message of ./y.sh.
| OS \ architecture | x86_64 | AArch64 | Riscv64 | s390x (System-Z) |
|---|---|---|---|---|
| Linux | ✅ | ✅ | ✅[^no-rustup] | ✅[^no-rustup] |
| FreeBSD | ✅[^no-rustup] | ❓ | ❓ | ❓ |
| AIX | ❌[^xcoff] | N/A | N/A | ❌[^xcoff] |
| Other unixes | ❓ | ❓ | ❓ | ❓ |
| macOS | ✅ | ✅ | N/A | N/A |
| Windows | ✅ | ❌ | N/A | N/A |
✅: Fully supported and tested ❓: Maybe supported, not tested ❌: Not supported at all
Not all targets are available as rustup component for nightly. See notes in the platform support matrix.
[^xcoff]: XCOFF object file format is not supported. [^no-rustup]: Not available as rustup component for nightly. You can build it yourself.
rustc_codegen_cranelift can be used as a near-drop-in replacement for cargo build or cargo run for existing projects.
Assuming $cg_clif_dir is the directory you cloned this repo into and you followed the instructions (y.sh prepare and y.sh build or test.sh).
In the directory with your project (where you can do the usual cargo build), run:
$ $cg_clif_dir/dist/cargo-clif build
This will build your project with rustc_codegen_cranelift instead of the usual LLVM backend.
For additional ways to use rustc_codegen_cranelift like the JIT mode see usage.md.
See rustc_testing.md.
std::simd fully works, std::arch is partially supported)-Cpanic=abort is enabled by default)Licensed under either of
at your option.
Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in the work by you shall be dual licensed as above, without any additional terms or conditions.