This document describes how to set up a development loop for people interested in contributing to Swift.
If you are only interested in building the toolchain as a one-off, there are a couple of differences:
mkdir -p swift-project/swift cd swift-project/swift
git clone git@github.com:apple/swift.git . utils/update-checkout --clone-with-ssh
git clone https://github.com/apple/swift.git . utils/update-checkout --clone
swift's sibling directories are present.ls ..This should list directories like
llvm-project, swiftpm and so on.swift's master branch and matching branches for other projects. If you are building the toolchain as a one-off, it is more likely that you want a specific branch or a tag, often corresponding to a specific release or a specific snapshot. You can update the branch/tag for all repositories as follows:utils/update-checkout --branch mybranchname # OR utils/update-checkout --tag mytagnameDetailed branching information, including names for release branches, can be found in Branches.md.
Note: The commands used in the rest of this guide assumes that the absolute path to your working directory is something like /path/to/swift-project/swift. Double-check that running pwd prints a path ending with swift.
update-checkout failed, double-check that the absolute path to your working directory does not have non-ASCII characters.update-checkout failed and the absolute path to your working directory had spaces in it, please file a bug report and change the path to work around it.update-checkout, double-check that swift is the only repository inside the swift-project directory. Otherwise, update-checkout may not clone the necessary dependencies.brew install cmake ninja sccache
brew bundle
For Ubuntu 16.04 LTS and 18.04 LTS, run the following:
sudo apt-get install \ clang \ cmake \ git \ icu-devtools \ libcurl4-openssl-dev \ libedit-dev \ libicu-dev \ libncurses5-dev \ libpython3-dev \ libsqlite3-dev \ libxml2-dev \ ninja-build \ pkg-config \ python \ python-six \ rsync \ swig \ systemtap-sdt-dev \ tzdata \ uuid-dev sudo snap install sccache --candidate --classic
Note: LLDB currently requires at least swig-1.3.40 but will successfully build with version 2 shipped with Ubuntu.
cmake --version: This should be 3.18.1 or higher for macOS.python3 --version: Check that this succeeds.ninja --version: Check that this succeeds.sccache --version: Check that this succeeds.At this point, it is worthwhile to pause for a moment to understand what the different tools do:
utils/update-checkout is a script to help you work with all the individual git repositories together, instead of manually cloning/updating each one.utils/build-script (we will introduce this shortly) is a high-level automation script that handles configuration (via CMake), building (via Ninja or Xcode), caching (via Sccache), running tests and more.Pro Tip: Most tools support
--helpflags describing the options they support. Additionally, both Clang and the Swift compiler have hidden flags (clang --help-hidden/swiftc --help-hidden) and frontend flags (clang -cc1 --help/swiftc -frontend --help) and the Swift compiler even has hidden frontend flags (swiftc -frontend --help-hidden). Sneaky!
Phew, that‘s a lot to digest! Now let’s proceed to the actual build itself!
sccache --start-server(Optional) Sccache defaults to a cache size of 10GB, which is relatively small compared to build artifacts. You can bump it up, say by setting
export SCCACHE_CACHE_SIZE="50G" in your dotfile(s). For more details, see the Sccache README.utils/build-script --skip-build-benchmarks \ --skip-ios --skip-watchos --skip-tvos --swift-darwin-supported-archs "x86_64" \ --cmake-c-launcher="$(which sccache)" --cmake-cxx-launcher="$(which sccache)" \ --release-debuginfo --test
utils/build-script --skip-build-benchmarks \ --skip-ios --skip-watchos --skip-tvos --swift-darwin-supported-archs "x86_64" \ --cmake-c-launcher="$(which sccache)" --cmake-cxx-launcher="$(which sccache)" \ --release-debuginfo --test \ --xcode
swift-project/build/Ninja-RelWithDebInfoAssert (with Xcode instead of Ninja if you used --xcode) containing the build artifacts.In the following sections, for simplicity, we will assume that you are using a Ninja-RelWithDebInfoAssert build on macOS running on an Intel-based Mac, unless explicitly mentioned otherwise. You will need to slightly tweak the paths for other build configurations.
Some contributors find it more convenient to use both Ninja and Xcode. Typically this configuration consists of:
--release-debuginfo.--release-debuginfo --debug-swift.The Ninja build can be used for fast incremental compilation and running tests quickly. The Xcode build can be used for debugging with high fidelity.
The additional flexibility comes with two issues: (1) consuming much more disk space and (2) you need to maintain the two builds in sync, which needs extra care when moving across branches.
git checkout to change the branch only for swift (often to a release branch), leading to an unsupported configuration. See Step 4 of Cloning the Project on how to fix this.build-script in the log. While build-script should work with paths containing spaces, sometimes bugs do slip through, such as SR-13441. If this is the case, please file a bug report and change the path to work around it.build-script invocation doesn't have typos. You can compare the flags you passed against the supported flags listed by utils/build-script --help.utils/update-checkout --dump-hashes.If you are building the toolchain for development and submitting patches, you will need to setup a GitHub fork.
First fork the apple/swift repository, using the “Fork” button in the web UI, near the top-right. This will create a repository username/swift for your GitHub username. Next, add it as a remote:
# Using 'my-remote' as a placeholder name. # If you set up SSH in step 2 git remote add my-remote git@github.com:username/swift.git # If you used HTTPS in step 2 git remote add my-remote https://github.com/username/swift.git
Finally, create a new branch.
# Using 'my-branch' as a placeholder name git checkout my-branch git push --set-upstream my-remote my-branch
If you used --xcode earlier, you will see an Xcode project generated under ../build/Xcode-RelWithDebInfoAssert/swift-macosx-x86_64. When you open the project, Xcode might helpfully suggest “Automatically Create Schemes”. Most of those schemes are not required in day-to-day work, so you can instead manually select the following schemes:
swift-frontend: If you will be working on the compiler.check-swift-all: This can be used to run the tests. The test runner does not integrate with Xcode though, so it may be easier to run tests directly on the commandline for more fine-grained control over which exact tests are run.Make changes to the code as appropriate. Implement a shiny new feature! Or fix a nasty bug! Update the documentation as you go! The codebase is your oyster!
:construction::construction_worker::building_construction:
Now that you have made some changes, you will need to rebuild...
To rebuild the compiler:
ninja -C ../build/Ninja-RelWithDebInfoAssert/swift-macosx-x86_64 swift-frontend
To rebuild everything, including the standard library:
ninja -C ../build/Ninja-RelWithDebInfoAssert/swift-macosx-x86_64
Rebuilding works the same way as with any other Xcode project; you can use ⌘+B or Product → Build.
As a quick test, go to lib/Basic/Version.cpp and tweak the version printing code slightly. Next, do an incremental build as above. This incremental build should be much faster than the from-scratch build at the beginning. Now check if the version string has been updated:
../build/Ninja-RelWithDebInfoAssert/swift-macosx-x86_64/bin/swift-frontend --version
This should print your updated version string.
Starter bugs typically have small code examples that fit within a single file. You can reproduce such an issue in various ways, such as compiling it from the commandline using /path/to/swiftc MyFile.swift, pasting the code into Compiler Explorer (aka godbolt) or using an Xcode Playground.
For files using frameworks from an SDK bundled with Xcode, you need the pass the SDK explicitly. Here are a couple of examples:
# Compile a file to an executable for your local machine. xcrun -sdk macosx /path/to/swiftc MyFile.swift # Say you are trying to compile a file importing an iOS-only framework. xcrun -sdk iphoneos /path/to/swiftc -target arm64-apple-ios13.0 MyFile.swift
You can see the full list of -sdk options using xcodebuild -showsdks, and check some potential -target options for different operating systems by skimming the compiler's test suite under test/.
Sometimes bug reports come with SwiftPM packages or Xcode projects as minimal reproducers. While we do not add packages or projects to the compiler‘s test suite, it is generally helpful to first reproduce the issue in context before trying to create a minimal self-contained test case. If that’s the case with the bug you're working on, check out our instructions on building packages and Xcode projects with a locally built compiler.
There are two main ways to run tests:
utils/run-test: By default, run-test builds the tests' dependencies before running them.# Rebuild all test dependencies and run all tests under test/. utils/run-test --lit ../llvm-project/llvm/utils/lit/lit.py \ ../build/Ninja-RelWithDebInfoAssert/swift-macosx-x86_64/test-macosx-x86_64 # Rebuild all test dependencies and run tests containing "MyTest". utils/run-test --lit ../llvm-project/llvm/utils/lit/lit.py \ ../build/Ninja-RelWithDebInfoAssert/swift-macosx-x86_64/test-macosx-x86_64 \ --filter="MyTest"
lit.py: lit doesn't know anything about dependencies. It just runs tests.# Run all tests under test/. ../llvm-project/llvm/utils/lit/lit.py -s -vv \ ../build/Ninja-RelWithDebInfoAssert/swift-macosx-x86_64/test-macosx-x86_64 # Run tests containing "MyTest" ../llvm-project/llvm/utils/lit/lit.py -s -vv \ ../build/Ninja-RelWithDebInfoAssert/swift-macosx-x86_64/test-macosx-x86_64 \ --filter="MyTest"The
-s and -vv flags print a progress bar and the executed commands respectively.If you making small changes to the compiler or some other component, you'll likely want to incrementally rebuild only the relevant Ninja/Xcode target and use lit.py with --filter. One potential failure mode with this approach is accidental use of stale binaries. For example, say that you want to rerun a SourceKit test but you only incrementally rebuilt the compiler. Then your changes will not be reflected when the test runs because the sourcekitd binary was not rebuilt. Using run-test instead is the safer option, but it will lead to a longer feedback loop due to more things getting rebuilt.
If you want to rerun all the tests, you can either rebuild the whole project and use lit.py without --filter or use run-test to handle both aspects.
Recall the baseline failures mentioned in the build section. If your baseline had failing tests, make sure you compare the failures seen after your changes to the baseline. If some test failures look totally unrelated to your changes, there is a good chance that they were already failing as part of the baseline.
For more details on running tests and understanding the various Swift-specific lit customizations, see Testing.md. Also check out the lit documentation to understand how the different lit commands work.
In this section, we briefly describe two common ways of debugging: print debugging and using LLDB.
Depending on the code you're interested in, LLDB may be significantly more effective when using a debug build. Depending on what components you are working on, you could turn off optimizations for only a few things. Here are some example invocations:
# optimized Stdlib + debug Swiftc + optimized Clang/LLVM utils/build-script --release-debuginfo --debug-swift # other flags... # debug Stdlib + optimized Swiftc + optimized Clang/LLVM utils/build-script --release-debuginfo --debug-swift-stdlib # other flags... # optimized Stdlib + debug Swiftc (expect typechecker) + optimized Clang/LLVM utils/build-script --release-debuginfo --debug-swift --force-optimized-typechecker # Last resort option, it is highly unlikely that you will need this # debug Stdlib + debug Swiftc + debug Clang/LLVM utils/build-script --debug # other flags...
Debug builds have two major drawbacks:
DebuggingTheCompiler.md goes into a LOT more detail on how you can level up your debugging skills! Make sure you check it out in case you‘re trying to debug a tricky issue and aren’t sure how to go about it.
A large number of types have dump(..)/print(..) methods which can be used along with llvm::errs() or other LLVM streams. For example, if you have a variable std::vector<CanType> canTypes that you want to print, you could do:
auto &e = llvm::errs(); e << "canTypes = ["; llvm::interleaveComma(canTypes, e, [&](auto ty) { ty.dump(e); }); e << "]\n";
You can also crash the compiler using assert/llvm_unreachable/ llvm::report_fatal_error, after accumulating the result in a stream:
std::string msg; llvm::raw_string_ostream os(msg); os << "unexpected canTypes = ["; llvm::interleaveComma(canTypes, os, [&](auto ty) { ty.dump(os); }); os << "] !!!\n"; llvm::report_fatal_error(os.str());
When the compiler crashes, the commandline arguments passed to it will be printed to stderr. It will likely look something like:
/path/to/swift-frontend <args>
Using LLDB on the commandline: Copy the entire invocation and pass it to LLDB.
lldb -- /path/to/swift-frontend <args>
Now you can use the usual LLDB commands like run, breakpoint set and so on. If you are new to LLDB, check out the official LLDB documentation and nesono's LLDB cheat sheet.
Using LLDB within Xcode: Select the current scheme ‘swift-frontend’ → Edit Scheme → Run phase → Arguments tab. Under “Arguments Passed on Launch”, copy-paste the <args> and make sure that “Expand Variables Based On” is set to swift-frontend. Close the scheme editor. If you now run the compiler (⌘+R or Product → Run), you will be able to use the Xcode debugger.
Xcode also has the ability to attach to and debug Swift processes launched elsewhere. Under Debug → Attach to Process by PID or name..., you can enter a compiler process's PID or name (swift-frontend) to debug a compiler instance invoked elsewhere. This can be helpful if you have a single compiler process being invoked by another tool, such as SwiftPM or another open Xcode project.
Pro Tip: Xcode 12's terminal does not support colors, so you may see explicit color codes printed by
dump()methods on various types. To avoid color codes in dumped output, runexpr llvm::errs().enable_color(false).
Make sure you check out the following resources:
If you see mistakes in the documentation (including typos, not just major errors) or identify gaps that you could potentially improve the contributing experience, please start a discussion on the forums, submit a pull request or file a bug report on Swift JIRA. Thanks!