[AutoDiff] Improve documentation for derivative function SILGen. (#27753)

Improve documentation and naming for derivative function SILGen.

SILGen generates thunks for derivative functions registered via
`@differentiable` and `@differentiating` attributes.

Currently, two SILGen derivative thunk kinds exist:
- `SILGenModule::getOrCreateAutoDiffDerivativeForwardingThunk`
  - This creates a simple thunk that forwards arguments and returns
    results. This is generated when no reabstraction or self reordering
    is necessary.
- `SILGenModule::getOrCreateAutoDiffDerivativeReabstractionThunk`
  - This creates a thunk that performs reabstraction and/or self reordering.
4 files changed
tree: 4b7858c3bee73db5aa37130133a6568e1c0ac20e
  1. .clang-format
  2. .dir-locals.el
  3. .flake8
  4. .github/
  5. .gitignore
  6. .mailmap
  7. CHANGELOG.md
  8. CMakeLists.txt
  9. CODE_OF_CONDUCT.md
  10. CODE_OWNERS.TXT
  11. CONTRIBUTING.md
  12. LICENSE.txt
  13. README.md
  14. apinotes/
  15. benchmark/
  16. bindings/
  17. cmake/
  18. docs/
  19. include/
  20. lib/
  21. stdlib/
  22. test/
  23. tools/
  24. unittests/
  25. utils/
  26. validation-test/
README.md

Swift for TensorFlow

OSCI platformx86_64GPU
macOSGoogle KokoroBuild Statuscoming soon
Ubuntu 16.04Swift.org CIBuild StatusN/A

Welcome to Swift for TensorFlow!

Swift for TensorFlow is a new programming language for TensorFlow. It is a copy of the compiler for the Swift Programming Language that adds first-class compiler and language support for machine learning.

This repository covers the compiler and standard libraries. Please visit the documentation repository for more information about the project, including a project overview, technical details, and guidelines for contributing. To use Swift for TensorFlow out of the box, follow the installation instructions. To build from source, follow the instructions below.

Note: Swift for TensorFlow is an early stage research project. It has been released to enable open source development and is not yet ready for general use by machine learning developers.

Building Swift for TensorFlow

Note: Building from source is necessary only if you want to modify the source code or build with a custom version of TensorFlow.

These instructions give the most direct path to a working Swift for TensorFlow development environment. To build from source you will need about 2 GB of disk space for the source code and up to 70 GB of disk space for the build artifacts with full debugging. Depending on your machine, a clean build can take a few minutes to several hours. Naturally, incremental builds are much faster.

System Requirements

macOS and Ubuntu Linux LTS 18.04 are the current supported host development operating systems.

Please make sure you use Python 2.x. Python 3.x is not supported currently.

macOS

To build for macOS, you need Xcode 11 beta 6. The required version of Xcode changes frequently, and is often a beta release. Check this document for the current required version.

You will also need CMake, Ninja, and Bazel, which can be installed via a package manager.

Homebrew

brew install cmake ninja
brew cask install java # required for Bazel

Additionally, Bazel between v0.24.1 and v0.25.2 (inclusive) is required to build with TensorFlow support. Instructions to download Bazel directly can be found below. You can find instructions for installing CMake, and Ninja directly below as well.

Linux

For Ubuntu, you'll need the following development dependencies:

sudo apt-get install git cmake ninja-build clang python uuid-dev libicu-dev icu-devtools libedit-dev libxml2-dev libsqlite3-dev swig libpython-dev libncurses5-dev pkg-config libcurl4-openssl-dev libblocksruntime-dev systemtap-sdt-dev tzdata rsync

Additionally, Bazel between v0.24.1 and v0.25.2 (inclusive) is required to build with TensorFlow support. Ubuntu installation instructions can be found below.

Note: LLDB currently requires at least swig-1.3.40 but will successfully build with version 2 shipped with Ubuntu.

Additional build instructions for Ubuntu 14.04 LTS can be found here. These are necessary for building Swift for TensorFlow correctly.

Getting Sources for Swift and Related Projects

First, create a directory for all of the Swift sources:

mkdir swift-source
cd swift-source

Note: This is important since update-checkout (see below) checks out repositories next to the Swift source directory. This means that if one clones Swift and has other unrelated repositories, update-checkout may not clone those repositories and will update them instead. Additionally, ensure Python 2.7 is being used for this step, whether via conda environment or other means.

TensorFlow Support: To build with TensorFlow support, the tensorflow scheme must be specified when cloning sources. The tensorflow scheme pins specific versions of every Swift companion directory and is updated with every upstream merge from the master branch.

Via HTTPS For those checking out sources as read-only, HTTPS works best:

git clone https://github.com/apple/swift.git -b tensorflow
./swift/utils/update-checkout --clone --scheme tensorflow
cd swift

Via SSH For those who plan on regularly making direct commits, cloning over SSH may provide a better experience (which requires uploading SSH keys to GitHub):

git clone git@github.com:apple/swift.git -b tensorflow
./swift/utils/update-checkout --clone-with-ssh --scheme tensorflow
cd swift

Building Swift with TensorFlow support

The build-script is a high-level build automation script that supports basic options such as building a Swift-compatible LLDB, building the Swift Package Manager, building for various platforms, running tests after builds, and more. TensorFlow support is enabled by the --enable-tensorflow flag. TensorFlow will be automatically cloned from GitHub and built from source using Bazel when this flag is specified.

There are two primary build systems to use: Xcode and Ninja. The Xcode build system allows you to work in Xcode, but Ninja is a bit faster and supports more environments.

To build using Ninja, run:

swift/utils/build-script --enable-tensorflow --release-debuginfo

When developing Swift, it helps to build what you're working on in a debug configuration while building the rest of the project with optimizations. Below are some examples of using debug variants:

swift/utils/build-script --enable-tensorflow --release-debuginfo --debug-swift # Swift frontend built in debug
swift/utils/build-script --enable-tensorflow --release-debuginfo --debug-swift-stdlib # Standard library built in debug
swift/utils/build-script --enable-tensorflow --release-debuginfo --debug-swift --force-optimized-typechecker # Swift frontend sans type checker built in debug

Limiting the amount of debug code in the compiler has a very large impact on Swift compile times, and in turn the test execution time. If you want to build the entire project in debug, you can run:

swift/utils/build-script  --enable-tensorflow --debug

For documentation of all available arguments, as well as additional usage information, see the inline help:

swift/utils/build-script -h

Customize TensorFlow support

If you want to build with custom TensorFlow headers and shared libraries, please specify the --tensorflow-host-include-dir and --tensorflow-host-lib-dir arguments:

utils/build-script --enable-tensorflow --tensorflow-host-include-dir=<path_to_tensorflow_headers> --tensorflow-host-lib-dir=<path_to_tensorflow_libraries>

You can assign specific values to these arguments after a double-dash -- in your build-script command. For example:

utils/build-script -- enable-tensorflow=True

Below is more information about TensorFlow-related build arguments.

  • enable-tensorflow: If true, enables TensorFlow support for Swift.
    • Default value: False.
  • build-tensorflow: If true, automatically clone and build TensorFlow from source.
    • Default value: If enable-tensorflow is True and tensorflow-host-lib-dir and tensorflow-host-include-dir are not specified, then True. Otherwise, False.
  • host-bazel: The absolute path to Bazel, used to build TensorFlow.
    • By default, the path is auto detected.
  • tensorflow-bazel-options: Comma separated options passed to Bazel when building TensorFlow, e.g. --copt=-mavx,--copt=-msse4.2.
    • Default: None.
  • tensorflow-host-include-dir: A directory containing custom TensorFlow headers.
    • Default value: None.
  • tensorflow-host-lib-dir: A directory containing custom TensorFlow shared libraries (libtensorflow.so).
    • Default value: None.
  • tensorflow-swift-apis: A path to the tensorflow/swift-apis deep learning library repository.

Build systems

Xcode

To build using Xcode, specify the --xcode argument on any of the above commands. Xcode can be used to edit the Swift source code, but it is not currently fully supported as a build environment for SDKs other than macOS. The generated Xcode project does not integrate with the test runner, but the tests can be run with the ‘check-swift’ target.

Build Products

All of the build products are placed in swift-source/build/${TOOL}-${MODE}/${PRODUCT}-${PLATFORM}/. If macOS Swift with Ninja in DebugAssert mode was built, all of the products would be in swift-source/build/Ninja-DebugAssert/swift-macosx-x86_64/. It helps to save this directory as an environment variable for future use.

export SWIFT_BUILD_DIR="~/swift-source/build/Ninja-DebugAssert/swift-macosx-x86_64"

Ninja

Once the first build has completed, Ninja can perform fast incremental builds of various products. These incremental builds are a big timesaver when developing and debugging.

cd ${SWIFT_BUILD_DIR}
ninja swift

This will build the Swift compiler, but will not rebuild the standard library or any other target. Building the swift-stdlib target as an additional layer of testing from time to time is also a good idea. To build just the standard library, run:

ninja swift-stdlib

It is always a good idea to do a full build after using update-checkout.

Using Xcode

To open the Swift project in Xcode, open ${SWIFT_BUILD_DIR}/Swift.xcodeproj. It will auto-create a lot of schemes for all of the available targets. A common debug flow would involve:

  • Select the ‘swift’ scheme.
  • Pull up the scheme editor (⌘⇧<).
  • Select the ‘Arguments’ tab and click the ‘+’.
  • Add the command line options.
  • Close the scheme editor.
  • Build and run.

Another option is to change the scheme to “Wait for executable to be launched”, then run the build product in Terminal.

Swift Toolchains

Building

Swift toolchains are created using the script build-toolchain-tensorflow. This script is used by swift.org's CI to produce snapshots and can allow for one to locally reproduce such builds for development or distribution purposes. E.x.:

  $ ./swift/utils/build-toolchain-tensorflow $BUNDLE_PREFIX

where $BUNDLE_PREFIX is a string that will be prepended to the build date to give the bundle identifier of the toolchain's Info.plist. For instance, if $BUNDLE_PREFIX was com.example, the toolchain produced will have the bundle identifier com.example.YYYYMMDD. It will be created in the directory you run the script with a filename of the form: swift-tensorflow-LOCAL-YYYY-MM-DD-a-osx.tar.gz.

Beyond building the toolchain, build-toolchain-tensorflow also supports the following (non-exhaustive) set of useful options:

  • --dry-run: Perform a dry run build. This is off by default.
  • --test: Test the toolchain after it has been compiled. This is off by default.
  • --gpu (Linux only): Build with GPU support. This is off by default.
  • --pkg (macOS only): Build a toolchain installer package (.pkg). This is off by default.

More options may be added over time. Please pass --help to build-toolchain-tensorflow to see the full set of options.

Installing into Xcode

On macOS if one wants to install such a toolchain into Xcode:

  1. Untar and copy the toolchain to one of /Library/Developer/Toolchains/ or ~/Library/Developer/Toolchains/. E.x.:
  $ sudo tar -xzf swift-LOCAL-YYYY-MM-DD-a-osx.tar.gz -C /
  $ tar -xzf swift-LOCAL-YYYY-MM-DD-a-osx.tar.gz -C ~/

The script also generates an archive containing debug symbols which can be installed over the main archive allowing symbolication of any compiler crashes.

  $ sudo tar -xzf swift-LOCAL-YYYY-MM-DD-a-osx-symbols.tar.gz -C /
  $ tar -xzf swift-LOCAL-YYYY-MM-DD-a-osx-symbols.tar.gz -C ~/
  1. Specify the local toolchain for Xcode's use via Xcode->Toolchains.

Build Failures

Make sure you are using the correct release of Xcode.

If you have changed Xcode versions but still encounter errors that appear to be related to the Xcode version, try passing --clean to build-script.

When a new version of Xcode is released, you can update your build without recompiling the entire project by passing the --reconfigure option.

Make sure all repositories are up to date with the update-checkout command described above.

Testing Swift

The simplest way to run the Swift test suite is using the tensorflow_test build preset, which runs the entire Swift test suite (including new TensorFlow tests):

utils/build-script --preset=tensorflow_test

Swift for TensorFlow adds the following new test suites:

Before submitting pull requests involving large code changes, please run the command above locally to ensure all tests pass.

For more details on testing, see docs/Testing.md, in particular the section on lit.py.

Build Dependencies

CMake

CMake is the core infrastructure used to configure builds of Swift and its companion projects; at least version 3.4.3 is required.

On macOS, you can download the CMake Binary Distribution, bundled as an application, copy it to /Applications, and add the embedded command line tools to your PATH:

export PATH=/Applications/CMake.app/Contents/bin:$PATH

On Linux, if you have not already installed Swift's development dependencies, you can download and install the CMake package separately using the following command:

sudo apt-get install cmake

Ninja

Ninja is the current recommended build system for building Swift and is the default configuration generated by CMake. Pre-built packages are available for macOS and Linux distributions. You can also clone Ninja next to the other projects and it will be bootstrapped automatically:

Via HTTPS

git clone https://github.com/ninja-build/ninja.git && cd ninja
git checkout release
cat README

Via SSH

git clone git@github.com:ninja-build/ninja.git && cd ninja
git checkout release
cat README

Bazel

Bazel is the build tool used to build TensorFlow. Installing Bazel is necessary for building Swift with TensorFlow support.

The Bazel website has detailed installation instructions for macOS and Ubuntu. When picking the version to download in step 2, select a version between v0.24.1 and v0.25.2 (inclusive) which can be found in the release notes here (v0.25.2).