commit | ce562774a7a251a378b027d4e442cba3f9df3e0e | [log] [tgz] |
---|---|---|
author | Dan Zheng <danielzheng@google.com> | Wed Sep 09 16:14:39 2020 -0700 |
committer | GitHub <noreply@github.com> | Wed Sep 09 16:14:39 2020 -0700 |
tree | 8ab7a4212b2f6f81902331a26606a11ae736ea59 | |
parent | 39c6ee112f9598042c5659d4328429beeb2007ca [diff] |
[AutoDiff] Fix forward-mode crashes related to tangent buffers (#33868) Fixes foward-mode crashes related to: - Missing tangent buffers for non-wrt `inout` parameters. - Tangent buffers not being initialized due to the corresponding original buffer intialization instructions being non-active. - Non-varied indirect results not being initialized. - `emitDestroyValue` crashes due to `TangentVector` value category mismatch. Resolves TF-984 and SR-13447. Co-authored-by: Alex Efremov <29282858+efremale@users.noreply.github.com>
OS | CI platform | x86_64 | GPU |
---|---|---|---|
macOS | Google Kokoro | - | |
Ubuntu 16.04 | Swift.org CI |
Architecture | Master | Package | |
---|---|---|---|
macOS | x86_64 | ||
Ubuntu 16.04 | x86_64 | ||
Ubuntu 18.04 | x86_64 | ||
Ubuntu 20.04 | x86_64 | ||
CentOS 8 | x86_64 | ||
Amazon Linux 2 | x86_64 |
Swift Community-Hosted CI Platforms
OS | Architecture | Build |
---|---|---|
Ubuntu 16.04 | PPC64LE | |
Ubuntu 18.04 | AArch64 | |
Ubuntu 20.04 | AArch64 | |
CentOS 8 | AArch64 | |
Amazon Linux 2 | AArch64 | |
Android | ARMv7 | |
Android | AArch64 | |
Windows 2019 (VS 2017) | x86_64 | |
Windows 2019 (VS 2019) | x86_64 |
Swift TensorFlow Community-Hosted CI Platforms
OS | Architecture | Build |
---|---|---|
Ubuntu 16.04 | x86_64 | |
macOS 10.13 | x86_64 | |
Ubuntu 16.04 (GPU) | x86_64 |
Swift for TensorFlow is a new programming language for TensorFlow. It is a copy of the compiler for the Swift Programming Language that adds first-class compiler and language support for machine learning.
This repository covers the compiler and standard libraries. Please visit the documentation repository for more information about the project, including a project overview, technical details, and guidelines for contributing. To use Swift for TensorFlow out of the box, follow the installation instructions. To build from source, follow the instructions below.
Swift is a high-performance system programming language. It has a clean and modern syntax, offers seamless access to existing C and Objective-C code and frameworks, and is memory safe by default.
Although inspired by Objective-C and many other languages, Swift is not itself a C-derived language. As a complete and independent language, Swift packages core features like flow control, data structures, and functions, with high-level constructs like objects, protocols, closures, and generics. Swift embraces modules, eliminating the need for headers and the code duplication they entail.
To learn more about the programming language, visit swift.org.
Contributions to Swift are welcomed and encouraged! Please see the Contributing to Swift guide.
To be a truly great community, Swift.org needs to welcome developers from all walks of life, with different backgrounds, and with a wide range of experience. A diverse and friendly community will have more great ideas, more unique perspectives, and produce more great code. We will work diligently to make the Swift community welcoming to everyone.
To give clarity of what is expected of our members, Swift has adopted the code of conduct defined by the Contributor Covenant. This document is used across many open source communities, and we think it articulates our values well. For more, see the Code of Conduct.
These instructions give the most direct path to a working Swift development environment. To build from source you will need about 2 GB of disk space for the source code and up to 70 GB of disk space for the build artifacts with full debugging. Depending on your machine, a clean build can take a few minutes to several hours. Naturally, incremental builds are much faster.
Once you are able to build things successfully and have a compile-test-debug loop going, check out the development tips for better productivity while working on the compiler.
You can also skim docs/README.md to understand what high-level documentation is available.
macOS and Ubuntu Linux LTS 18.04 are the current supported host development operating systems.
Please make sure you use Python 2.x. Python 3.x is not supported currently.
To build for macOS, you need Xcode 12 beta 3. The required version of Xcode changes frequently, and is often a beta release. Check this document for the current required version.
Swift‘s build tooling is meant to support spaces in the paths passed to them, but using spaces sometimes tickles bugs in Swift’s build scripts or the tools they rely on. For example, SR-13441 is caused by a space in the Xcode path used on macOS. If you see Swift's build tooling misbehave due to a space in a path, please report the bug on the Swift bug tracker and then change the path to work around it.
You will also need CMake and Ninja, and Bazel, which can be installed via a package manager:
brew install cmake ninja
You can also use homebrew-bundle from the root of this repository's working directory to install all of these dependencies:
brew bundle
sudo port install cmake ninja
Additionally, Bazel is required to build with TensorFlow support. Instructions to download Bazel directly can be found below. You can find instructions for installing CMake, and Ninja directly below as well.
For Ubuntu, you'll need the following development dependencies:
sudo apt-get install \ clang \ cmake \ git \ icu-devtools \ libcurl4-openssl-dev \ libedit-dev \ libicu-dev \ libncurses5-dev \ libpython-dev \ libsqlite3-dev \ libxml2-dev \ ninja-build \ pkg-config \ python \ python-six \ rsync \ swig \ systemtap-sdt-dev \ tzdata \ uuid-dev
Note: LLDB currently requires at least swig-1.3.40
but will successfully build with version 2 shipped with Ubuntu.
Note: For Ubuntu 20.04, use libpython2-dev
in place of the libpython-dev package above.
First, create a directory for all of the Swift sources:
mkdir swift-source cd swift-source
Note: This is important since update-checkout (see below) checks out repositories next to the Swift source directory. This means that if one clones Swift and has other unrelated repositories, update-checkout may not clone those repositories and will update them instead. Be aware that update-checkout
currently does not support paths with non-ASCII characters. If such characters are present in the path to swift-source
, update-checkout
will fail.
TensorFlow Support: To build with TensorFlow support, the tensorflow
scheme must be specified when cloning sources. The tensorflow
scheme pins specific versions of every Swift companion directory and is updated with every upstream merge from the master branch.
Via HTTPS For those checking out sources as read-only, HTTPS works best:
git clone https://github.com/apple/swift.git -b tensorflow ./swift/utils/update-checkout --clone --scheme tensorflow cd swift
Via SSH For those who plan on regularly making direct commits, cloning over SSH may provide a better experience (which requires uploading SSH keys to GitHub):
git clone git@github.com:apple/swift.git -b tensorflow ./swift/utils/update-checkout --clone-with-ssh --scheme tensorflow cd swift
The build-script
is a high-level build automation script that supports basic options such as building a Swift-compatible LLDB, building the Swift Package Manager, building for various platforms, running tests after builds, and more.
There are two primary build systems to use: Xcode and Ninja. The Xcode build system allows you to work in Xcode, but Ninja is a bit faster and supports more environments.
First, make sure that you're in the swift directory:
cd swift
To build using Ninja, run:
swift/utils/build-script --release-debuginfo
When developing Swift, it helps to build what you're working on in a debug configuration while building the rest of the project with optimizations. Below are some examples of using debug variants:
swift/utils/build-script --release-debuginfo --debug-swift # Swift frontend built in debug swift/utils/build-script --release-debuginfo --debug-swift-stdlib # Standard library built in debug swift/utils/build-script --release-debuginfo --debug-swift --force-optimized-typechecker # Swift frontend sans type checker built in debug
Limiting the amount of debug code in the compiler has a very large impact on Swift compile times, and in turn the test execution time. If you want to build the entire project in debug, you can run:
swift/utils/build-script --debug
For documentation of all available arguments, as well as additional usage information, see the inline help:
utils/build-script -h
To build using Xcode, specify the --xcode
argument on any of the above commands. Xcode can be used to edit the Swift source code, but it is not currently fully supported as a build environment for SDKs other than macOS. The generated Xcode project does not integrate with the test runner, but the tests can be run with the ‘check-swift’ target.
All of the build products are placed in swift-source/build/${TOOL}-${MODE}/${PRODUCT}-${PLATFORM}/
. If macOS Swift with Ninja in DebugAssert mode was built, all of the products would be in swift-source/build/Ninja-DebugAssert/swift-macosx-x86_64/
. It helps to save this directory as an environment variable for future use.
export SWIFT_BUILD_DIR="~/swift-source/build/Ninja-DebugAssert/swift-macosx-x86_64"
Once the first build has completed, Ninja can perform fast incremental builds of various products. These incremental builds are a big timesaver when developing and debugging.
cd ${SWIFT_BUILD_DIR} ninja swift-frontend
This will build the Swift compiler, but will not rebuild the standard library or any other target. Building the swift-stdlib
target as an additional layer of testing from time to time is also a good idea. To build just the standard library, run:
ninja swift-stdlib
It is always a good idea to do a full build after using update-checkout
.
To open the Swift project in Xcode, open ${SWIFT_BUILD_DIR}/Swift.xcodeproj
. It will auto-create a lot of schemes for all of the available targets. A common debug flow would involve:
Another option is to change the scheme to “Wait for executable to be launched”, then run the build product in Terminal.
Swift for TensorFlow toolchains are created using the script build-toolchain-tensorflow. This script is used by swift.org's CI to produce snapshots and can allow for one to locally reproduce such builds for development or distribution purposes. A typical invocation looks like the following:
$ ./swift/utils/build-toolchain-tensorflow $BUNDLE_PREFIX
where $BUNDLE_PREFIX
is a string that will be prepended to the build date to give the bundle identifier of the toolchain's Info.plist
. For instance, if $BUNDLE_PREFIX
was com.example
, the toolchain produced will have the bundle identifier com.example.YYYYMMDD
. It will be created in the directory you run the script with a filename of the form: swift-tensorflow-LOCAL-YYYY-MM-DD-a-osx.tar.gz
.
Beyond building the toolchain, build-toolchain-tensorflow
also supports the following (non-exhaustive) set of useful options:
--dry-run
: Perform a dry run build. This is off by default.--test
: Test the toolchain after it has been compiled. This is off by default.--pkg
(macOS only): Build a toolchain installer package (.pkg
). This is off by default.More options may be added over time. Please pass --help
to build-toolchain-tensorflow
to see the full set of options.
On macOS if one wants to install such a toolchain into Xcode:
/Library/Developer/Toolchains/
or ~/Library/Developer/Toolchains/
. E.x.:$ sudo tar -xzf swift-LOCAL-YYYY-MM-DD-a-osx.tar.gz -C / $ tar -xzf swift-LOCAL-YYYY-MM-DD-a-osx.tar.gz -C ~/
The script also generates an archive containing debug symbols which can be installed over the main archive allowing symbolication of any compiler crashes.
$ sudo tar -xzf swift-LOCAL-YYYY-MM-DD-a-osx-symbols.tar.gz -C / $ tar -xzf swift-LOCAL-YYYY-MM-DD-a-osx-symbols.tar.gz -C ~/
Xcode->Toolchains
.Make sure you are using the correct release of Xcode.
If you have changed Xcode versions but still encounter errors that appear to be related to the Xcode version, try passing --clean
to build-script
.
When a new version of Xcode is released, you can update your build without recompiling the entire project by passing the --reconfigure
option.
Make sure all repositories are up to date with the update-checkout
command described above.
See docs/Testing.md, in particular the section on lit.py.
Be sure to look through the docs directory for more information about the compiler. In particular, the documents titled Debugging the Swift Compiler and Continuous Integration for Swift are very helpful to understand before submitting your first PR.
To read the compiler documentation, start by installing the Sphinx documentation generator tool by running the command:
easy_install -U "Sphinx < 2.0"
Once complete, you can build the Swift documentation by changing directory into docs and typing make
. This compiles the .rst
files in the docs directory into HTML in the docs/_build/html
directory.
Many of the docs are out of date, but you can see some historical design documents in the docs
directory.
Another source of documentation is the standard library itself, located in stdlib
. Much of the language is actually implemented in the library (including Int
), and the standard library gives some examples of what can be expressed today.
CMake is the core infrastructure used to configure builds of Swift and its companion projects; at least version 3.16.5 is required.
On macOS, you can download the CMake Binary Distribution, bundled as an application, copy it to /Applications
, and add the embedded command line tools to your PATH
:
export PATH=/Applications/CMake.app/Contents/bin:$PATH
On Linux, if you have not already installed Swift's development dependencies, you can download and install the CMake package separately using the following command:
sudo apt-get install cmake
Ninja is the current recommended build system for building Swift and is the default configuration generated by CMake. Pre-built packages are available for macOS and Linux distributions. You can also clone Ninja next to the other projects and it will be bootstrapped automatically:
Via HTTPS
git clone https://github.com/ninja-build/ninja.git && cd ninja git checkout release cat README
Via SSH
git clone git@github.com:ninja-build/ninja.git && cd ninja git checkout release cat README
Bazel is the build tool used to build TensorFlow. Installing Bazel is necessary for building Swift with TensorFlow support.
The Bazel website has detailed installation instructions for macOS and Ubuntu. When picking the version to download in step 2, select version 2.0.0 which can be found in the release notes here (v2.0.0).