| :orphan: |
| |
| .. highlight:: none |
| |
| ============================ |
| Debugging the Swift Compiler |
| ============================ |
| |
| .. contents:: |
| |
| Abstract |
| -------- |
| |
| This document contains some useful information for debugging the |
| Swift compiler and Swift compiler output. |
| |
| Basic Utilities |
| --------------- |
| |
| Often, the first step to debug a compiler problem is to re-run the compiler |
| with a command line, which comes from a crash trace or a build log. |
| |
| The script ``split-cmdline`` in ``utils/dev-scripts`` splits a command line |
| into multiple lines. This is helpful to understand and edit long command lines. |
| |
| Printing the Intermediate Representations |
| ----------------------------------------- |
| |
| The most important thing when debugging the compiler is to examine the IR. |
| Here is how to dump the IR after the main phases of the Swift compiler |
| (assuming you are compiling with optimizations enabled): |
| |
| #. **Parser**. To print the AST after parsing:: |
| |
| swiftc -dump-ast -O file.swift |
| |
| #. **SILGen**. To print the SIL immediately after SILGen:: |
| |
| swiftc -emit-silgen -O file.swift |
| |
| #. **Mandatory SIL passes**. To print the SIL after the mandatory passes:: |
| |
| swiftc -emit-sil -Onone file.swift |
| |
| Well, this is not quite true, because the compiler is running some passes |
| for -Onone after the mandatory passes, too. But for most purposes you will |
| get what you want to see. |
| |
| #. **Performance SIL passes**. To print the SIL after the complete SIL |
| optimization pipeline:: |
| |
| swiftc -emit-sil -O file.swift |
| |
| #. **IRGen**. To print the LLVM IR after IR generation:: |
| |
| swiftc -emit-ir -Xfrontend -disable-llvm-optzns -O file.swift |
| |
| 4. **LLVM passes**. To print the LLVM IR after LLVM passes:: |
| |
| swiftc -emit-ir -O file.swift |
| |
| 5. **Code generation**. To print the final generated code:: |
| |
| swiftc -S -O file.swift |
| |
| Compilation stops at the phase where you print the output. So if you want to |
| print the SIL *and* the LLVM IR, you have to run the compiler twice. |
| The output of all these dump options (except ``-dump-ast``) can be redirected |
| with an additional ``-o <file>`` option. |
| |
| Debugging the Type Checker |
| -------------------------- |
| |
| Enabling Logging |
| ~~~~~~~~~~~~~~~~ |
| |
| To enable logging in the type checker, use the following argument: ``-Xfrontend -debug-constraints``. |
| This will cause the typechecker to log its internal state as it solves |
| constraints and present the final type checked solution, e.g.:: |
| |
| ---Constraint solving for the expression at [test.swift:3:10 - line:3:10]--- |
| ---Initial constraints for the given expression--- |
| (integer_literal_expr type='$T0' location=test.swift:3:10 range=[test.swift:3:10 - line:3:10] value=0) |
| Score: 0 0 0 0 0 0 0 0 0 0 0 0 0 |
| Contextual Type: Int |
| Type Variables: |
| #0 = $T0 [inout allowed] |
| |
| Active Constraints: |
| |
| Inactive Constraints: |
| $T0 literal conforms to ExpressibleByIntegerLiteral [[locator@0x7ffa3a865a00 [IntegerLiteral@test.swift:3:10]]]; |
| $T0 conv Int [[locator@0x7ffa3a865a00 [IntegerLiteral@test.swift:3:10]]]; |
| ($T0 literal=3 bindings=(subtypes of) (default from ExpressibleByIntegerLiteral) Int) |
| Active bindings: $T0 := Int |
| (trying $T0 := Int |
| (found solution 0 0 0 0 0 0 0 0 0 0 0 0 0) |
| ) |
| ---Solution--- |
| Fixed score: 0 0 0 0 0 0 0 0 0 0 0 0 0 |
| Type variables: |
| $T0 as Int |
| |
| Overload choices: |
| |
| Constraint restrictions: |
| |
| Disjunction choices: |
| |
| Conformances: |
| At locator@0x7ffa3a865a00 [IntegerLiteral@test.swift:3:10] |
| (normal_conformance type=Int protocol=ExpressibleByIntegerLiteral lazy |
| (normal_conformance type=Int protocol=_ExpressibleByBuiltinIntegerLiteral lazy)) |
| (found solution 0 0 0 0 0 0 0 0 0 0 0 0 0) |
| ---Type-checked expression--- |
| (call_expr implicit type='Int' location=test.swift:3:10 range=[test.swift:3:10 - line:3:10] arg_labels=_builtinIntegerLiteral: |
| (constructor_ref_call_expr implicit type='(_MaxBuiltinIntegerType) -> Int' location=test.swift:3:10 range=[test.swift:3:10 - line:3:10] |
| (declref_expr implicit type='(Int.Type) -> (_MaxBuiltinIntegerType) -> Int' location=test.swift:3:10 range=[test.swift:3:10 - line:3:10] decl=Swift.(file).Int.init(_builtinIntegerLiteral:) function_ref=single) |
| (type_expr implicit type='Int.Type' location=test.swift:3:10 range=[test.swift:3:10 - line:3:10] typerepr='Int')) |
| (tuple_expr implicit type='(_builtinIntegerLiteral: Int2048)' location=test.swift:3:10 range=[test.swift:3:10 - line:3:10] names=_builtinIntegerLiteral |
| (integer_literal_expr type='Int2048' location=test.swift:3:10 range=[test.swift:3:10 - line:3:10] value=0))) |
| |
| When using the integrated swift-repl, one can dump the same output for each |
| expression as one evaluates the expression by enabling constraints debugging by |
| typing ``:constraints debug on``:: |
| |
| $ swift -frontend -repl -enable-objc-interop -module-name REPL |
| *** You are running Swift's integrated REPL, *** |
| *** intended for compiler and stdlib *** |
| *** development and testing purposes only. *** |
| *** The full REPL is built as part of LLDB. *** |
| *** Type ':help' for assistance. *** |
| (swift) :constraints debug on |
| |
| Asserting on First Error |
| ~~~~~~~~~~~~~~~~~~~~~~~~ |
| |
| When changing the typechecker, one can cause a series of cascading errors. Since |
| Swift doesn't assert on such errors, one has to know more about the typechecker |
| to know where to stop in the debugger. Rather than doing that, one can use the |
| option ``-Xllvm -swift-diagnostics-assert-on-error=1`` to cause the |
| DiagnosticsEngine to assert upon the first error, providing the signal that the |
| debugger needs to know that it should attach. |
| |
| Debugging on SIL Level |
| ---------------------- |
| |
| Options for Dumping the SIL |
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
| |
| Often it is not sufficient to dump the SIL at the beginning or end of the |
| optimization pipeline. |
| The SILPassManager supports useful options to dump the SIL also between |
| pass runs. |
| |
| The option ``-Xllvm -sil-print-all`` dumps the whole SIL module after all |
| passes. Although it prints only functions which were changed by a pass, the |
| output can get *very* large. |
| |
| It is useful if you identified a problem in the final SIL and you want to |
| check which pass did introduce the wrong SIL. |
| |
| There are several other options available, e.g. to filter the output by |
| function names (``-Xllvm -sil-print-only-function``/``s``) or by pass names |
| (``-Xllvm -sil-print-before``/``after``/``around``). |
| For details see ``PassManager.cpp``. |
| |
| Dumping the SIL and other Data in LLDB |
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
| |
| When debugging the Swift compiler with LLDB (or Xcode, of course), there is |
| even a more powerful way to examine the data in the compiler, e.g. the SIL. |
| Following LLVM's dump() convention, many SIL classes (as well as AST classes) |
| provide a dump() function. You can call the dump function with LLDB's |
| ``expression --`` or ``print`` or ``p`` command. |
| |
| For example, to examine a SIL instruction:: |
| |
| (lldb) p Inst->dump() |
| %12 = struct_extract %10 : $UnsafeMutablePointer<X>, #UnsafeMutablePointer._rawValue // user: %13 |
| |
| To dump a whole function at the beginning of a function pass:: |
| |
| (lldb) p getFunction()->dump() |
| |
| SIL modules and even functions can get very large. Often it is more convenient |
| to dump their contents into a file and open the file in a separate editor. |
| This can be done with:: |
| |
| (lldb) p getFunction()->dump("myfunction.sil") |
| |
| You can also dump the CFG (control flow graph) of a function:: |
| |
| (lldb) p Func->viewCFG() |
| |
| This opens a preview window containing the CFG of the function. To continue |
| debugging press <CTRL>-C on the LLDB prompt. |
| Note that this only works in Xcode if the PATH variable in the scheme's |
| environment setting contains the path to the dot tool. |
| |
| Debugging and Profiling on SIL level |
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
| |
| The compiler provides a way to debug and profile on SIL level. To enable SIL |
| debugging add the front-end option -gsil together with -g. Example:: |
| |
| swiftc -g -Xfrontend -gsil -O test.swift -o a.out |
| |
| This writes the SIL after optimizations into a file and generates debug info |
| for it. In the debugger and profiler you can then see the SIL code instead of |
| the Swift source code. |
| For details see the SILDebugInfoGenerator pass. |
| |
| To enable SIL debugging and profiling for the Swift standard library, use |
| the build-script-impl option ``--build-sil-debugging-stdlib``. |
| |
| ViewCFG: Regex based CFG Printer |
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
| |
| ViewCFG (``./utils/viewcfg``) is a script that parses a textual CFG (e.g. a llvm |
| or sil function) and displays a .dot file of the CFG. Since the parsing is done |
| using regular expressions (i.e. ignoring language semantics), ViewCFG can: |
| |
| 1. Parse both SIL and LLVM IR |
| 2. Parse blocks and functions without needing to know contextual |
| information. Ex: types and declarations. |
| |
| The script assumes that the relevant text is passed in via stdin and uses open |
| to display the .dot file. |
| |
| Additional, both emacs and vim integration is provided. For vim integration add |
| the following commands to your .vimrc:: |
| |
| com! -nargs=? Funccfg silent ?{$?,/^}/w !viewcfg <args> |
| com! -range -nargs=? Viewcfg silent <line1>,<line2>w !viewcfg <args> |
| |
| This will add:: |
| |
| :Funccfg displays the CFG of the current SIL/LLVM function. |
| :<range>Viewcfg displays the sub-CFG of the selected range. |
| |
| For emacs users, we provide in sil-mode (``./utils/sil-mode.el``) the function:: |
| |
| sil-mode-display-function-cfg |
| |
| To use this feature, placed the point in the sil function that you want viewcfg |
| to graph and then run ``sil-mode-display-function-cfg``. This will cause viewcfg |
| to be invoked with the sil function body. Note, |
| ``sil-mode-display-function-cfg`` does not take any arguments. |
| |
| **NOTE** viewcfg must be in the $PATH for viewcfg to work. |
| |
| **NOTE** Since we use open, .dot files should be associated with the Graphviz app for viewcfg to work. |
| |
| There is another useful script to view the CFG of a disassembled function: |
| ``./utils/dev-scripts/blockifyasm``. |
| It splits a disassembled function up into basic blocks which can then be |
| used with viewcfg:: |
| |
| (lldb) disassemble |
| <copy-paste output to file.s> |
| $ blockifyasm < file.s | viewcfg |
| |
| Using Breakpoints |
| ~~~~~~~~~~~~~~~~~ |
| |
| LLDB has very powerful breakpoints, which can be utilized in many ways to debug |
| the compiler and Swift executables. The examples in this section show the LLDB |
| command lines. In Xcode you can set the breakpoint properties by clicking 'Edit |
| breakpoint'. |
| |
| Let's start with a simple example: sometimes you see a function in the SIL |
| output and you want to know where the function was created in the compiler. |
| In this case you can set a conditional breakpoint in SILFunction constructor |
| and check for the function name in the breakpoint condition:: |
| |
| (lldb) br set -c 'hasName("_TFC3nix1Xd")' -f SILFunction.cpp -l 91 |
| |
| Sometimes you may want to know which optimization inserts, removes or moves a |
| certain instruction. To find out, set a breakpoint in |
| ``ilist_traits<SILInstruction>::addNodeToList`` or |
| ``ilist_traits<SILInstruction>::removeNodeFromList``, which are defined in |
| ``SILInstruction.cpp``. |
| The following command sets a breakpoint which stops if a ``strong_retain`` |
| instruction is removed:: |
| |
| (lldb) br set -c 'I->getKind() == ValueKind::StrongRetainInst' -f SILInstruction.cpp -l 63 |
| |
| The condition can be made more precise e.g. by also testing in which function |
| this happens:: |
| |
| (lldb) br set -c 'I->getKind() == ValueKind::StrongRetainInst && |
| I->getFunction()->hasName("_TFC3nix1Xd")' |
| -f SILInstruction.cpp -l 63 |
| |
| Let's assume the breakpoint hits somewhere in the middle of compiling a large |
| file. This is the point where the problem appears. But often you want to break |
| a little bit earlier, e.g. at the entrance of the optimization's ``run`` |
| function. |
| |
| To achieve this, set another breakpoint and add breakpoint commands:: |
| |
| (lldb) br set -n GlobalARCOpts::run |
| Breakpoint 2 |
| (lldb) br com add 2 |
| > p int $n = $n + 1 |
| > c |
| > DONE |
| |
| Run the program (this can take quite a bit longer than before). When the first |
| breakpoint hits see what value $n has:: |
| |
| (lldb) p $n |
| (int) $n = 5 |
| |
| Now remove the breakpoint commands from the second breakpoint (or create a new |
| one) and set the ignore count to $n minus one:: |
| |
| (lldb) br delete 2 |
| (lldb) br set -i 4 -n GlobalARCOpts::run |
| |
| Run your program again and the breakpoint hits just before the first breakpoint. |
| |
| Another method for accomplishing the same task is to set the ignore count of the |
| breakpoint to a large number, i.e.:: |
| |
| (lldb) br set -i 9999999 -n GlobalARCOpts::run |
| |
| Then whenever the debugger stops next time (due to hitting another |
| breakpoint/crash/assert) you can list the current breakpoints:: |
| |
| (lldb) br list |
| 1: name = 'GlobalARCOpts::run', locations = 1, resolved = 1, hit count = 85 Options: ignore: 1 enabled |
| |
| which will then show you the number of times that each breakpoint was hit. In |
| this case, we know that ``GlobalARCOpts::run`` was hit 85 times. So, now |
| we know to ignore swift_getGenericMetadata 84 times, i.e.:: |
| |
| (lldb) br set -i 84 -n GlobalARCOpts::run |
| |
| LLDB Scripts |
| ~~~~~~~~~~~~ |
| |
| LLDB has powerful capabilities of scripting in Python among other languages. An |
| often overlooked, but very useful technique is the -s command to lldb. This |
| essentially acts as a pseudo-stdin of commands that lldb will read commands |
| from. Each time lldb hits a stopping point (i.e. a breakpoint or a |
| crash/assert), it will run the earliest command that has not been run yet. As an |
| example of this consider the following script (which without any loss of |
| generality will be called test.lldb):: |
| |
| env DYLD_INSERT_LIBRARIES=/usr/lib/libgmalloc.dylib |
| break set -n swift_getGenericMetadata |
| break mod 1 -i 83 |
| process launch -- --stdlib-unittest-in-process --stdlib-unittest-filter "DefaultedForwardMutableCollection<OpaqueValue<Int>>.Type.subscript(_: Range)/Set/semantics" |
| break set -l 224 |
| c |
| expr pattern->CreateFunction |
| break set -a $0 |
| c |
| dis -f |
| |
| TODO: Change this example to apply to the Swift compiler instead of to the |
| stdlib unittests. |
| |
| Then by running ``lldb test -s test.lldb``, lldb will: |
| |
| 1. Enable guard malloc. |
| 2. Set a break point on swift_getGenericMetadata and set it to be ignored for 83 hits. |
| 3. Launch the application and stop at swift_getGenericMetadata after 83 hits have been ignored. |
| 4. In the same file as swift_getGenericMetadata introduce a new breakpoint at line 224 and continue. |
| 5. When we break at line 224 in that file, evaluate an expression pointer. |
| 6. Set a breakpoint at the address of the expression pointer and continue. |
| 7. When we hit the breakpoint set at the function pointer's address, disassemble |
| the function that the function pointer was passed to. |
| |
| Using LLDB scripts can enable one to use complex debugger workflows without |
| needing to retype the various commands perfectly every time. |
| |
| Custom LLDB Commands |
| ~~~~~~~~~~~~~~~~~~~~ |
| |
| If you've ever found yourself repeatedly entering a complex sequence of |
| commands within a debug session, consider using custom lldb commands. Custom |
| commands are a handy way to automate debugging tasks. |
| |
| For example, say we need a command that prints the contents of the register |
| ``rax`` and then steps to the next instruction. Here's how to define that |
| command within a debug session:: |
| |
| (lldb) script |
| Python Interactive Interpreter. To exit, type 'quit()', 'exit()' or Ctrl-D. |
| >>> def custom_step(): |
| ... print "rax =", lldb.frame.FindRegister("rax") |
| ... lldb.thread.StepInstruction(True) |
| ... |
| >>> ^D |
| |
| You can call this function using the ``script`` command, or via an alias:: |
| |
| (lldb) script custom_step() |
| rax = ... |
| <debugger steps to the next instruction> |
| |
| (lldb) command alias cs script custom_step() |
| (lldb) cs |
| rax = ... |
| <debugger steps to the next instruction> |
| |
| Printing registers and single-stepping are by no means the only things you can |
| do with custom commands. The LLDB Python API surfaces a lot of useful |
| functionality, such as arbitrary expression evaluation. |
| |
| There are some pre-defined custom commands which can be especially useful while |
| debugging the swift compiler. These commands live in |
| ``swift/utils/lldb/lldbToolBox.py``. There is a wrapper script available in |
| ``SWIFT_BINARY_DIR/bin/lldb-with-tools`` which launches lldb with those |
| commands loaded. |
| |
| A command named ``sequence`` is included in lldbToolBox. ``sequence`` runs |
| multiple semicolon separated commands together as one command. This can be used |
| to define custom commands using just other lldb commands. For example, |
| ``custom_step()`` function defined above could be defined as:: |
| |
| (lldb) command alias cs sequence p/x $rax; stepi |
| |
| Reducing SIL test cases using bug_reducer |
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
| |
| There is functionality provided in ./swift/utils/bug_reducer/bug_reducer.py for |
| reducing SIL test cases by: |
| |
| 1. Producing intermediate sib files that only require some of the passes to |
| trigger the crasher. |
| 2. Reducing the size of the sil test case by extracting functions or |
| partitioning a module into unoptimized and optimized modules. |
| |
| For more information and a high level example, see: |
| ./swift/utils/bug_reducer/README.md. |
| |
| |
| Debugging Swift Executables |
| --------------------------- |
| |
| One can use the previous tips for debugging the Swift compiler with Swift |
| executables as well. Here are some additional useful techniques that one can use |
| in Swift executables. |
| |
| Determining the mangled name of a function in LLDB |
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
| |
| One problem that often comes up when debugging Swift code in LLDB is that LLDB |
| shows the demangled name instead of the mangled name. This can lead to mistakes |
| where due to the length of the mangled names one will look at the wrong |
| function. Using the following command, one can find the mangled name of the |
| function in the current frame:: |
| |
| (lldb) image lookup -va $pc |
| Address: CollectionType3[0x0000000100004db0] (CollectionType3.__TEXT.__text + 16000) |
| Summary: CollectionType3`ext.CollectionType3.CollectionType3.MutableCollectionType2<A where A: CollectionType3.MutableCollectionType2>.(subscript.materializeForSet : (Swift.Range<A.Index>) -> Swift.MutableSlice<A>).(closure #1) |
| Module: file = "/Volumes/Files/work/solon/build/build-swift/validation-test-macosx-x86_64/stdlib/Output/CollectionType.swift.gyb.tmp/CollectionType3", arch = "x86_64" |
| Symbol: id = {0x0000008c}, range = [0x0000000100004db0-0x00000001000056f0), name="ext.CollectionType3.CollectionType3.MutableCollectionType2<A where A: CollectionType3.MutableCollectionType2>.(subscript.materializeForSet : (Swift.Range<A.Index>) -> Swift.MutableSlice<A>).(closure #1)", mangled="_TFFeRq_15CollectionType322MutableCollectionType2_S_S0_m9subscriptFGVs5Rangeqq_s16MutableIndexable5Index_GVs12MutableSliceq__U_FTBpRBBRQPS0_MS4__T_" |
| |
| Debugging failures in LLDB |
| ~~~~~~~~~~~~~~~~~~~~~~~~~~ |
| |
| Sometimes one needs to be able to while debugging actually debug LLDB and its |
| interaction with Swift itself. Some examples of problems where this can come up |
| are: |
| |
| 1. Compiler bugs when LLDB attempts to evaluate an expression. (expression |
| debugging) |
| 2. Swift variables being shown with no types. (type debugging) |
| |
| To gain further insight into these sorts of failures, we use LLDB log |
| categories. LLDB log categories provide introspection by causing LLDB to dump |
| verbose information relevant to the category into the log as it works. The two |
| log channels that are useful for debugging Swift issues are the "types" and |
| "expression" log channels. |
| |
| For more details about any of the information below, please run:: |
| |
| (lldb) help log enable |
| |
| "Types" Log |
| ``````````` |
| |
| The "types" log reports on LLDB's process of constructing SwiftASTContexts and |
| errors that may occur. The two main tasks here are: |
| |
| 1. Constructing the SwiftASTContext for a specific single Swift module. This is |
| used to implement frame local variable dumping via the lldb ``frame |
| variable`` command, as well as the Xcode locals view. On failure, local |
| variables will not have types. |
| |
| 2. Building a SwiftASTContext in which to run Swift expressions using the |
| "expression" command. Upon failure, one will see an error like: "Shared Swift |
| state for has developed fatal errors and is being discarded." |
| |
| These errors can be debugged by turning on the types log:: |
| |
| (lldb) log enable -f /tmp/lldb-types-log.txt lldb types |
| |
| That will write the types log to the file passed to the -f option. |
| |
| **NOTE** Module loading can happen as a side-effect of other operations in lldb |
| (e.g. the "file" command). To be sure that one has enabled logging before /any/ |
| module loading has occurred, place the command into either:: |
| |
| ~/.lldbinit |
| $PWD/.lldbinit |
| |
| This will ensure that the type import command is run before /any/ modules are |
| imported. |
| |
| "Expression" Log |
| ```````````````` |
| |
| The "expression" log reports on the process of wrapping, parsing, SILGen'ing, |
| JITing, and inserting an expression into the current Swift module. Since this can |
| only be triggered by the user manually evaluating expression, this can be turned |
| on at any point before evaluating an expression. To enable expression logging, |
| first run:: |
| |
| (lldb) log enable -f /tmp/lldb-expr-log.txt lldb expression |
| |
| and then evaluate the expression. The expression log dumps, in order, the |
| following non-exhaustive list of state: |
| |
| 1. The unparsed, textual expression passed to the compiler. |
| 2. The parsed expression. |
| 3. The initial SILGen. |
| 4. SILGen after SILLinking has occurred. |
| 5. SILGen after SILLinking and Guaranteed Optimizations have occurred. |
| 6. The resulting LLVM IR. |
| 7. The assembly code that will be used by the JIT. |
| |
| **NOTE** LLDB runs a handful of preparatory expressions that it uses to set up |
| for running Swift expressions. These can make the expression logs hard to read |
| especially if one evaluates multiple expressions with the logging enabled. In |
| such a situation, run all expressions before the bad expression, turn on the |
| logging, and only then run the bad expression. |
| |
| Multiple Logs at a Time |
| ``````````````````````` |
| |
| Note, you can also turn on more than one log at a time as well, e.x.:: |
| |
| (lldb) log enable -f /tmp/lldb-types-log.txt lldb types expression |