% The Borrow Checker
WARNING: This README is more or less obsolete, and will be removed soon! The new system is described in the rustc guide.
This pass has the job of enforcing memory safety. This is a subtle topic. This docs aim to explain both the practice and the theory behind the borrow checker. They start with a high-level overview of how it works, and then proceed to dive into the theoretical background. Finally, they go into detail on some of the more subtle aspects.
These docs are long. Search for the section you are interested in.
The borrow checker checks one function at a time. It operates in two passes. The first pass, called gather_loans, walks over the function and identifies all of the places where borrows (e.g., & expressions and ref bindings) and moves (copies or captures of a linear value) occur. It also tracks initialization sites. For each borrow and move, it checks various basic safety conditions at this time (for example, that the lifetime of the borrow doesn't exceed the lifetime of the value being borrowed, or that there is no move out of an &T referent).
It then uses the dataflow module to propagate which of those borrows may be in scope at each point in the procedure. A loan is considered to come into scope at the expression that caused it and to go out of scope when the lifetime of the resulting reference expires.
Once the in-scope loans are known for each point in the program, the borrow checker walks the IR again in a second pass called check_loans. This pass examines each statement and makes sure that it is safe with respect to the in-scope loans.
Throughout the docs we'll consider a simple subset of Rust in which you can only borrow from places, defined like so:
P = x | P.f | *P
Here x represents some variable, P.f is a field reference, and *P is a pointer dereference. There is no auto-deref or other niceties. This means that if you have a type like:
struct S { f: i32 }
and a variable a: Box<S>, then the rust expression a.f would correspond to an P of (*a).f.
Here is the formal grammar for the types we'll consider:
TY = i32 | bool | S<'LT...> | Box<TY> | & 'LT MQ TY MQ = mut | imm
Most of these types should be pretty self explanatory. Here S is a struct name and we assume structs are declared like so:
SD = struct S<'LT...> { (f: TY)... }
Now, imagine we had a program like this:
struct Foo { f: i32, g: i32 } ... 'a: { let mut x: Box<Foo> = ...; let y = &mut (*x).f; x = ...; }
This is of course dangerous because mutating x will free the old value and hence invalidate y. The borrow checker aims to prevent this sort of thing.
The way the borrow checker works is that it analyzes each borrow expression (in our simple model, that's stuff like &P, though in real life there are a few other cases to consider). For each borrow expression, it computes a Loan, which is a data structure that records (1) the value being borrowed, (2) the mutability and scope of the borrow, and (3) a set of restrictions. In the code, Loan is a struct defined in middle::borrowck. Formally, we define LOAN as follows:
LOAN = (P, LT, MQ, RESTRICTION*) RESTRICTION = (P, ACTION*) ACTION = MUTATE | CLAIM | FREEZE
Here the LOAN tuple defines the place P being borrowed; the lifetime LT of that borrow; the mutability MQ of the borrow; and a list of restrictions. The restrictions indicate actions which, if taken, could invalidate the loan and lead to type safety violations.
Each RESTRICTION is a pair of a restrictive place P (which will either be the path that was borrowed or some prefix of the path that was borrowed) and a set of restricted actions. There are three kinds of actions that may be restricted for the path P:
MUTATE means that P cannot be assigned to;CLAIM means that the P cannot be borrowed mutably;FREEZE means that the P cannot be borrowed immutably;Finally, it is never possible to move from a place that appears in a restriction. This implies that the “empty restriction” (P, []), which contains an empty set of actions, still has a purpose---it prevents moves from P. I chose not to make MOVE a fourth kind of action because that would imply that sometimes moves are permitted from restricted values, which is not the case.
To give you a better feeling for what kind of restrictions derived from a loan, let's look at the loan L that would be issued as a result of the borrow &mut (*x).f in the example above:
L = ((*x).f, 'a, mut, RS) where RS = [((*x).f, [MUTATE, CLAIM, FREEZE]), (*x, [MUTATE, CLAIM, FREEZE]), (x, [MUTATE, CLAIM, FREEZE])]
The loan states that the expression (*x).f has been loaned as mutable for the lifetime 'a. Because the loan is mutable, that means that the value (*x).f may be mutated via the newly created reference (and only via that pointer). This is reflected in the restrictions RS that accompany the loan.
The first restriction ((*x).f, [MUTATE, CLAIM, FREEZE]) states that the lender may not mutate, freeze, nor alias (*x).f. Mutation is illegal because (*x).f is only supposed to be mutated via the new reference, not by mutating the original path (*x).f. Freezing is illegal because the path now has an &mut alias; so even if we the lender were to consider (*x).f to be immutable, it might be mutated via this alias. They will be enforced for the lifetime 'a of the loan. After the loan expires, the restrictions no longer apply.
The second restriction on *x is interesting because it does not apply to the path that was lent ((*x).f) but rather to a prefix of the borrowed path. This is due to the rules of inherited mutability: if the user were to assign to (or freeze) *x, they would indirectly overwrite (or freeze) (*x).f, and thus invalidate the reference that was created. In general it holds that when a path is lent, restrictions are issued for all the owning prefixes of that path. In this case, the path *x owns the path (*x).f and, because x has ownership, the path x owns the path *x. Therefore, borrowing (*x).f yields restrictions on both *x and x.
Once we have computed the loans introduced by each borrow, the borrow checker uses a data flow propagation to compute the full set of loans in scope at each expression and then uses that set to decide whether that expression is legal. Remember that the scope of loan is defined by its lifetime LT. We sometimes say that a loan which is in-scope at a particular point is an “outstanding loan”, and the set of restrictions included in those loans as the “outstanding restrictions”.
The kinds of expressions which in-scope loans can render illegal are:
lv = v): illegal if there is an in-scope restriction against mutating lv;lv at all;&mut lv): illegal there is an in-scope restriction against claiming lv;&lv): illegal there is an in-scope restriction against freezing lv.Now that we hopefully have some kind of intuitive feeling for how the borrow checker works, let's look a bit more closely now at the precise conditions that it uses.
I will present the rules in a modified form of standard inference rules, which looks as follows:
PREDICATE(X, Y, Z) // Rule-Name Condition 1 Condition 2 Condition 3
The initial line states the predicate that is to be satisfied. The indented lines indicate the conditions that must be met for the predicate to be satisfied. The right-justified comment states the name of this rule: there are comments in the borrowck source referencing these names, so that you can cross reference to find the actual code that corresponds to the formal rule.
I want to collect, at a high-level, the invariants the borrow checker maintains. I will give them names and refer to them throughout the text. Together these invariants are crucial for the overall soundness of the system.
Mutability requires uniqueness. To mutate a path
Unique mutability. There is only one usable mutable path to any given memory at any given time. This implies that when claiming memory with an expression like p = &mut x, the compiler must guarantee that the borrowed value x can no longer be mutated so long as p is live. (This is done via restrictions, read on.)
.
gather_loans passWe start with the gather_loans pass, which walks the AST looking for borrows. For each borrow, there are three bits of information: the place P being borrowed and the mutability MQ and lifetime LT of the resulting pointer. Given those, gather_loans applies four validity tests:
MUTABILITY(P, MQ): The mutability of the reference is compatible with the mutability of P (i.e., not borrowing immutable data as mutable).
ALIASABLE(P, MQ): The aliasability of the reference is compatible with the aliasability of P. The goal is to prevent &mut borrows of aliasability data.
LIFETIME(P, LT, MQ): The lifetime of the borrow does not exceed the lifetime of the value being borrowed.
RESTRICTIONS(P, LT, ACTIONS) = RS: This pass checks and computes the restrictions to maintain memory safety. These are the restrictions that will go into the final loan. We'll discuss in more detail below.
Checking mutability is fairly straightforward. We just want to prevent immutable data from being borrowed as mutable. Note that it is ok to borrow mutable data as immutable, since that is simply a freeze. The judgement MUTABILITY(P, MQ) means the mutability of P is compatible with a borrow of mutability MQ. The Rust code corresponding to this predicate is the function check_mutability in middle::borrowck::gather_loans.
Code pointer: Function check_mutability() in gather_loans/mod.rs, but also the code in mem_categorization.
Let's begin with the rules for variables, which state that if a variable is declared as mutable, it may be borrowed any which way, but otherwise the variable must be borrowed as immutable:
MUTABILITY(X, MQ) // M-Var-Mut DECL(X) = mut MUTABILITY(X, imm) // M-Var-Imm DECL(X) = imm
Fields and boxes inherit their mutability from their base expressions, so both of their rules basically delegate the check to the base expression P:
MUTABILITY(P.f, MQ) // M-Field MUTABILITY(P, MQ) MUTABILITY(*P, MQ) // M-Deref-Unique TYPE(P) = Box<Ty> MUTABILITY(P, MQ)
Immutable pointer types like &T can only be borrowed if MQ is immutable:
MUTABILITY(*P, imm) // M-Deref-Borrowed-Imm TYPE(P) = &Ty
&mut T can be frozen, so it is acceptable to borrow it as either imm or mut:
MUTABILITY(*P, MQ) // M-Deref-Borrowed-Mut TYPE(P) = &mut Ty
The goal of the aliasability check is to ensure that we never permit &mut borrows of aliasable data. The judgement ALIASABLE(P, MQ) means the aliasability of P is compatible with a borrow of mutability MQ. The Rust code corresponding to this predicate is the function check_aliasability() in middle::borrowck::gather_loans.
Local variables are never aliasable as they are accessible only within the stack frame.
ALIASABLE(X, MQ) // M-Var-Mut
Owned content is aliasable if it is found in an aliasable location:
ALIASABLE(P.f, MQ) // M-Field ALIASABLE(P, MQ) ALIASABLE(*P, MQ) // M-Deref-Unique ALIASABLE(P, MQ)
Immutable pointer types like &T are aliasable, and hence can only be borrowed immutably:
ALIASABLE(*P, imm) // M-Deref-Borrowed-Imm TYPE(P) = &Ty
&mut T can be frozen, so it is acceptable to borrow it as either imm or mut:
ALIASABLE(*P, MQ) // M-Deref-Borrowed-Mut TYPE(P) = &mut Ty
These rules aim to ensure that no data is borrowed for a scope that exceeds its lifetime. These two computations wind up being intimately related. Formally, we define a predicate LIFETIME(P, LT, MQ), which states that “the place P can be safely borrowed for the lifetime LT with mutability MQ”. The Rust code corresponding to this predicate is the module middle::borrowck::gather_loans::lifetime.
The rule for variables states that a variable can only be borrowed a lifetime LT that is a subregion of the variable's scope:
LIFETIME(X, LT, MQ) // L-Local LT <= block where X is declared
The lifetime of a field or box is the same as the lifetime of its owner:
LIFETIME(P.f, LT, MQ) // L-Field LIFETIME(P, LT, MQ) LIFETIME(*P, LT, MQ) // L-Deref-Send TYPE(P) = Box<Ty> LIFETIME(P, LT, MQ)
References have a lifetime LT' associated with them. The data they point at has been guaranteed to be valid for at least this lifetime. Therefore, the borrow is valid so long as the lifetime LT of the borrow is shorter than the lifetime LT' of the pointer itself:
LIFETIME(*P, LT, MQ) // L-Deref-Borrowed TYPE(P) = <' Ty OR <' mut Ty LT <= LT'
The final rules govern the computation of restrictions, meaning that we compute the set of actions that will be illegal for the life of the loan. The predicate is written RESTRICTIONS(P, LT, ACTIONS) = RESTRICTION*, which can be read “in order to prevent ACTIONS from occurring on P, the restrictions RESTRICTION* must be respected for the lifetime of the loan”.
Note that there is an initial set of restrictions: these restrictions are computed based on the kind of borrow:
&mut P => RESTRICTIONS(P, LT, MUTATE|CLAIM|FREEZE) &P => RESTRICTIONS(P, LT, MUTATE|CLAIM)
The reasoning here is that a mutable borrow must be the only writer, therefore it prevents other writes (MUTATE), mutable borrows (CLAIM), and immutable borrows (FREEZE). An immutable borrow permits other immutable borrows but forbids writes and mutable borrows.
The simplest case is a borrow of a local variable X:
RESTRICTIONS(X, LT, ACTIONS) = (X, ACTIONS) // R-Variable
In such cases we just record the actions that are not permitted.
Restricting a field is the same as restricting the owner of that field:
RESTRICTIONS(P.f, LT, ACTIONS) = RS, (P.f, ACTIONS) // R-Field RESTRICTIONS(P, LT, ACTIONS) = RS
The reasoning here is as follows. If the field must not be mutated, then you must not mutate the owner of the field either, since that would indirectly modify the field. Similarly, if the field cannot be frozen or aliased, we cannot allow the owner to be frozen or aliased, since doing so indirectly freezes/aliases the field. This is the origin of inherited mutability.
Because the mutability of owned referents is inherited, restricting an owned referent is similar to restricting a field, in that it implies restrictions on the pointer. However, boxes have an important twist: if the owner P is mutated, that causes the owned referent *P to be freed! So whenever an owned referent *P is borrowed, we must prevent the box P from being mutated, which means that we always add MUTATE and CLAIM to the restriction set imposed on P:
RESTRICTIONS(*P, LT, ACTIONS) = RS, (*P, ACTIONS) // R-Deref-Send-Pointer TYPE(P) = Box<Ty> RESTRICTIONS(P, LT, ACTIONS|MUTATE|CLAIM) = RS
Immutable borrowed referents are freely aliasable, meaning that the compiler does not prevent you from copying the pointer. This implies that issuing restrictions is useless. We might prevent the user from acting on *P itself, but there could be another path *P1 that refers to the exact same memory, and we would not be restricting that path. Therefore, the rule for &Ty pointers always returns an empty set of restrictions, and it only permits restricting MUTATE and CLAIM actions:
RESTRICTIONS(*P, LT, ACTIONS) = [] // R-Deref-Imm-Borrowed TYPE(P) = <' Ty LT <= LT' // (1) ACTIONS subset of [MUTATE, CLAIM]
The reason that we can restrict MUTATE and CLAIM actions even without a restrictions list is that it is never legal to mutate nor to borrow mutably the contents of a &Ty pointer. In other words, those restrictions are already inherent in the type.
Clause (1) in the rule for &Ty deserves mention. Here I specify that the lifetime of the loan must be less than the lifetime of the &Ty pointer. In simple cases, this clause is redundant, since the LIFETIME() function will already enforce the required rule:
fn foo(point: &'a Point) -> &'static i32 { &point.x // Error }
The above example fails to compile both because of clause (1) above but also by the basic LIFETIME() check. However, in more advanced examples involving multiple nested pointers, clause (1) is needed:
fn foo(point: &'a &'b mut Point) -> &'b i32 { &point.x // Error }
The LIFETIME rule here would accept 'b because, in fact, the memory is guaranteed to remain valid (i.e., not be freed) for the lifetime 'b, since the &mut pointer is valid for 'b. However, we are returning an immutable reference, so we need the memory to be both valid and immutable. Even though point.x is referenced by an &mut pointer, it can still be considered immutable so long as that &mut pointer is found in an aliased location. That means the memory is guaranteed to be immutable for the lifetime of the & pointer, which is only 'a, not 'b. Hence this example yields an error.
As a final twist, consider the case of two nested immutable pointers, rather than a mutable pointer within an immutable one:
fn foo(point: &'a &'b Point) -> &'b i32 { &point.x // OK }
This function is legal. The reason for this is that the inner pointer (*point : &'b Point) is enough to guarantee the memory is immutable and valid for the lifetime 'b. This is reflected in RESTRICTIONS() by the fact that we do not recurse (i.e., we impose no restrictions on P, which in this particular case is the pointer point : &'a &'b Point).
LIFETIME() and RESTRICTIONS()?Given the previous text, it might seem that LIFETIME and RESTRICTIONS should be folded together into one check, but there is a reason that they are separated. They answer separate concerns. The rules pertaining to LIFETIME exist to ensure that we don't create a borrowed pointer that outlives the memory it points at. So LIFETIME prevents a function like this:
fn get_1<'a>() -> &'a i32 { let x = 1; &x }
Here we would be returning a pointer into the stack. Clearly bad.
However, the RESTRICTIONS rules are more concerned with how memory is used. The example above doesn‘t generate an error according to RESTRICTIONS because, for local variables, we don’t require that the loan lifetime be a subset of the local variable lifetime. The idea here is that we can guarantee that x is not (e.g.) mutated for the lifetime 'a, even though 'a exceeds the function body and thus involves unknown code in the caller -- after all, x ceases to exist after we return and hence the remaining code in 'a cannot possibly mutate it. This distinction is important for type checking functions like this one:
fn inc_and_get<'a>(p: &'a mut Point) -> &'a i32 { p.x += 1; &p.x }
In this case, we take in a &mut and return a frozen borrowed pointer with the same lifetime. So long as the lifetime of the returned value doesn‘t exceed the lifetime of the &mut we receive as input, this is fine, though it may seem surprising at first (it surprised me when I first worked it through). After all, we’re guaranteeing that *p won‘t be mutated for the lifetime 'a, even though we can’t “see” the entirety of the code during that lifetime, since some of it occurs in our caller. But we do know that nobody can mutate *p except through p. So if we don‘t mutate *p and we don’t return p, then we know that the right to mutate *p has been lost to our caller -- in terms of capability, the caller passed in the ability to mutate *p, and we never gave it back. (Note that we can't return p while *p is borrowed since that would be a move of p, as &mut pointers are affine.)
Mutable borrowed pointers are guaranteed to be the only way to mutate their referent. This permits us to take greater license with them; for example, the referent can be frozen simply be ensuring that we do not use the original pointer to perform mutate. Similarly, we can allow the referent to be claimed, so long as the original pointer is unused while the new claimant is live.
The rule for mutable borrowed pointers is as follows:
RESTRICTIONS(*P, LT, ACTIONS) = RS, (*P, ACTIONS) // R-Deref-Mut-Borrowed TYPE(P) = <' mut Ty LT <= LT' // (1) RESTRICTIONS(P, LT, ACTIONS) = RS // (2)
Let's examine the two numbered clauses:
Clause (1) specifies that the lifetime of the loan (LT) cannot exceed the lifetime of the &mut pointer (LT'). The reason for this is that the &mut pointer is guaranteed to be the only legal way to mutate its referent -- but only for the lifetime LT'. After that lifetime, the loan on the referent expires and hence the data may be modified by its owner again. This implies that we are only able to guarantee that the referent will not be modified or aliased for a maximum of LT'.
Here is a concrete example of a bug this rule prevents:
// Test region-reborrow-from-shorter-mut-ref.rs: fn copy_borrowed_ptr<'a,'b,T>(x: &'a mut &'b mut T) -> &'b mut T { &mut **p // ERROR due to clause (1) } fn main() { let mut x = 1; let mut y = &mut x; // <-'b-----------------------------+ // +-'a--------------------+ | // v v | let z = copy_borrowed_ptr(&mut y); // y is lent | *y += 1; // Here y==z, so both should not be usable... | *z += 1; // ...and yet they would be, but for clause 1. | } // <------------------------------------------------------+
Clause (2) propagates the restrictions on the referent to the pointer itself. This is the same as with an box, though the reasoning is mildly different. The basic goal in all cases is to prevent the user from establishing another route to the same data. To see what I mean, let's examine various cases of what can go wrong and show how it is prevented.
Example danger 1: Moving the base pointer. One of the simplest ways to violate the rules is to move the base pointer to a new name and access it via that new name, thus bypassing the restrictions on the old name. Here is an example:
// src/test/compile-fail/borrowck-move-mut-base-ptr.rs fn foo(t0: &mut i32) { let p: &i32 = &*t0; // Freezes `*t0` let t1 = t0; //~ ERROR cannot move out of `t0` *t1 = 22; // OK, not a write through `*t0` }
Remember that &mut pointers are linear, and hence let t1 = t0 is a move of t0 -- or would be, if it were legal. Instead, we get an error, because clause (2) imposes restrictions on P (t0, here), and any restrictions on a path make it impossible to move from that path.
Example danger 2: Claiming the base pointer. Another possible danger is to mutably borrow the base path. This can lead to two bad scenarios. The most obvious is that the mutable borrow itself becomes another path to access the same data, as shown here:
// src/test/compile-fail/borrowck-mut-borrow-of-mut-base-ptr.rs fn foo<'a>(mut t0: &'a mut i32, mut t1: &'a mut i32) { let p: &i32 = &*t0; // Freezes `*t0` let mut t2 = &mut t0; //~ ERROR cannot borrow `t0` **t2 += 1; // Mutates `*t0` }
In this example, **t2 is the same memory as *t0. Because t2 is an &mut pointer, **t2 is a unique path and hence it would be possible to mutate **t2 even though that memory was supposed to be frozen by the creation of p. However, an error is reported -- the reason is that the freeze &*t0 will restrict claims and mutation against *t0 which, by clause 2, in turn prevents claims and mutation of t0. Hence the claim &mut t0 is illegal.
Another danger with an &mut pointer is that we could swap the t0 value away to create a new path:
// src/test/compile-fail/borrowck-swap-mut-base-ptr.rs fn foo<'a>(mut t0: &'a mut i32, mut t1: &'a mut i32) { let p: &i32 = &*t0; // Freezes `*t0` swap(&mut t0, &mut t1); //~ ERROR cannot borrow `t0` *t1 = 22; }
This is illegal for the same reason as above. Note that if we added back a swap operator -- as we used to have -- we would want to be very careful to ensure this example is still illegal.
Example danger 3: Freeze the base pointer. In the case where the referent is claimed, even freezing the base pointer can be dangerous, as shown in the following example:
// src/test/compile-fail/borrowck-borrow-of-mut-base-ptr.rs fn foo<'a>(mut t0: &'a mut i32, mut t1: &'a mut i32) { let p: &mut i32 = &mut *t0; // Claims `*t0` let mut t2 = &t0; //~ ERROR cannot borrow `t0` let q: &i32 = &*t2; // Freezes `*t0` but not through `*p` *p += 1; // violates type of `*q` }
Here the problem is that *t0 is claimed by p, and hence p wants to be the controlling pointer through which mutation or freezes occur. But t2 would -- if it were legal -- have the type & &mut i32, and hence would be a mutable pointer in an aliasable location, which is considered frozen (since no one can write to **t2 as it is not a unique path). Therefore, we could reasonably create a frozen &i32 pointer pointing at *t0 that coexists with the mutable pointer p, which is clearly unsound.
However, it is not always unsafe to freeze the base pointer. In particular, if the referent is frozen, there is no harm in it:
// src/test/ui/borrowck-borrow-of-mut-base-ptr-safe.rs fn foo<'a>(mut t0: &'a mut i32, mut t1: &'a mut i32) { let p: &i32 = &*t0; // Freezes `*t0` let mut t2 = &t0; let q: &i32 = &*t2; // Freezes `*t0`, but that's ok... let r: &i32 = &*t0; // ...after all, could do same thing directly. }
In this case, creating the alias t2 of t0 is safe because the only thing t2 can be used for is to further freeze *t0, which is already frozen. In particular, we cannot assign to *t0 through the new alias t2, as demonstrated in this test case:
// src/test/ui/borrowck-borrow-mut-base-ptr-in-aliasable-loc.rs fn foo(t0: & &mut i32) { let t1 = t0; let p: &i32 = &**t0; **t1 = 22; //~ ERROR cannot assign }
This distinction is reflected in the rules. When doing an &mut borrow -- as in the first example -- the set ACTIONS will be CLAIM|MUTATE|FREEZE, because claiming the referent implies that it cannot be claimed, mutated, or frozen by anyone else. These restrictions are propagated back to the base path and hence the base path is considered unfreezable.
In contrast, when the referent is merely frozen -- as in the second example -- the set ACTIONS will be CLAIM|MUTATE, because freezing the referent implies that it cannot be claimed or mutated but permits others to freeze. Hence when these restrictions are propagated back to the base path, it will still be considered freezable.
FIXME RFC 1751 Restrictions against mutating the base pointer. When an &mut pointer is frozen or claimed, we currently pass along the restriction against MUTATE to the base pointer. I do not believe this restriction is needed. It dates from the days when we had a way to mutate that preserved the value being mutated (i.e., swap). Nowadays the only form of mutation is assignment, which destroys the pointer being mutated -- therefore, a mutation cannot create a new path to the same data. Rather, it removes an existing path. This implies that not only can we permit mutation, we can have mutation kill restrictions in the dataflow sense.
WARNING: We do not currently have const borrows in the language. If they are added back in, we must ensure that they are consistent with all of these examples. The crucial question will be what sorts of actions are permitted with a &const &mut pointer. I would suggest that an &mut referent found in an &const location be prohibited from both freezes and claims. This would avoid the need to prevent const borrows of the base pointer when the referent is borrowed.
[ Previous revisions of this document discussed &const in more detail. See the revision history. ]
The borrow checker is also in charge of ensuring that:
These are two separate dataflow analyses built on the same framework. Let's look at checking that memory is initialized first; the checking of immutable local variable assignments works in a very similar way.
To track the initialization of memory, we actually track all the points in the program that create uninitialized memory, meaning moves and the declaration of uninitialized variables. For each of these points, we create a bit in the dataflow set. Assignments to a variable x or path a.b.c kill the move/uninitialization bits for those paths and any subpaths (e.g., x, x.y, a.b.c, *a.b.c). Bits are unioned when two control-flow paths join. Thus, the presence of a bit indicates that the move may have occurred without an intervening assignment to the same memory. At each use of a variable, we examine the bits in scope, and check that none of them are moves/uninitializations of the variable that is being used.
Let's look at a simple example:
fn foo(a: Box<i32>) { let b: Box<i32>; // Gen bit 0. if cond { // Bits: 0 use(&*a); b = a; // Gen bit 1, kill bit 0. use(&*b); } else { // Bits: 0 } // Bits: 0,1 use(&*a); // Error. use(&*b); // Error. } fn use(a: &i32) { }
In this example, the variable b is created uninitialized. In one branch of an if, we then move the variable a into b. Once we exit the if, therefore, it is an error to use a or b since both are only conditionally initialized. I have annotated the dataflow state using comments. There are two dataflow bits, with bit 0 corresponding to the creation of b without an initializer, and bit 1 corresponding to the move of a. The assignment b = a both generates bit 1, because it is a move of a, and kills bit 0, because b is now initialized. On the else branch, though, b is never initialized, and so bit 0 remains untouched. When the two flows of control join, we union the bits from both sides, resulting in both bits 0 and 1 being set. Thus any attempt to use a uncovers the bit 1 from the “then” branch, showing that a may be moved, and any attempt to use b uncovers bit 0, from the “else” branch, showing that b may not be initialized.
Initialization of immutable variables works in a very similar way, except that:
Thus the presence of an assignment bit indicates that the assignment may have occurred. Note that assignments are only killed when the variable goes out of scope, as it is not relevant whether or not there has been a move in the meantime. Using these bits, we can declare that an assignment to an immutable variable is legal iff there is no other assignment bit to that same variable in scope.
It may seem surprising that we assign dataflow bits to each move rather than each path being moved. This is somewhat less efficient, since on each use, we must iterate through all moves and check whether any of them correspond to the path in question. Similar concerns apply to the analysis for double assignments to immutable variables. The main reason to do it this way is that it allows us to print better error messages, because when a use occurs, we can print out the precise move that may be in scope, rather than simply having to say “the variable may not be initialized”.
The move analysis maintains several data structures that enable it to cross-reference moves and assignments to determine when they may be moving/assigning the same memory. These are all collected into the MoveData and FlowedMoveData structs. The former represents the set of move paths, moves, and assignments, and the latter adds in the results of a dataflow computation.
The MovePath tree tracks every path that is moved or assigned to. These paths have the same form as the LoanPath data structure, which in turn is the "real world version of the places P that we introduced earlier. The difference between a MovePath and a LoanPath is that move paths are:
a.b move path, we can easily find the move path a and also the move paths a.b.c)The mechanism that we use is to create a MovePath record for each move path. These are arranged in an array and are referenced using MovePathIndex values, which are newtype'd indices. The MovePath structs are arranged into a tree, representing using the standard Knuth representation where each node has a child ‘pointer’ and a “next sibling” ‘pointer’. In addition, each MovePath has a parent ‘pointer’. In this case, the ‘pointers’ are just MovePathIndex values.
In this way, if we want to find all base paths of a given move path, we can just iterate up the parent pointers (see each_base_path() in the move_data module). If we want to find all extensions, we can iterate through the subtree (see each_extending_path()).
There are structs to represent moves (Move) and assignments (Assignment), and these are also placed into arrays and referenced by index. All moves of a particular path are arranged into a linked lists, beginning with MovePath.first_move and continuing through Move.next_move.
We distinguish between “var” assignments, which are assignments to a variable like x = foo, and “path” assignments (x.f = foo). This is because we need to assign dataflows to the former, but not the latter, so as to check for double initialization of immutable variables.
Like loans, we distinguish two phases. The first, gathering, is where we uncover all the moves and assignments. As with loans, we do some basic sanity checking in this phase, so we'll report errors if you attempt to move out of a borrowed pointer etc. Then we do the dataflow (see FlowedMoveData::new). Finally, in the check_loans.rs code, we walk back over, identify all uses, assignments, and captures, and check that they are legal given the set of dataflow bits we have computed for that program point.
In addition to the job of enforcing memory safety, the borrow checker code is also responsible for identifying the structural fragments of data in the function, to support out-of-band dynamic drop flags allocated on the stack. (For background, see RFC PR #320.)
Semantically, each piece of data that has a destructor may need a boolean flag to indicate whether or not its destructor has been run yet. However, in many cases there is no need to actually maintain such a flag: It can be apparent from the code itself that a given path is always initialized (or always deinitialized) when control reaches the end of its owner's scope, and thus we can unconditionally emit (or not) the destructor invocation for that path.
A simple example of this is the following:
struct D { p: i32 } impl D { fn new(x: i32) -> D { ... } impl Drop for D { ... } fn foo(a: D, b: D, t: || -> bool) { let c: D; let d: D; if t() { c = b; } }
At the end of the body of foo, the compiler knows that a is initialized, introducing a drop obligation (deallocating the boxed integer) for the end of a's scope that is run unconditionally. Likewise the compiler knows that d is not initialized, and thus it leave out the drop code for d.
The compiler cannot statically know the drop-state of b nor c at the end of their scope, since that depends on the value of t. Therefore, we need to insert boolean flags to track whether we need to drop b and c.
However, the matter is not as simple as just mapping local variables to their corresponding drop flags when necessary. In particular, in addition to being able to move data out of local variables, Rust allows one to move values in and out of structured data.
Consider the following:
struct S { x: D, y: D, z: D } fn foo(a: S, mut b: S, t: || -> bool) { let mut c: S; let d: S; let e: S = a.clone(); if t() { c = b; b.x = e.y; } if t() { c.y = D::new(4); } }
As before, the drop obligations of a and d can be statically determined, and again the state of b and c depend on dynamic state. But additionally, the dynamic drop obligations introduced by b and c are not just per-local boolean flags. For example, if the first call to t returns false and the second call true, then at the end of their scope, b will be completely initialized, but only c.y in c will be initialized. If both calls to t return true, then at the end of their scope, c will be completely initialized, but only b.x will be initialized in b, and only e.x and e.z will be initialized in e.
Note that we need to cover the z field in each case in some way, since it may (or may not) need to be dropped, even though z is never directly mentioned in the body of the foo function. We call a path like b.z a fragment sibling of b.x, since the field z comes from the same structure S that declared the field x in b.x.
In general we need to maintain boolean flags that match the S-structure of both b and c. In addition, we need to consult such a flag when doing an assignment (such as c.y = D::new(4); above), in order to know whether or not there is a previous value that needs to be dropped before we do the assignment.
So for any given function, we need to determine what flags are needed to track its drop obligations. Our strategy for determining the set of flags is to represent the fragmentation of the structure explicitly: by starting initially from the paths that are explicitly mentioned in moves and assignments (such as b.x and c.y above), and then traversing the structure of the path's type to identify leftover unmoved fragments: assigning into c.y means that c.x and c.z are leftover unmoved fragments. Each fragment represents a drop obligation that may need to be tracked. Paths that are only moved or assigned in their entirety (like a and d) are treated as a single drop obligation.
The fragment construction process works by piggy-backing on the existing move_data module. We already have callbacks that visit each direct move and assignment; these form the basis for the sets of moved_leaf_paths and assigned_leaf_paths. From these leaves, we can walk up their parent chain to identify all of their parent paths. We need to identify the parents because of cases like the following:
struct Pair<X,Y>{ x: X, y: Y } fn foo(dd_d_d: Pair<Pair<Pair<D, D>, D>, D>) { other_function(dd_d_d.x.y); }
In this code, the move of the path dd_d.x.y leaves behind not only the fragment drop-obligation dd_d.x.x but also dd_d.y as well.
Once we have identified the directly-referenced leaves and their parents, we compute the left-over fragments, in the function fragments::add_fragment_siblings. As of this writing this works by looking at each directly-moved or assigned path P, and blindly gathering all sibling fields of P (as well as siblings for the parents of P, etc). After accumulating all such siblings, we filter out the entries added as siblings of P that turned out to be directly-referenced paths (or parents of directly referenced paths) themselves, thus leaving the never-referenced “left-overs” as the only thing left from the gathering step.
A special case of the structural fragments discussed above are the elements of an array that has been passed by value, such as the following:
fn foo(a: [D; 10], i: i32) -> D { a[i] }
The above code moves a single element out of the input array a. The remainder of the array still needs to be dropped; i.e., it is a structural fragment. Note that after performing such a move, it is not legal to read from the array a. There are a number of ways to deal with this, but the important thing to note is that the semantics needs to distinguish in some manner between a fragment that is the entire array versus a fragment that represents all-but-one element of the array. A place where that distinction would arise is the following:
fn foo(a: [D; 10], b: [D; 10], i: i32, t: bool) -> D { if t { a[i] } else { b[i] } // When control exits, we will need either to drop all of `a` // and all-but-one of `b`, or to drop all of `b` and all-but-one // of `a`. }
There are a number of ways that the codegen backend could choose to compile this (e.g. a [bool; 10] array for each such moved array; or an Option<usize> for each moved array). From the viewpoint of the borrow-checker, the important thing is to record what kind of fragment is implied by the relevant moves.
While writing up these docs, I encountered some rules I believe to be stricter than necessary:
I think restricting the &mut P against moves and ALIAS is sufficient, MUTATE and CLAIM are overkill. MUTATE was necessary when swap was a built-in operator, but as it is not, it is implied by CLAIM, and CLAIM is implied by ALIAS. The only net effect of this is an extra error message in some cases, though.
I have not described how closures interact. Current code is unsound. I am working on describing and implementing the fix.
If we wish, we can easily extend the move checking to allow finer-grained tracking of what is initialized and what is not, enabling code like this:
a = x.f.g; // x.f.g is now uninitialized // here, x and x.f are not usable, but x.f.h *is* x.f.g = b; // x.f.g is not initialized // now x, x.f, x.f.g, x.f.h are all usable
What needs to change here, most likely, is that the moves module should record not only what paths are moved, but what expressions are actual uses. For example, the reference to x in x.f.g = b is not a true use in the sense that it requires x to be fully initialized. This is in fact why the above code produces an error today: the reference to x in x.f.g = b is considered illegal because x is not fully initialized.
There are also some possible refactorings: