| ====== |
| TODO |
| ====== |
| |
| Generic |
| ======= |
| |
| * It would be nice to have some basic infrastructure for handling command line |
| parsing better. |
| |
| * It would be nice to have some infrastructure for statistics (a la LLVM). |
| |
| Ninja Manifests |
| =============== |
| |
| * Diagnose multiple build decls with same output. |
| |
| Build Engine |
| ============ |
| |
| * Generalize key type. |
| |
| * Generalize value type (cleanly, current solution is gross). |
| |
| * Support multiple outputs. |
| |
| * Support active scheduling (?). |
| |
| * Consider moving the task model to one where the build engine just invokes |
| callbacks on the delegate (probably using task IDs, and maybe kind IDs for the |
| use of clients which want to follow class-based dispatch models). This has two |
| advantages, (1) it maps more neatly to an obvious C API, and (2) it should |
| make it easier to cleanly generalize the value type because only the engine |
| and delegate need to be templated, and neither support subclassing. |
| |
| * Figure out when taskNeedsInput() should be allowed, and if |
| taskDiscoveredDependency() should be eliminated in favor of loosened rules |
| about when it can be invoked. |
| |
| * Figure out if Rule should be subclassed, with virtual methods for action |
| generation and input checking. |
| |
| * Implement proper error handling. |
| |
| * Think about how to implement dynamic command failure -- how should this be |
| communicated downstream? This is another area where flexibility might be nice. |
| |
| * Investigate how Ninja handles syncing the build graph with the on disk state |
| when it has no DB (nevermind, seems to rebuild). What happens when an output |
| of a compile command is touched? |
| |
| * Introspection features for watching build status. |
| |
| * Performance |
| |
| * Think about # of allocations in engine loop. |
| |
| * Think about whether there is any way to get of per-task variable length |
| list. |
| |
| * Think about making-progress policy, what order do we prefer starting tasks |
| vs. providing inputs vs. finishing tasks. |
| |
| * Should we finalize immediately instead of moving things around on queues. |
| |
| * We need to avoid the duplicate stating we currently do of input files. |
| |
| * Implement an efficient shared queue for FinishedTaskInfos, if we stay with |
| the current design. |
| |
| * Figure out if we should change the Rule interface so that the engine can |
| manage a free-list of allocated Tasks. |
| |
| Build Database |
| ============== |
| |
| * Performance |
| |
| * Should we support DB stashing extra information in Rule (its own ID |
| number). Should the engine officially maintain the ID (maybe useful once we |
| generalize anyway). |
| |
| * Should we support the DB doing a bulk load of the rule data? Probably more |
| efficient for most databases, but on the other hand this would go against |
| moving to a model where the data is *only* stored in the DB layer, and the |
| BuildEngine always has to go to the DB for it. That model might be slower, |
| but more scalable. |
| |
| * Investigate using DB entirely on demand, per above comment. Very scalable. |
| |
| * Normalize key and dependency node types to a unique entry to reduce |
| duplication of large key values. |
| |
| * Many clients end up having additional information about a key that then gets |
| lost when they serialize it and get it back somewhere else (e.g., one task |
| requests the key, another starts the key). For example, the client most |
| likely has some internal state associated with that key that would be really |
| nice to be able to pass around. |
| |
| We can solve this by making the key type richer, and allowing it to have |
| serialization as just one of its methods. We could use a virtual interface |
| or just a simple mechanism to attach a payload. |
| |
| |
| Ninja Specific |
| ============== |
| |
| Tasks for Usable Tool |
| --------------------- |
| |
| * Implement path normalization (for Nodes as well as things like imported |
| dependencies from compiler output). |
| |
| * Implement Ninja failure semantics for order-only dependencies, which block the |
| downstream command. |
| |
| * Handle removed implicit dependencies properly, currently this generates an |
| error and then builds ok on the next iteration. The latter problem may |
| indicate a latent issue in handling of discovered dependencies. |
| |
| * Handle rerunning a command on introduction of new dependencies. For example, |
| before we reran on command changes we wouldn't rebuild a library just because |
| it depended on a new file. We should probably make sure this happens even if |
| the command doesn't change, although it might be good to check vs Ninja. |
| |
| * Support update-if-newer for commands with discovered dependencies. |
| |
| * We should probably store a rule type along with the result, and always |
| invalidate if the rule type has changed. |
| |
| Random Tasks |
| ------------ |
| |
| * Ninja reruns a command if the restat= flag changes? What triggers this? This |
| behavior probably also happens for depfiles and things, we should match. |
| |
| * There are some subtle differences in how we handle restat = 0, and generally |
| we don't do the same thing Ninja would (we can rebuild less). This is because |
| Ninja will just run things downstream if an incoming edge was dirty, but we |
| will do so and also allow the update-if-newer behavior to trigger on interior |
| commands. As an example, look at how the multiple-outputs test case behaves in |
| Ninja with restat = 0. |
| |
| * Tasks should have a way to examine their discovered dependencies from a |
| previous run. For example, this is necessary to implement update-if-newer for |
| commands with discovered dependencies. |
| |
| * Add support for cleaning up output files (deleting them on failed tasks)? |
| |
| * Investigate using pselect mechanisms vs blocked threads. |
| |
| * Support traditional style handling of updated outputs (in which consumer |
| commands are rerun but not the producer itself). |
| |
| * Performance |
| |
| * Need to optimize the evalString() stuff. A good performance test for this is |
| to compare ``ninja -n`` on the chromium fake build.ninja file vs ``llb ninja |
| build --quiet --no-execute`` (62% of which is currently the loading, and 43% |
| of which is evalString()). |
| |
| * It would be super cool to use the build engine itself for managing the |
| loading of the .ninja file, if we cached the result in a compact binary |
| representation (the build engine would then be used to recompute that as |
| necessary). This would be a nice validation of the generic engine approach |
| too. |
| |
| * There is some bad-smelling redundancy between how we check the file info in |
| the `*IsValid()` functions, and how we then recompute that info later as part |
| of the task (and the engine internally will compare it to see if it has |
| changed to know if it needs to propagate the change). We need to think about |
| this and figure out what is ideal. There might be a cleaner modeling where |
| we discretely represent each stat-of-file as an input that is then consumed |
| by each item that requires it. This would make it easy to guarantee we |
| compute such things once. |
| |
| * I have heard a claim that one can actually improve performance by |
| strategically purging the OS buffer cache -- the claim was that it is faster |
| to build Swift after building LLVM & Clang if there is a purge in |
| between. If true, this may be better things we can do to communicate to the |
| kernel the purpose and lifetime of things like object files. |
| |
| * We should consider allowing the write of the target result to go directly |
| into the stored Result field. That would avoid the need for spurious |
| allocations when updating results. |
| |
| * We need to switch the Rule Dependencies to be stored using the ID of the |
| rule (which means we need to assign rule IDs, but the DB would like that |
| anyway). This dramatically reduces the storage required by the database |
| (although a lot of that is because of our subpar phony command |
| implementation, and it would drop significantly if we switch to a |
| specialized implementation for phony commands, because we don't need the |
| clunky giant composite-key). |
| |
| * We should use a custom task for Phony commands, they have a lot of special |
| cases (like the one above about the composite key size). |
| |
| * We should move to a compact encoding for the build value. Not worth doing |
| until we address the rule_dependencies table size. |
| |
| |
| Build System |
| ============ |
| |
| Build File |
| ---------- |
| |
| * We will probably want some way to define properties shared by groups of tasks |
| (for example, common flags), for efficiencies sake. There are a couple ways |
| to do this: |
| |
| * We could make the build file an "immediate-mode" sort of interface, and |
| allow interleaving of tool and task maps. Then the client could just |
| generate the file with updated information interleaved. This would be |
| similar to how Ninja files get generated in practice by nice generators |
| (`gyp`, not `CMake`). |
| |
| * We could allow the definition of tool aliases, that can define additional |
| properties. This lets the format be better definite and not have immediate |
| mode stateful problems. |
| |
| * We want some way to allow the task name and one of the tasks outputs to be |
| the same, without having a redundant specification. |
| |
| * We might need a mechanism for defining default properties for nodes. |
| |
| * We may want to add a notion of types for nodes. We could try and be context |
| dependent too, but having a type here would make it easier for the client to |
| bind the node to the right type during loading. |
| |
| * We may want some provision for providing inline node attributes with the task |
| definitions. Otherwise we cannot really stream the file to the build system |
| in cases where node attributes are required. |