Refine the way Ninja waits for subprocess completion.
This CL refactors how the Posix SubprocessSet implementation
delays with subprocess completion. Before this CL, the
Fuchsia version of Ninja would always wait for the output
pipe of a command to be closed to determine that the
corresponding "work" has completed.
This is important because a command sub-process can launch
child programs in the background and passing them their
stdout/stderr descriptors, and may exit *before* the child
itself.
However, this doesn't work in certain cases, for example if
the child process daemonizes itself, and keeps the output
descriptor opened, without writing anything to it. It is
suspected that this is the root cause of the Ninja
build timeouts that were are experiencing in CI (see
associated bug).
As a work-around, this CL changes the logic to used by
Ninja to the following:
- Each Subprocess instance can be in one of three
states now:
* Running: its PID is active, and its pipe is open.
* Reaped: the PID has exited, but the pipe is still
open and Ninja is waiting for more data out of it.
* Finished: the PID has exited, and the pipe is
closed.
NOTE: The special case where the Subprocess has no
pipe is no longer enabled in the Fuchsia version of
Ninja, but corresponds to the upstream behavior
for "console" processes. This didn't allow recording
their output though.
- When the pipe is closed, the Subprocess is reaped
and finished immediately.
- SIGCHLD is used to detect when any Subprocess PID exits
and reaps it immediately (before that it would only
check for upstream console processes). However, if
the pipe is still open, a timer is setup to wait up
to 30 seconds for incoming data on the pipe, and
the Subprocess is kept in the running_ queue
(but with 'reaped_ == true').
- When data arrives on the pipe of a reaped process,
the timer is re-armed to wait for 30 more seconds
(just in case).
- Otherwise, if the timer expires, the Subprocess
is force-finished.
- A SIGINT / SIGTERM / SIGHUP should also force-finish
reaped Subprocess instances immediately.
+ Adjust the process tree to print "REAPED [PID <pid>]"
instead of "[PID <pid>]" when printing the command
corresponding to a Reaped Subprocess instance.
+ Add a regression test that verifies that SIGINT
stops Ninja immediately when it is waiting for
a reaped subprocess.
+ Add a regression test that verifies that SIGTERM
stops Ninja immediately when it is waiting for
a reaped subprocess, and that the diagnostic
message lists its pid with the REAPED tag.
Bug: 498320348
Fuchsia-Only: graceful-shutdown
Change-Id: Ic7e81aca2680fdc603873227be7b7515d8969248
Ninja is a small build system with a focus on speed. https://ninja-build.org/
See the manual or doc/manual.asciidoc included in the distribution for background and more details.
Binaries for Linux, Mac and Windows are available on GitHub. Run ./ninja -h for Ninja help.
Installation is not necessary because the only required file is the resulting ninja binary. However, to enable features like Bash completion and Emacs and Vim editing modes, some files in misc/ must be copied to appropriate locations.
If you're interested in making changes to Ninja, read CONTRIBUTING.md first.
You can either build Ninja via the custom generator script written in Python or via CMake. For more details see the wiki.
./configure.py --bootstrap
This will generate the ninja binary and a build.ninja file you can now use to build Ninja with itself.
If you have a GoogleTest source directory, you can build the tests by passing its path with --gtest-source-dir=PATH option, or the GTEST_SOURCE_DIR environment variable, e.g.:
./configure.py --bootstrap --gtest-source-dir=/path/to/googletest ./ninja all # build ninja_test and other auxiliary binaries ./ninja_test` # run the unit-test suite.
Use the CMake build below if you want to use a preinstalled binary version of the library.
To build the ninja binary without building the unit tests, disable test building by setting BUILD_TESTING to OFF:
cmake -Bbuild-cmake -DBUILD_TESTING=OFF cmake --build build-cmake
The ninja binary will now be inside the build-cmake directory (you can choose any other name you like).
To run the unit tests, omit the -DBUILD_TESTING=OFF option, and after building, run:
./build-cmake/ninja_test
You must have asciidoc and xsltproc in your PATH, then do:
./configure.py ninja manual doc/manual.html
Which will generate doc/manual.html.
To generate the PDF version of the manual, you must have dblatext in your PATH then do:
./configure.py # only if you didn't do it previously. ninja doc/manual.pdf
Which will generate doc/manual.pdf.
If you have doxygen installed, you can build documentation extracted from C++ declarations and comments to help you navigate the code. Note that Ninja is a standalone executable, not a library, so there is no public API, all details exposed here are internal.
./configure.py # if needed ninja doxygen
Then open doc/doxygen/html/index.html in a browser to look at it.