Merge topic 'fetchcontent-reduce-boilerplate'
a94355c7b7 FetchContent: Add new command FetchContent_MakeAvailable()
611d5274de Sphinx: Add limited support for nested variables in docs
1a07e1b47d FetchContent: Trivial doc corrections
Acked-by: Kitware Robot <kwrobot@kitware.com>
Merge-request: !2830
diff --git a/Help/release/dev/fetchcontent-reduce-boilerplate.rst b/Help/release/dev/fetchcontent-reduce-boilerplate.rst
new file mode 100644
index 0000000..6157668
--- /dev/null
+++ b/Help/release/dev/fetchcontent-reduce-boilerplate.rst
@@ -0,0 +1,8 @@
+fetchcontent-reduce-boilerplate
+-------------------------------
+
+* The FetchContent module gained a new :command:`FetchContent_MakeAvailable`
+ command. This new command accepts a list of dependency names, which it then
+ iterates over, populating and adding each one to the main build using the
+ canonical pattern. This significantly reduces the amount of boilerplate
+ needed in the project.
diff --git a/Modules/FetchContent.cmake b/Modules/FetchContent.cmake
index c65ae9e..5716b01 100644
--- a/Modules/FetchContent.cmake
+++ b/Modules/FetchContent.cmake
@@ -20,9 +20,12 @@
:command:`include` or :command:`file` operations.
Content population details would normally be defined separately from the
-command that performs the actual population. Projects should also
-check whether the content has already been populated somewhere else in the
-project hierarchy. Typical usage would look something like this:
+command that performs the actual population. This separation ensures that
+all of the dependency details are defined before anything may try to use those
+details to populate content. This is particularly important in more complex
+project hierarchies where dependencies may be shared between multiple projects.
+
+The following shows a typical example of declaring content details:
.. code-block:: cmake
@@ -32,21 +35,37 @@
GIT_TAG release-1.8.0
)
+For most typical cases, populating the content can then be done with a single
+command like so:
+
+.. code-block:: cmake
+
+ FetchContent_MakeAvailable(googletest)
+
+The above command not only populates the content, it also adds it to the main
+build (if possible) so that the main build can use the populated project's
+targets, etc. In some cases, the main project may need to have more precise
+control over the population or may be required to explicitly define the
+population steps (e.g. if CMake versions earlier than 3.14 need to be
+supported). The typical pattern of such custom steps looks like this:
+
+.. code-block:: cmake
+
FetchContent_GetProperties(googletest)
if(NOT googletest_POPULATED)
FetchContent_Populate(googletest)
add_subdirectory(${googletest_SOURCE_DIR} ${googletest_BINARY_DIR})
endif()
-When using the above pattern with a hierarchical project arrangement,
-projects at higher levels in the hierarchy are able to define or override
-the population details of content specified anywhere lower in the project
-hierarchy. The ability to detect whether content has already been
-populated ensures that even if multiple child projects want certain content
-to be available, the first one to populate it wins. The other child project
-can simply make use of the already available content instead of repeating
-the population for itself. See the
-:ref:`Examples <fetch-content-examples>` section which demonstrates
+Regardless of which population method is used, when using the
+declare-populate pattern with a hierarchical project arrangement, projects at
+higher levels in the hierarchy are able to override the population details of
+content specified anywhere lower in the project hierarchy. The ability to
+detect whether content has already been populated ensures that even if
+multiple child projects want certain content to be available, the first one
+to populate it wins. The other child project can simply make use of the
+already available content instead of repeating the population for itself.
+See the :ref:`Examples <fetch-content-examples>` section which demonstrates
this scenario.
The ``FetchContent`` module also supports defining and populating
@@ -113,6 +132,38 @@
Populating The Content
^^^^^^^^^^^^^^^^^^^^^^
+For most common scenarios, population means making content available to the
+main build according to previously declared details for that dependency.
+There are two main patterns for populating content, one based on calling
+:command:`FetchContent_GetProperties` and
+:command:`FetchContent_Populate` for more precise control and the other on
+calling :command:`FetchContent_MakeAvailable` for a simpler, more automated
+approach. The former generally follows this canonical pattern:
+
+.. _`fetch-content-canonical-pattern`:
+
+.. code-block:: cmake
+
+ # Check if population has already been performed
+ FetchContent_GetProperties(<name>)
+ string(TOLOWER "<name>" lcName)
+ if(NOT ${lcName}_POPULATED)
+ # Fetch the content using previously declared details
+ FetchContent_Populate(<name>)
+
+ # Set custom variables, policies, etc.
+ # ...
+
+ # Bring the populated content into the build
+ add_subdirectory(${${lcName}_SOURCE_DIR} ${${lcName}_BINARY_DIR})
+ endif()
+
+The above is such a common pattern that, where no custom steps are needed
+between the calls to :command:`FetchContent_Populate` and
+:command:`add_subdirectory`, equivalent logic can be obtained by calling
+:command:`FetchContent_MakeAvailable` instead (and should be preferred where
+it meets the needs of the project).
+
.. command:: FetchContent_Populate
.. code-block:: cmake
@@ -309,9 +360,6 @@
on the command line invoking the script.
-Retrieve Population Properties
-^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-
.. command:: FetchContent_GetProperties
When using saved content details, a call to :command:`FetchContent_Populate`
@@ -343,28 +391,65 @@
FetchContent_GetProperties(foobar)
if(NOT foobar_POPULATED)
FetchContent_Populate(foobar)
-
- # Set any custom variables, etc. here, then
- # populate the content as part of this build
-
- add_subdirectory(${foobar_SOURCE_DIR} ${foobar_BINARY_DIR})
+ ...
endif()
The above pattern allows other parts of the overall project hierarchy to
re-use the same content and ensure that it is only populated once.
+.. command:: FetchContent_MakeAvailable
+
+ .. code-block:: cmake
+
+ FetchContent_MakeAvailable( <name1> [<name2>...] )
+
+ This command implements the common pattern typically needed for most
+ dependencies. It iterates over each of the named dependencies in turn
+ and for each one it loosely follows the same
+ :ref:`canonical pattern <fetch-content-canonical-pattern>` as
+ presented at the beginning of this section. One small difference to
+ that pattern is that it will only call :command:`add_subdirectory` on the
+ populated content if there is a ``CMakeLists.txt`` file in its top level
+ source directory. This allows the command to be used for dependencies
+ that make downloaded content available at a known location but which do
+ not need or support being added directly to the build.
+
+
.. _`fetch-content-examples`:
Examples
^^^^^^^^
-Consider a project hierarchy where ``projA`` is the top level project and it
-depends on projects ``projB`` and ``projC``. Both ``projB`` and ``projC``
-can be built standalone and they also both depend on another project
-``projD``. For simplicity, this example will assume that all four projects
-are available on a company git server. The ``CMakeLists.txt`` of each project
-might have sections like the following:
+This first fairly straightforward example ensures that some popular testing
+frameworks are available to the main build:
+
+.. code-block:: cmake
+
+ include(FetchContent)
+ FetchContent_Declare(
+ googletest
+ GIT_REPOSITORY https://github.com/google/googletest.git
+ GIT_TAG release-1.8.0
+ )
+ FetchContent_Declare(
+ Catch2
+ GIT_REPOSITORY https://github.com/catchorg/Catch2.git
+ GIT_TAG v2.5.0
+ )
+
+ # After the following call, the CMake targets defined by googletest and
+ # Catch2 will be defined and available to the rest of the build
+ FetchContent_MakeAvailable(googletest Catch2)
+
+
+In more complex project hierarchies, the dependency relationships can be more
+complicated. Consider a hierarchy where ``projA`` is the top level project and
+it depends directly on projects ``projB`` and ``projC``. Both ``projB`` and
+``projC`` can be built standalone and they also both depend on another project
+``projD``. ``projB`` additionally depends on ``projE``. This example assumes
+that all five projects are available on a company git server. The
+``CMakeLists.txt`` of each project might have sections like the following:
*projA*:
@@ -373,31 +458,27 @@
include(FetchContent)
FetchContent_Declare(
projB
- GIT_REPOSITORY git@mycompany.com/git/projB.git
+ GIT_REPOSITORY git@mycompany.com:git/projB.git
GIT_TAG 4a89dc7e24ff212a7b5167bef7ab079d
)
FetchContent_Declare(
projC
- GIT_REPOSITORY git@mycompany.com/git/projC.git
+ GIT_REPOSITORY git@mycompany.com:git/projC.git
GIT_TAG 4ad4016bd1d8d5412d135cf8ceea1bb9
)
FetchContent_Declare(
projD
- GIT_REPOSITORY git@mycompany.com/git/projD.git
+ GIT_REPOSITORY git@mycompany.com:git/projD.git
GIT_TAG origin/integrationBranch
)
+ FetchContent_Declare(
+ projE
+ GIT_REPOSITORY git@mycompany.com:git/projE.git
+ GIT_TAG origin/release/2.3-rc1
+ )
- FetchContent_GetProperties(projB)
- if(NOT projb_POPULATED)
- FetchContent_Populate(projB)
- add_subdirectory(${projb_SOURCE_DIR} ${projb_BINARY_DIR})
- endif()
-
- FetchContent_GetProperties(projC)
- if(NOT projc_POPULATED)
- FetchContent_Populate(projC)
- add_subdirectory(${projc_SOURCE_DIR} ${projc_BINARY_DIR})
- endif()
+ # Order is important, see notes in the discussion further below
+ FetchContent_MakeAvailable(projD projB projC)
*projB*:
@@ -406,16 +487,16 @@
include(FetchContent)
FetchContent_Declare(
projD
- GIT_REPOSITORY git@mycompany.com/git/projD.git
+ GIT_REPOSITORY git@mycompany.com:git/projD.git
GIT_TAG 20b415f9034bbd2a2e8216e9a5c9e632
)
+ FetchContent_Declare(
+ projE
+ GIT_REPOSITORY git@mycompany.com:git/projE.git
+ GIT_TAG 68e20f674a48be38d60e129f600faf7d
+ )
- FetchContent_GetProperties(projD)
- if(NOT projd_POPULATED)
- FetchContent_Populate(projD)
- add_subdirectory(${projd_SOURCE_DIR} ${projd_BINARY_DIR})
- endif()
-
+ FetchContent_MakeAvailable(projD projE)
*projC*:
@@ -424,48 +505,77 @@
include(FetchContent)
FetchContent_Declare(
projD
- GIT_REPOSITORY git@mycompany.com/git/projD.git
+ GIT_REPOSITORY git@mycompany.com:git/projD.git
GIT_TAG 7d9a17ad2c962aa13e2fbb8043fb6b8a
)
+ # This particular version of projD requires workarounds
FetchContent_GetProperties(projD)
if(NOT projd_POPULATED)
FetchContent_Populate(projD)
+
+ # Copy an additional/replacement file into the populated source
+ file(COPY someFile.c DESTINATION ${projd_SOURCE_DIR}/src)
+
add_subdirectory(${projd_SOURCE_DIR} ${projd_BINARY_DIR})
endif()
A few key points should be noted in the above:
- ``projB`` and ``projC`` define different content details for ``projD``,
- but ``projA`` also defines a set of content details for ``projD`` and
- because ``projA`` will define them first, the details from ``projB`` and
+ but ``projA`` also defines a set of content details for ``projD``.
+ Because ``projA`` will define them first, the details from ``projB`` and
``projC`` will not be used. The override details defined by ``projA``
are not required to match either of those from ``projB`` or ``projC``, but
it is up to the higher level project to ensure that the details it does
define still make sense for the child projects.
-- While ``projA`` defined content details for ``projD``, it did not need
- to explicitly call ``FetchContent_Populate(projD)`` itself. Instead, it
- leaves that to a child project to do (in this case it will be ``projB``
- since it is added to the build ahead of ``projC``). If ``projA`` needed to
- customize how the ``projD`` content was brought into the build as well
- (e.g. define some CMake variables before calling
- :command:`add_subdirectory` after populating), it would do the call to
- ``FetchContent_Populate()``, etc. just as it did for the ``projB`` and
- ``projC`` content. For higher level projects, it is usually enough to
- just define the override content details and leave the actual population
- to the child projects. This saves repeating the same thing at each level
- of the project hierarchy unnecessarily.
-- Even though ``projA`` is the top level project in this example, it still
- checks whether ``projB`` and ``projC`` have already been populated before
- going ahead to do those populations. This makes ``projA`` able to be more
- easily incorporated as a child of some other higher level project in the
- future if required. Always protect a call to
- :command:`FetchContent_Populate` with a check to
- :command:`FetchContent_GetProperties`, even in what may be considered a top
- level project at the time.
+- In the ``projA`` call to :command:`FetchContent_MakeAvailable`, ``projD``
+ is listed ahead of ``projB`` and ``projC`` to ensure that ``projA`` is in
+ control of how ``projD`` is populated.
+- While ``projA`` defines content details for ``projE``, it does not need
+ to explicitly call ``FetchContent_MakeAvailable(projE)`` or
+ ``FetchContent_Populate(projD)`` itself. Instead, it leaves that to the
+ child ``projB``. For higher level projects, it is often enough to just
+ define the override content details and leave the actual population to the
+ child projects. This saves repeating the same thing at each level of the
+ project hierarchy unnecessarily.
-The following example demonstrates how one might download and unpack a
+Projects don't always need to add the populated content to the build.
+Sometimes the project just wants to make the downloaded content available at
+a predictable location. The next example ensures that a set of standard
+company toolchain files (and potentially even the toolchain binaries
+themselves) is available early enough to be used for that same build.
+
+.. code-block:: cmake
+
+ cmake_minimum_required(VERSION 3.14)
+
+ include(FetchContent)
+ FetchContent_Declare(
+ mycom_toolchains
+ URL https://intranet.mycompany.com//toolchains_1.3.2.tar.gz
+ )
+ FetchContent_MakeAvailable(mycom_toolchains)
+
+ project(CrossCompileExample)
+
+The project could be configured to use one of the downloaded toolchains like
+so:
+
+.. code-block:: shell
+
+ cmake -DCMAKE_TOOLCHAIN_FILE=_deps/mycom_toolchains-src/toolchain_arm.cmake /path/to/src
+
+When CMake processes the ``CMakeLists.txt`` file, it will download and unpack
+the tarball into ``_deps/mycompany_toolchains-src`` relative to the build
+directory. The :variable:`CMAKE_TOOLCHAIN_FILE` variable is not used until
+the :command:`project` command is reached, at which point CMake looks for the
+named toolchain file relative to the build directory. Because the tarball has
+already been downloaded and unpacked by then, the toolchain file will be in
+place, even the very first time that ``cmake`` is run in the build directory.
+
+Lastly, the following example demonstrates how one might download and unpack a
firmware tarball using CMake's :manual:`script mode <cmake(1)>`. The call to
:command:`FetchContent_Populate` specifies all the content details and the
unpacked firmware will be placed in a ``firmware`` directory below the
@@ -921,3 +1031,31 @@
set(${contentNameLower}_POPULATED True PARENT_SCOPE)
endfunction()
+
+# Arguments are assumed to be the names of dependencies that have been
+# declared previously and should be populated. It is not an error if
+# any of them have already been populated (they will just be skipped in
+# that case). The command is implemented as a macro so that the variables
+# defined by the FetchContent_GetProperties() and FetchContent_Populate()
+# calls will be available to the caller.
+macro(FetchContent_MakeAvailable)
+
+ foreach(contentName IN ITEMS ${ARGV})
+ string(TOLOWER ${contentName} contentNameLower)
+ FetchContent_GetProperties(${contentName})
+ if(NOT ${contentNameLower}_POPULATED)
+ FetchContent_Populate(${contentName})
+
+ # Only try to call add_subdirectory() if the populated content
+ # can be treated that way. Protecting the call with the check
+ # allows this function to be used for projects that just want
+ # to ensure the content exists, such as to provide content at
+ # a known location.
+ if(EXISTS ${${contentNameLower}_SOURCE_DIR}/CMakeLists.txt)
+ add_subdirectory(${${contentNameLower}_SOURCE_DIR}
+ ${${contentNameLower}_BINARY_DIR})
+ endif()
+ endif()
+ endforeach()
+
+endmacro()
diff --git a/Tests/RunCMake/FetchContent/MakeAvailable-stdout.txt b/Tests/RunCMake/FetchContent/MakeAvailable-stdout.txt
new file mode 100644
index 0000000..6e6c730
--- /dev/null
+++ b/Tests/RunCMake/FetchContent/MakeAvailable-stdout.txt
@@ -0,0 +1,2 @@
+Confirmation project has been added
+.*Confirmation script has been called
diff --git a/Tests/RunCMake/FetchContent/MakeAvailable.cmake b/Tests/RunCMake/FetchContent/MakeAvailable.cmake
new file mode 100644
index 0000000..a93f1f7
--- /dev/null
+++ b/Tests/RunCMake/FetchContent/MakeAvailable.cmake
@@ -0,0 +1,20 @@
+include(FetchContent)
+
+FetchContent_Declare(
+ WithProject
+ SOURCE_DIR ${CMAKE_CURRENT_LIST_DIR}/WithProject
+)
+FetchContent_Declare(
+ WithoutProject
+ SOURCE_DIR ${CMAKE_CURRENT_LIST_DIR}/WithoutProject
+)
+
+# Order is important and will be verified by test output
+FetchContent_MakeAvailable(WithProject WithoutProject)
+
+get_property(addedWith GLOBAL PROPERTY FetchWithProject SET)
+if(NOT addedWith)
+ message(SEND_ERROR "Subdir with CMakeLists.txt not added")
+endif()
+
+include(${withoutproject_SOURCE_DIR}/confirmMessage.cmake)
diff --git a/Tests/RunCMake/FetchContent/MakeAvailableTwice-stdout.txt b/Tests/RunCMake/FetchContent/MakeAvailableTwice-stdout.txt
new file mode 100644
index 0000000..8d3b7c0
--- /dev/null
+++ b/Tests/RunCMake/FetchContent/MakeAvailableTwice-stdout.txt
@@ -0,0 +1,4 @@
+-- Before first[
+ ]+-- Confirmation project has been added[
+ ]+-- Between both[
+ ]+-- After last
diff --git a/Tests/RunCMake/FetchContent/MakeAvailableTwice.cmake b/Tests/RunCMake/FetchContent/MakeAvailableTwice.cmake
new file mode 100644
index 0000000..a9af020
--- /dev/null
+++ b/Tests/RunCMake/FetchContent/MakeAvailableTwice.cmake
@@ -0,0 +1,12 @@
+include(FetchContent)
+
+FetchContent_Declare(
+ WithProject
+ SOURCE_DIR ${CMAKE_CURRENT_LIST_DIR}/WithProject
+)
+
+message(STATUS "Before first")
+FetchContent_MakeAvailable(WithProject)
+message(STATUS "Between both")
+FetchContent_MakeAvailable(WithProject)
+message(STATUS "After last")
diff --git a/Tests/RunCMake/FetchContent/MakeAvailableUndeclared-result.txt b/Tests/RunCMake/FetchContent/MakeAvailableUndeclared-result.txt
new file mode 100644
index 0000000..d00491f
--- /dev/null
+++ b/Tests/RunCMake/FetchContent/MakeAvailableUndeclared-result.txt
@@ -0,0 +1 @@
+1
diff --git a/Tests/RunCMake/FetchContent/MakeAvailableUndeclared-stderr.txt b/Tests/RunCMake/FetchContent/MakeAvailableUndeclared-stderr.txt
new file mode 100644
index 0000000..9715b78
--- /dev/null
+++ b/Tests/RunCMake/FetchContent/MakeAvailableUndeclared-stderr.txt
@@ -0,0 +1 @@
+No content details recorded for NoDetails
diff --git a/Tests/RunCMake/FetchContent/MakeAvailableUndeclared.cmake b/Tests/RunCMake/FetchContent/MakeAvailableUndeclared.cmake
new file mode 100644
index 0000000..bd57cbe
--- /dev/null
+++ b/Tests/RunCMake/FetchContent/MakeAvailableUndeclared.cmake
@@ -0,0 +1,3 @@
+include(FetchContent)
+
+FetchContent_MakeAvailable(NoDetails)
diff --git a/Tests/RunCMake/FetchContent/RunCMakeTest.cmake b/Tests/RunCMake/FetchContent/RunCMakeTest.cmake
index 9c1ab66..e28ae96 100644
--- a/Tests/RunCMake/FetchContent/RunCMakeTest.cmake
+++ b/Tests/RunCMake/FetchContent/RunCMakeTest.cmake
@@ -11,6 +11,9 @@
run_cmake(GetProperties)
run_cmake(DirOverrides)
run_cmake(UsesTerminalOverride)
+run_cmake(MakeAvailable)
+run_cmake(MakeAvailableTwice)
+run_cmake(MakeAvailableUndeclared)
# We need to pass through CMAKE_GENERATOR and CMAKE_MAKE_PROGRAM
# to ensure the test can run on machines where the build tool
diff --git a/Tests/RunCMake/FetchContent/WithProject/CMakeLists.txt b/Tests/RunCMake/FetchContent/WithProject/CMakeLists.txt
new file mode 100644
index 0000000..b6a3750
--- /dev/null
+++ b/Tests/RunCMake/FetchContent/WithProject/CMakeLists.txt
@@ -0,0 +1,5 @@
+cmake_minimum_required(VERSION 3.13)
+project(WithProject LANGUAGES NONE)
+
+set_property(GLOBAL PROPERTY FetchWithProject YES)
+message(STATUS "Confirmation project has been added")
diff --git a/Tests/RunCMake/FetchContent/WithoutProject/confirmMessage.cmake b/Tests/RunCMake/FetchContent/WithoutProject/confirmMessage.cmake
new file mode 100644
index 0000000..a2f5c61
--- /dev/null
+++ b/Tests/RunCMake/FetchContent/WithoutProject/confirmMessage.cmake
@@ -0,0 +1 @@
+message(STATUS "Confirmation script has been called")
diff --git a/Utilities/Sphinx/cmake.py b/Utilities/Sphinx/cmake.py
index 882cdc1..d903dbe 100644
--- a/Utilities/Sphinx/cmake.py
+++ b/Utilities/Sphinx/cmake.py
@@ -21,6 +21,8 @@
# - Unix paths are recognized by '/'; support for Windows paths may be added if needed
# - (\\.) allows for \-escapes (used in manual/cmake-language.7)
# - $<..$<..$>..> nested occurence in cmake-buildsystem
+# - Nested variable evaluations are only supported in a limited capacity. Only
+# one level of nesting is supported and at most one nested variable can be present.
CMakeLexer.tokens["root"] = [
(r'\b(\w+)([ \t]*)(\()', bygroups(Name.Function, Text, Name.Function), '#push'), # fctn(
@@ -34,7 +36,8 @@
(r'[<>]=', Punctuation), # used in FindPkgConfig.cmake
(r'\$<', Operator, '#push'), # $<...>
(r'<[^<|]+?>(\w*\.\.\.)?', Name.Variable), # <expr>
- (r'(\$\w*\{)(.+?)(\})', bygroups(Operator, Name.Tag, Operator)), # ${..} $ENV{..}
+ (r'(\$\w*\{)([^\}\$]*)?(?:(\$\w*\{)([^\}]+?)(\}))?([^\}]*?)(\})', # ${..} $ENV{..}, possibly nested
+ bygroups(Operator, Name.Tag, Operator, Name.Tag, Operator, Name.Tag, Operator)),
(r'([A-Z]+\{)(.+?)(\})', bygroups(Operator, Name.Tag, Operator)), # DATA{ ...}
(r'[a-z]+(@|(://))((\\.)|[\w.+-:/\\])+', Name.Attribute), # URL, git@, ...
(r'/\w[\w\.\+-/\\]*', Name.Attribute), # absolute path
