Help/guide/tutorial/Configuration and Cache Variables.rst

Step 3: Configuration and Cache Variables
=========================================

CMake projects often have some project-specific configuration variables which
users and packagers are interested in. CMake has many ways that an invoking
user or process can communicate these configuration choices, but the most
fundamental of them are :option:`-D <cmake -D>` flags.

In this step we'll explore the ins and out of how to provide project
configuration options from within a CML, and how to invoke CMake to take
advantage of configuration options provided by both CMake and individual
projects.

Background
^^^^^^^^^^

If we had a CMake project for compression software which supported multiple
compression algorithms, we might want to let the packager of the project decide
which algorithms to enable when they build our software. We can do so by
consuming variables set via :option:`-D <cmake -D>` flags.

.. code-block:: cmake

  if(COMPRESSION_SOFTWARE_USE_ZLIB)
    message("I will use Zlib!")
    # ...
  endif()

  if(COMPRESSION_SOFTWARE_USE_ZSTD)
    message("I will use Zstd!")
    # ...
  endif()

.. code-block:: console

  $ cmake -B build \
      -DCOMPRESSION_SOFTWARE_USE_ZLIB=ON \
      -DCOMPRESSION_SOFTWARE_USE_ZSTD=OFF
  ...
  I will use Zlib!

Of course, we will want to provide reasonable defaults for these configuration
choices, and a way to communicate the purpose of a given option. This function
is provided by the :command:`option` command.

.. code-block:: cmake

  option(COMPRESSION_SOFTWARE_USE_ZLIB "Support Zlib compression" ON)
  option(COMPRESSION_SOFTWARE_USE_ZSTD "Support Zstd compression" ON)

  if(COMPRESSION_SOFTWARE_USE_ZLIB)
    # Same as before
  # ...

.. code-block:: console

  $ cmake -B build \
      -DCOMPRESSION_SOFTWARE_USE_ZLIB=OFF
  ...
  I will use Zstd!

The names created by :option:`-D <cmake -D>` flags and :command:`option` are
not normal variables, they are **cache** variables. Cache variables are globally
visible variables which are *sticky*, their value is difficult to change after
it is initially set. In fact they are so sticky that, in project mode, CMake
will save and restore cache variables across multiple configurations. If a
cache variable is set once, it will remain until another :option:`-D <cmake -D>`
flag preempts the saved variable.

.. note::
  CMake itself has dozens of normal and cache variables used for configuration.
  These are documented at :manual:`cmake-variables(7)` and operate in the same
  manner as project-provided variables for configuration.

:command:`set` can also be used to manipulate cache variables, but will not
change a variable which has already been created.

.. code-block:: cmake

  set(StickyCacheVariable "I will not change" CACHE STRING "")
  set(StickyCacheVariable "Overwrite StickyCache" CACHE STRING "")

  message("StickyCacheVariable: ${StickyCacheVariable}")

.. code-block:: console

  $ cmake -B build
  ...
  StickyCacheVariable: I will not change

Because :option:`-D <cmake -D>` flags are processed before any project commands,
they take precedence for setting the value of a cache variable.

.. code-block:: console

  $ cmake -B build \
    -DStickyCacheVariable="Commandline always wins"
  ...
  StickyCacheVariable: Commandline always wins

While cache variables cannot ordinarily be changed, they can be *shadowed* by
normal variables. We can observe this by :command:`set`'ing a variable to have
the same name as a cache variable, and then using :command:`unset` to remove
the normal variable.

.. code-block:: cmake

  set(ShadowVariable "In the shadows" CACHE STRING "")
  set(ShadowVariable "Hiding the cache variable")
  message("ShadowVariable: ${ShadowVariable}")

  unset(ShadowVariable)
  message("ShadowVariable: ${ShadowVariable}")

.. code-block:: console

  $ cmake -B build
  ...
  ShadowVariable: Hiding the cache variable
  ShadowVariable: In the shadows

.. note::
  :ref:`Script mode <Script Processing Mode>` operates slightly differently,
  only :option:`-D <cmake -D>` flags provided before the :option:`-P <cmake -P>`
  flag in the command are evaluated and available in the running script.

Exercise 1 - Using Options
^^^^^^^^^^^^^^^^^^^^^^^^^^

We can imagine a scenario where consumers really want our ``MathFunctions``
library, and the ``Tutorial`` utility is a "take it or leave it" add-on. In
that case, we might want to add an option to allow consumers to disable
building our ``Tutorial`` binary, building only the ``MathFunctions`` library.

With our knowledge of options, conditionals, and cache variables we have all
the pieces we need to make this configuration available.

Goal
----

Add an option named ``TUTORIAL_BUILD_UTILITIES`` to control if the ``Tutorial``
binary is configured and built.

.. note::
  CMake allows us to determine which targets are built after configuration. Our
  users could ask for the ``MathFunctions`` library alone without ``Tutorial``.
  CMake also has mechanisms to exclude targets from ``ALL``, the default target
  which builds all the other available targets.

  However, options which completely exclude targets from the configuration are
  convenient and popular, especially if configuring those targets involves
  heavy-weight steps which might take some time.

  It also simplifies :command:`install()` logic, which we'll discuss in later
  steps, if targets the packager is uninterested in are completely excluded.

Helpful Resources
-----------------

* :command:`option`
* :command:`if`

Files to Edit
-------------

* ``CMakeLists.txt``

Getting Started
---------------

The ``Help/guide/tutorial/Step3`` folder contains the complete, recommended
solution to ``Step1`` and the relevant ``TODOs`` for this step. Take a minute
to review and refamiliarize yourself with the ``Tutorial`` project.

When you feel you have an understanding of the current code, start with
``TODO 1`` and complete through ``TODO 2``.

Build and Run
-------------

We can now reconfigure our project. However, this time we want to control the
configuration via :option:`-D <cmake -D>` flags. We again start by navigating
to ``Help/guide/tutorial/Step3`` and invoking CMake, but this time with our
configuration options.

.. code-block:: console

  cmake -B build -DTUTORIAL_BUILD_UTILITIES=OFF

We can now build as usual.

.. code-block:: console

  cmake --build build

After the build we should observe no Tutorial executable is produced. Because
cache variables are sticky even a reconfigure shouldn't change this, despite
the default-``ON`` option.

.. code-block:: console

  cmake -B build
  cmake --build build

Will not produce the Tutorial executable, the cache variables are "locked in".
To change this we have two options. First, we can edit the file which stores
the cache variables between CMake configuration runs, the "CMake Cache". This
file is ``build/CMakeCache.txt``, in it we can find the option cache variable.

.. code-block:: text

  //Build the Tutorial executable
  TUTORIAL_BUILD_UTILITIES:BOOL=OFF

We can change this from ``OFF`` to ``ON``, rerun the build, and we will get
our ``Tutorial`` executable.

.. note::
  ``CMakeCache.txt`` entries are of the form ``<Name>:<Type>=<Value>``, however
  the "type" is only a hint. All objects in CMake are strings, regardless of
  what the cache says.

Alternatively, we can change the value of the cache variable on the command
line, because the command line runs before ``CMakeCache.txt`` is loaded its
value take precedence over those in the cache file.

.. code-block:: console

  cmake -B build -DTUTORIAL_BUILD_UTILITIES=ON
  cmake --build build

Doing so we observe the value in ``CMakeCache.txt`` has flipped from ``OFF``
to ``ON``, and that the ``Tutorial`` executable is built.

Solution
--------

First we create our :command:`option` to provide our cache variable with a
reasonable default value.

.. raw:: html

  <details><summary>TODO 1: Click to show/hide answer</summary>

.. literalinclude:: Step4/CMakeLists.txt
  :caption: TODO 1: CMakeLists.txt
  :name: CMakeLists.txt-option-TUTORIAL_BUILD_UTILITIES
  :language: cmake
  :start-at: option(TUTORIAL_BUILD_UTILITIES
  :end-at: option(TUTORIAL_BUILD_UTILITIES

.. raw:: html

  </details>

Then we can check the cache variable to conditionally enable the ``Tutorial``
executable (by way of adding its subdirectory).

.. raw:: html

  <details><summary>TODO 2: Click to show/hide answer</summary>

.. literalinclude:: Step4/CMakeLists.txt
  :caption: TODO 2: CMakeLists.txt
  :name: CMakeLists.txt-if-TUTORIAL_BUILD_UTILITIES
  :language: cmake
  :start-at: if(TUTORIAL_BUILD_UTILITIES)
  :end-at: endif()

.. raw:: html

  </details>

Exercise 2 - ``CMAKE`` Variables
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

CMake has several important normal and cache variables provided to allow
packagers to control the build. Decisions such as compilers, default flags,
search locations for packages, and much more are all controlled by CMake's
own configuration variables.

Among the most important are language standards. As the language standard can
have significant impact on the ABI presented by a given package. For example,
it's quite common for libraries to use standard C++ templates on later
standards, and provide polyfills on earlier standards. If a library is consumed
under different standards then ABI incompatibilities between the standard
templates and the polyfills can result in incomprehensible errors and runtime
crashes.

Ensuring all of our targets are built under the same language standard is
achieved with the :variable:`CMAKE_<LANG>_STANDARD` cache variables. For C++,
this is ``CMAKE_CXX_STANDARD``.

.. note::
  Because these variables are so important, it is equally important that
  developers not override or shadow them in their CMLs. Shadowing
  :variable:`CMAKE_<LANG>_STANDARD` in a CML because the library wants C++20,
  when the packager has decided to build the rest of their libraries and
  applications with C++23, can lead to the aforementioned terrible,
  incomprehensible errors.

  Do not :command:`set` ``CMAKE_`` globals without very strong reasons for
  doing so. We'll discuss better methods for targets to communicate
  requirements like definitions and minimum standards in later steps.

In this exercise, we'll introduce some C++20 code into our library and
executable and build them with C++20 by setting the appropriate cache variable.

Goal
----

Use ``std::format`` to format printed strings instead of stream operators. To
ensure availability of ``std::format``, configure CMake to use the C++20
standard for C++ targets.

Helpful Resources
-----------------

* :option:`cmake -D`
* :variable:`CMAKE_<LANG>_STANDARD`
* :variable:`CMAKE_CXX_STANDARD`
* :prop_tgt:`CXX_STANDARD`
* `cppreference \<format\> <https://en.cppreference.com/cpp/utility/format/format>`_

Files to Edit
-------------

* ``Tutorial/Tutorial.cxx``
* ``MathFunctions/MathFunctions.cxx``

Getting Started
---------------

Continue to edit files from ``Step3``. Complete ``TODO 3`` through ``TODO 7``.
We'll be modifying our prints to use ``std::format`` instead of stream
operators.

Ensure your cache variables are set such that the Tutorial executable will be
built, using any of the methods discussed in the previous exercise.

Build and Run
-------------

We need to reconfigure our project with the new standard, we can do this
using the same method as our ``TUTORIAL_BUILD_UTILITIES`` cache variable.

.. code-block:: console

  cmake -B build -DCMAKE_CXX_STANDARD=20

.. note::
  Configuration variables are, by convention, prefixed with the provider of the
  variable. CMake configuration variables are prefixed with ``CMAKE_``, while
  projects should prefix their variables with ``<PROJECT>_``.

  The tutorial configuration variables follow this convention, and are prefixed
  with ``TUTORIAL_``.

Now that we've configured with C++20, we can build as usual.

.. code-block:: console

  cmake --build build

Solution
--------

We need to include ``<format>`` and then use it.

.. raw:: html

  <details><summary>TODO 3-5: Click to show/hide answer</summary>

.. literalinclude:: Step4/Tutorial/Tutorial.cxx
  :caption: TODO 3: Tutorial/Tutorial.cxx
  :name: Tutorial/Tutorial.cxx-include-format
  :language: c++
  :start-at: #include <format>
  :end-at: #include <string>

.. literalinclude:: Step4/Tutorial/Tutorial.cxx
  :caption: TODO 4: Tutorial/Tutorial.cxx
  :name: Tutorial/Tutorial.cxx-format1
  :language: c++
  :start-at: if (argc < 2) {
  :end-at: return 1;
  :append: }
  :dedent: 2

.. literalinclude:: Step4/Tutorial/Tutorial.cxx
  :caption: TODO 5: Tutorial/Tutorial.cxx
  :name: Tutorial/Tutorial.cxx-format3
  :language: c++
  :start-at: // calculate square root
  :end-at: outputValue);
  :dedent: 2

.. raw:: html

  </details>

And again for the ``MathFunctions`` library.

.. raw:: html

  <details><summary>TODO 6-7: Click to show/hide answer</summary>

.. literalinclude:: Step4/MathFunctions/MathFunctions.cxx
  :caption: TODO 6: MathFunctions.cxx
  :name: MathFunctions/MathFunctions.cxx-include-format
  :language: c++
  :start-at: #include <format>
  :end-at: #include <iostream>

.. literalinclude:: Step4/MathFunctions/MathFunctions.cxx
  :caption: TODO 7: MathFunctions.cxx
  :name: MathFunctions/MathFunctions.cxx-format
  :language: c++
  :start-at: double delta
  :end-at: std::format
  :dedent: 4

.. raw:: html

  </details>

Exercise 3 - CMakePresets.json
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Managing these configuration values can quickly become overwhelming. In CI
systems it is appropriate to record these as part of a given CI step. For
example in a Github Actions CI step we might see something akin to the
following:

.. code-block:: yaml

  - name: Configure and Build
    run: |
      cmake \
        -B build \
        -DCMAKE_BUILD_TYPE=Release \
        -DCMAKE_CXX_STANDARD=20 \
        -DCMAKE_CXX_EXTENSIONS=ON \
        -DTUTORIAL_BUILD_UTILITIES=OFF \
        # Possibly many more options
        # ...

      cmake --build build

When developing code locally, typing all these options even once might be error
prone. If a fresh configuration is needed for any reason, doing so multiple
times could be exhausting.

There are many and varied solutions to this problem, and your choice is
ultimately up to your preferences as a developer. CLI-oriented developers
commonly use task runners to invoke CMake with their desired options for a
project. Most IDEs also have a custom mechanism for controlling CMake
configuration.

It would be impossible to fully enumerate every possible configuration workflow
here. Instead we will explore CMake's built-in solution, known as
:manual:`CMake Presets <cmake-presets(7)>`. Presets give us a format to name
and express collections of CMake configuration options.

.. note::
    Presets are capable of expressing entire CMake workflows, from
    configuration, through building, all the way to installing the software
    package.

    They are far more flexible than can we have room for here. We'll limit
    ourselves to using them for configuration.

CMake Presets come in two standard files, ``CMakePresets.json``, which is
intended to be a part of the project and tracked in source control; and
``CMakeUserPresets.json``, which is intended for local user configuration
and should not be tracked in source control.

The simplest preset which would be of use to a developer does nothing more
than configure variables.

.. code-block:: json

  {
    "version": 4,
    "configurePresets": [
      {
        "name": "example-preset",
        "cacheVariables": {
          "EXAMPLE_FOO": "Bar",
          "EXAMPLE_QUX": "Baz"
        }
      }
    ]
  }

When invoking CMake, where previously we would have done:

.. code-block:: console

  cmake -B build -DEXAMPLE_FOO=Bar -DEXAMPLE_QUX=Baz

We can now use the preset:

.. code-block:: console

  cmake -B build --preset example-preset

CMake will search for files named ``CMakePresets.json`` and
``CMakeUserPresets.json``, and load the named configuration from them if
available.

.. note::
  Command line flags can be mixed with presets. Command line flags have
  precedence over values found in a preset.

.. note::
  On CMake 4.4 and newer, CMake can also load presets from any file specified
  with :option:`cmake --presets-file`. This can be useful when reusing settings
  across multiple projects, since it avoids having to duplicate them in
  separate ``CMakePresets.json`` files for each project.

Presets also support limited macros, variables that can be brace-expanded
inside the preset. The only one of interest to us is the ``${sourceDir}`` macro,
which expands to the root directory of the project. We can use this to set our
build directory, skipping the :option:`-B <cmake -B>` flag when configuring
the project.

.. code-block:: json

  {
    "name": "example-preset",
    "binaryDir": "${sourceDir}/build"
  }

Goal
----

Configure and build the tutorial using a CMake Preset instead of command line
flags.

Helpful Resources
-----------------

* :manual:`cmake-presets(7)`

Files to Edit
-------------

* ``CMakePresets.json``

Getting Started
---------------

Continue to edit files from ``Step3``. Complete ``TODO 8`` and ``TODO 9``.

.. note::
  ``TODOs`` inside ``CMakePresets.json`` need to be *replaced*. There should
  be no ``TODO`` keys left inside the file when you have completed the exercise.

You can verify the preset is working correctly by deleting the existing build
folder before you configure, this will ensure you're not reusing the existing
CMake Cache for configuration.

.. note::
  On CMake 3.24 and newer, the same effect can be achieved by configuring with
  :option:`cmake --fresh`.

All future configuration changes will be via the ``CMakePresets.json`` file.

Build and Run
-------------

We can now use the presets file to manage our configuration.

.. code-block:: console

  cmake --preset tutorial

Presets are capable of running the build step for us, but for this tutorial
we'll continue to run the build ourselves.

.. code-block:: console

  cmake --build build

Solution
--------

There are two changes we need to make, first we want to set the build
directory (also called the "binary directory") to the ``build`` subdirectory
of our project folder, and second we need to set the ``CMAKE_CXX_STANDARD`` to
``20``.

.. raw:: html

  <details><summary>TODO 8-9: Click to show/hide answer</summary>

.. code-block:: json
  :caption: TODO 8-9: CMakePresets.json
  :name: CMakePresets.json-initial

  {
    "version": 4,
    "configurePresets": [
      {
        "name": "tutorial",
        "displayName": "Tutorial Preset",
        "description": "Preset to use with the tutorial",
        "binaryDir": "${sourceDir}/build",
        "cacheVariables": {
          "CMAKE_CXX_STANDARD": "20"
        }
      }
    ]
  }

.. raw:: html

  </details>