Building CMake projects

For this overview, we will assume we have:

• A foo-sdk toolchain containing a packages, named bar
• A worktree containing two projects, the world project, and a hello project, which depends on world and bar.
• A build profile named my-profile configured in the worktree.

This overview guides you through all what happens from the moment you run:

$ qibuild configure --worktree=/path/to/worktree \
                    --config foo-sdk \
                    --profile my-profile \
                    --release \
                    -DWITH_SPAM=ON \
                    hello

to every cmake code that is generated, and what CMake flags are passed.

(We chose a complex example here on purpose to show all the various cases that need to be handled)

Command line parsing - first pass

This is done by qisys.script.root_command_main from bin/qibuild script.

We look for every module in qibuild.actions, and find the configure.py module.

Then, we create a argparse.ArgumentParser parser, and run qibuild.configure.configure_parser on it.

We parse the command line arguments using this parser, and we now have a argparse.NameSpace object we can pass to qibuild.configure.do.

# in qibuild.actions.configure

def configure_parser(parser):
   # Calls functions in ``qibuild.parsers`` to add all the options
   # this command recognize


def do(args):
   # In this point we have a argparse.Namespace object with the "raw"
   # result of the arguments parsing
   args.worktree = "/path/to/worktree"
   args.build_type = "Release"
   args.config = "foo-sdk"
   args.projects = ["hello"]
   args.profiles = ["my-profile"]

This “raw” command parsing already took care of simple tasks, like making sure the --debug or --release arguments are converted to a proper CMAKE_BUILD_TYPE.

Command line parsing - second pass

The goal is to get a correctly initialized CMakeBuilder object

This is done in just a single line:

# in qibuild.actions.configure

def do(args):
    cmake_builder = qibuild.parsers.get_cmake_builder(args)

The get_ functions in qibuild.parsers are here to factorize code that must be called in every action that uses a BuildWorkTree.

The get_cmake_builder action looks like

# in qibuild.parsers


def get_cmake_builder(args):
    """ Get a CMakeBuilder object from the command line

    """
    build_worktree = get_build_worktree(args)
    # dep solving will be made later by the CMakeBuilder
    build_projects = get_build_projects(build_worktree, args, solve_deps=False)
    cmake_builder = qibuild.cmake_builder.CMakeBuilder(build_worktree, build_projects)
    cmake_builder.dep_types = get_dep_types(args)
    return cmake_builder

Here’s what those functions do:

get_build_worktree

• A new WorkTree object is initialized using the path given in args.worktree, or by exploring parent directories until a .qi directory is found if --worktree is not given At this point, every path registered in the worktree can be found in worktree.projects

• A new BuildWorkTree is initialized. A list of BuildProject objects is built from every project in worktree.projects, by inspecting the various qiproject.xml and looking for <qibuild> tags. Note that at this moment build_project.build_depends, build_project.run_depends, and build_project.test_depends are sets of names because no dependency resolution has been done yet.

• A new CMakeBuildConfig object is initialized, using the .qi/qibuild.xml file to read the default config that should be used. If the user has an incorrect default config specified in the .qi/qibuild.xml file, an error is raised immediately.

• Then, the build_config object is configured using the args object and the qibuild.xml configuration files.

  First, the -c argument is checked to see if it matches a known toolchain. If not, an error is raised.

  Then, the configuration specific settings and the default settings in ~/.config/qi/qibuild.xml are read.

  For instance, if the user specified -c foo-sdk on the command line there is a <cmake gererator="Ninja"> tag in the <config name="foo-sdk"> section of ~/.config/qi/qibuild.xml, build_config.cmake_generator is set to Ninja and build_config.toolchain_name to foo-sdk

  Lastly, the options coming from the command line are applied to the build_config object.

  This is done after reading the config files, so that settings can be overwritten. Thus the user can for instance specify --cmake-generator="Unix Makefiles" to overwrite the default CMake generator configured in ~/.config/qi/qibuild.xml

• Lastly, the build_config is applied to the BuildWorkTree: worktree.build_config = build_config.

Note: the code later looks like:

# in BuildProject

def configure(self, **kwargs)
    cmake_args = self.cmake_args
    build_directory = self.build_directory

But actually, cmake_args and build_directory are both properties.

This means that the build dir will always match the latest build settings, and that the list of CMake args in the BuildProject will always be up to date.

# in CMakeBuildConfig

@property
def cmake_args(self):
    # Transform all the "high level" settings into a list of
    # CMake arguments

>>> build_config.cmake_generator == "Ninja"
>>> build_config.cmake_args
["-G", "Ninja"]
>>> build_config.toolchain_name = "foo-sdk"
>>> build_config.cmake_args
["-DCMAKE_TOOLCHAIN_FILE=/path/to/foo-sdk/toolchain.cmake"]

The build config also manages the environment variables, so that you can for instance set a suitable PATH when using mingw on windows without to mess with the registry base.

# in BuildProject
@property
def build_directory(self):
    #  Create a sensible build dir, using
    # self.build_worktree.build_config

>>> build_config.build_type = "Release"
>>> hello_project.build_directory = "/path/to/hello/build-release"
>>> build_config.profiles = ["my-profile"]
>>> hello_project.build_directory = "/path/to/hello/build-my-profile"

get_build_projects

The goal here is to get a list of BuildProject objects to build.

• If no build project named is specified, the parent directories are explored until a qiproject.xml containing a <qibuild> tag is found.

  If no such project is registered in the BuildWorkTree yet, it will be automatically added to the worktree cache.

• If the user specified some projects in the command line, a matching build_project is searched in the build_worktree for every project name specified on the command line. If no build project is found, an error is raised.

Note that at this point, no dependency solving has been done yet. Meaning that the projects list only contains the hello project

get_dep_types

Here, get_dep_types is used to converting the --runtime, --build-deps-only, --single arguments into a list of build types:

• default: ["build", "runtime"]
• --runtime: ["build", "runtime"]
• -s, --single : []
• --build-deps-only : ["build"]

Finally, CMakeBuilder.dep_types is set In our examples, no argument was specified at all, so the build and the runtime dependencies are going to be used.

Configuring the project and its dependencies

Here’s what the code looks like:

# in qibuild.cmake_builder

class CMakeBuilder:
    def __init__(self, build_worktree, projects):
        self.build_worktree = build_worktree
        self.projects = projects
        self.deps_solver = BuildDepsSolver(self)



    def configure(self, **kwargs):
        self.bootstrap_projects()
        projects = self.deps_solver.get_dep_projects(self.projects, self.dep_types)
        for project in projects:
          project.configure(**kwargs)

Note that the CMakeBuilder contains a BuildDepsSolver to delegates all the dependencies solving.

For instance, configuring hello, by default should call configure() on the world project, unless -s was specified.

Also, since hello has a runtime dependency on the bar package, qibuild install --runtime hello /tmp/hl should install both hello and bar to /tmp/hl

Also note that CMakeBuilder delegates the actual call to cmake to the build project itself

Generating the dependencies.cmake

For the CMake call to work, a dependencies.cmake must be written in the build directory

This is done by cmake_builder.bootstrap_projects

Here it is important that the dependencies.cmake always contains the list of every build dependencies, even if -s is used.

Calling CMake

Here deps_solver uses self.dep_types, so that when qibuild configure -s hello, is used, world.configure() is not called.

Installing

When installing a project, the deps_solver is again used to get a list of packages to install.

Then either:

• the whole contents of the packages are installed (the “-config.cmake” files, the headers, the static and shared libraries, etc.)
• if solving_type was set to runtime, only the runtime parts of the packages (shared libraries) will be installed.

Building projects outside a qiBuild action

This could be part of a continuous integration script, for instance:

worktree = qisys.worktree.WorkTree(worktree_root)
build_worktree = BuildWorkTree(worktree)
build_config = build_worktree.build_config

Note

Here the build_config has already been initialized from the various config files, and default values, but you can still use:

build_config.set_active_config("mytoolchain")
build_config.build_type = "Release"

project = build_worktree.get_build_project(name)

cmake_builder = CMakeBuilder(build_worktree, [projet])

# Configure and build the build and runtime deps of the
# project:
cmake_builder.configure()
cmake_builder.build()

If you then need to install the runtime parts only (to make a redistributable package for instance)

cmake_builder.dep_types = ["runtime"]
cmake_builder.install(destdir="package")