Compiling and building

Content

Gnu tools

Gnu tools

GCC's external interface follows Unix conventions. Users invoke a language-specific driver program (gcc for C, g++ for C++, etc.), which interprets command arguments, calls the actual compiler, runs the assembler on the output, and then optionally runs the linker to produce a complete executable binary.

Each of the language compilers is a separate program that reads source code and outputs machine code. All have a common internal structure. A per-language front end parses the source code in that language and produces an abstract syntax tree ("tree" for short).

These are, if necessary, converted to the middle end's input representation, called GENERIC form; the middle end then gradually transforms the program towards its final form. Compiler optimizations and static code analysis techniques (such as FORTIFY_SOURCE, a compiler directive to discover buffer overflows) are applied to the code. These work on multiple representations, mostly the architecture-independent GIMPLE representation and the architecture-dependent RTL representation. Finally, machine code is produced using architecture-specific pattern matching originally based on an algorithm of Jack Davidson and Chris Fraser.

Other tools

The make ecosystem

The configure script to prepare makefiles

See also https://thoughtbot.com/blog/the-magic-behind-configure-make-make-install

Overview

Usage: Since the install step is also defined in the Makefile, where the software is installed can change based on options passed to the configure script, or things the configure script discovered about your system.

Manual for make: https://www.gnu.org/software/make/manual/

A simple makefile consists of “rules” with the following shape: 
target … : prerequisites …
        recipe

A phony target is one that is not really the name of a file; rather it is just a name for a recipe to be executed when you make an explicit request. There are two reasons to use a phony target: to avoid a conflict with a file of the same name, and to improve performance. If you write a rule whose recipe will not create the target file, the recipe will be executed every time the target comes up for remaking. Here is an example: 
clean:
        rm *.o temp
Because the rm command does not create a file named clean, probably no such file will ever exist. Therefore, the rm command will be executed every time you say ‘make clean’. In this example, the clean target will not work properly if a file named clean is ever created in this directory. Since it has no prerequisites, clean would always be considered up to date and its recipe would not be executed. To avoid this problem you can explicitly declare the target to be phony by making it a prerequisite of the special target .PHONY

By default, make starts with the first target (not targets whose names start with ‘.’ unless they also contain one or more ‘/’). This is called the default goal.

Autotools to create configure scripts

As configure scripts are fairly large and complex, they are not written by hand but generated by a suite of programs called the autotools, e.g. autoconf and automake. See 
On the maintainer’s system:

aclocal # Set up an m4 environment
autoconf # Generate configure from configure.ac
automake --add-missing # Generate Makefile.in from Makefile.am
./configure # Generate Makefile from Makefile.in
make distcheck # Use Makefile to build and test a tarball to distribute

On the end-user’s system:

./configure # Generate Makefile from Makefile.in
make # Use Makefile to build the program
make install # Use Makefile to install the program

Make

Make is a build automation tool that builds executable programs and libraries from source code by reading files called makefiles which specify how to derive the target program. Telling make what to do Make invocation

Make searches the current directory for a makefile, e.g., GNU Make searches files in order for a file named one of GNUmakefile, makefile, or Makefile. and then invokes the specified (or default) target(s) from that file.

The various commands invoked by make when building the target need to be on the PATH.

Make without arguments

Without arguments, Make

Make targets

Make decides whether a target needs to be regenerated by comparing file modification times. This solves the problem of avoiding the building of files which are already up to date, but it fails when a file changes but its modification time stays in the past. Such changes could be caused by restoring an older version of a source file, or when a network filesystem is a source of files and its clock or time zone is not synchronized with the machine running Make. The user must handle this situation by forcing a complete build. Conversely, if a source file's modification time is in the future, it triggers unnecessary rebuilding, which may inconvenience users.

Makefiles are traditionally used for compiling code (*.c, *.cc, *.C, etc.), but they can also be used for providing commands to automate common tasks. One such makefile is called from the command line: The makefile language is similar to declarative programming.

Makefiles may contain five types of constructs:

A makefile consists of rules. Each rule begins with a textual dependency line which defines a target followed by a colon (:) and optionally an enumeration of components (files or other targets) on which the target depends. The dependency line is arranged so that the target (left hand of the colon) depends on components (right hand of the colon). It is common to refer to components as prerequisites of the target. 
target [target ...]: [component ...]
Tab ↹[command 1]
	   .
	   .
	   .
Tab ↹[command n]
Usually each rule has a single unique target, rather than multiple targets.

For example, a C .o object file is created from .c files, so .c files come first (i.e. specific object file target depends on a C source file and header files). Because Make itself does not understand, recognize or distinguish different kinds of files, this opens up a possibility for human error.

Each dependency line may be followed by a series of TAB indented command lines which define how to transform the components (usually source files) into the target (usually the "output"). If any of the prerequisites has a more recent modification time than the target, the command lines are run. The GNU Make documentation refers to the commands associated with a rule as a "recipe".

The first command may appear on the same line after the prerequisites, separated by a semicolon,

targets: prerequisites ; command

for example, hello: ; @echo "hello"

Make can decide where to start through topological sorting.

Each command line must begin with a tab character to be recognized as a command. The tab is a whitespace character, but the space character does not have the same special meaning. This is problematic, since there may be no visual difference between a tab and a series of space characters. This aspect of the syntax of makefiles is often subject to criticism.

Each command is executed by a separate shell or command-line interpreter instance. Since operating systems use different command-line interpreters this can lead to unportable makefiles. For example, GNU Make (all POSIX Makes) executes commands with /bin/sh by default, where Unix commands like cp are normally used. In contrast to that, Microsoft's nmake executes commands with cmd.exe where batch commands like copy are available but not necessarily cp.

A rule may omit the recipe. The dependency line can consist solely of components that refer to other targets, for example: realclean: clean distclean

The command lines of a rule are usually arranged so that they generate the target. An example: if file.html is newer, it is converted to text. The contents of the makefile: 
file.txt: file.html
	lynx -dump file.html > file.txt
The rule above would be triggered when Make updates "file.txt". In the following invocation, Make would typically use this rule to update the "file.txt" target if "file.html" were newer.

Command lines can have one or more of the following three prefixes:

Make listing targets

Approach 1
Run '$ make help'.
Approach 2
Use . However, in more advanced files this misses special targets such as .PHONY, lines containing URLs, variable definitions that use :=, lines defining target-specific variables, and more.
Approach 3
Include the following in your Makefile 
list:
    @grep '^[^#[:space:]].*:' Makefile
Then run '$ make list'.
Approach 4
Define an alias (in .bashrc): alias makefile-targets='grep "^[^#[:space:]].*:" Makefile' . Bash completion expands the alias.

Make macros

A makefile can contain definitions of macros. Macros are usually referred to as variables when they hold simple string definitions, like CC=clang. Macros in makefiles may be overridden in the command-line arguments passed to the Make utility. Environment variables are also available as macros.

Macros allow users to specify the programs invoked and other custom behavior during the build process. For example, the macro CC is frequently used in makefiles to refer to the location of a C compiler, and the user may wish to specify a particular compiler to use.

Some directives in makefiles can include other makefiles.

Line continuation is indicated with a backslash \ character at the end of a line.

Make uses variables through flags

Unless specified, make uses the default value for variables such as CFLAGS, CXXFLAGS, or TARGET_ARCH, which is nothing.

List of possible flags is at https://www.gnu.org/software/make/manual/make.html#index-ARFLAGS .

Make other features

Make supports creating basic functions. You "define" the function just by creating a variable, but use the parameters $(0), $(1), etc. You then call the function with the special call builtin function. The syntax is $(call variable,param,param). $(0) is the variable, while $(1), $(2), etc. are the params. 
sweet_new_fn = Variable Name: $(0) First: $(1) Second: $(2) Empty Variable: $(3)

all:
	# Outputs "Variable Name: sweet_new_fn First: go Second: tigers Empty Variable:"
	@echo $(call sweet_new_fn, go, tigers)

Contents of makefiles

Contents of makefiles

Contents is essentially variable definitions and commands. Some pre-defined variables exist (automatic variables, such as $@). At first sight there's no easy way to identify all possible variables used by the makefile. Some systems use configuration files for this, see e.g. OPTEE's approach for configuring makefile variables.

Using and setting variables

Automatic variables

Debugging makefiles

See chapter 12 of https://www.oreilly.com/openbook/make3/book/

The make warning function

Use as 
$(warning A top-level warning)
FOO := $(warning Right-hand side of a simple variable)bar

Command line options

Buildroot makefiles

Buildroot is a set of makefiles and patches that simplifies and automates the process of building a complete and bootable Linux environment for an embedded system, while using cross-compilation to allow building for multiple target platforms on a single Linux-based development system. Buildroot can automatically build the required cross-compilation toolchain, create a root file system, compile a Linux kernel image, and generate a boot loader for the targeted embedded system, or it can perform any independent combination of these steps.

Meson and Ninja

In the Meson view, there are three phases to getting a development environment running:
  1. Installing a compiler toolchain - Debian, Ubuntu and derivatives: sudo apt install build-essential // gets you gcc and friends
  2. Installing Meson - Debian, Ubuntu and derivatives: sudo apt install meson ninja-build
  3. Creating a project and building it
    • mkdir testproject
    • cd testproject
    • meson init --name testproject --build // creates a project skeleton and compiles it, result is put in the build subdirectory and can be run directly from there
    • build/testproject
    • The project is now ready for development. You can edit the code with any editor and it is rebuilt by going in the build subdirectory and executing the meson compile command. If your version of Meson is too old, you can compile the project by running the command ninja instead.
Info:

Other tools