Chapter 6. Building

Erlang.mk can do a lot of things, but it is, first and foremost, a build tool. In this chapter we will cover the basics of building a project with Erlang.mk.

For most of this chapter, we will assume that you are using a project generated by Erlang.mk.

6.1. How to build

To build a project, all you have to do is type make:

$ make

It will work regardless of your project: OTP applications, library applications, NIFs, port drivers or even releases. Erlang.mk also automatically downloads and compiles the dependencies for your project.

All this is possible thanks to a combination of configuration and conventions. Most of the conventions come from Erlang/OTP itself so any seasoned Erlang developers should feel right at home.

Erlang.mk supports multi-threaded building. Parallel execution is supported for all targets. To execute Erlang.mk in parallel the -j option must be used:

$ make -j8

The MAKEFLAGS variable can be used to enable parallel building permanently on your system. It can be set in your .zshrc, .bashrc or equivalent file.

MAKEFLAGS="-j8"

6.2. What to build

Erlang.mk gives you control over three steps of the build process, allowing you to do a partial build if needed.

A build has three phases: first any dependency is fetched and built, then the project itself is built and finally a release may be generated when applicable. A release is only generated for projects specifically configured to do so.

Erlang.mk handles those three phases automatically when you type make. But sometimes you just want to repeat one or two of them.

The commands detailed in this section are most useful after you have a successful build as they allow you to quickly redo a step instead of going through everything. This is especially useful for large projects or projects that end up generating releases.

6.2.1. Application

You can build your application and dependencies without generating a release by running the following command:

$ make app

To build your application without touching dependencies at all, you can use the SKIP_DEPS variable:

$ make app SKIP_DEPS=1

This command is very useful if you have a lot of dependencies and develop on a machine with slow file access, like the Raspberry Pi and many other embedded devices.

Note that this command may fail if a required dependency is missing.

6.2.2. Dependencies

You can build all dependencies, and nothing else, by running the following command:

$ make deps

This will fetch and compile all dependencies and their dependencies, recursively.

Packages and dependencies are covered in the next chapter.

6.2.3. Release

It is not possible to build the release without at least building the application itself, unless of course if there’s no application to begin with.

To generate the release, make will generally suffice with a normal Erlang.mk. A separate target is however available, and will take care of building the release, after building the application and all dependencies:

$ make rel

Consult the Releases chapter for more information about what releases are and how they are generated.

6.3. Application resource file

When building your application, Erlang.mk will generate the application resource file. This file is mandatory for all Erlang applications and is found in ebin/$(PROJECT).app.

PROJECT is a variable defined in your Makefile and taken from the name of the directory when Erlang.mk bootstraps your project.

Erlang.mk can build the ebin/$(PROJECT).app in two different ways: from the configuration found in the Makefile, or from the src/$(PROJECT).app.src file.

6.3.1. Application configuration

Erlang.mk automatically fills the PROJECT variable when bootstrapping a new project, but everything else is up to you. None of the values are required to build your project, although it is recommended to fill everything relevant to your situation.

PROJECT
The name of the OTP application or library.
PROJECT_DESCRIPTION
Short description of the project.
PROJECT_VERSION
Current version of the project.
PROJECT_MOD
The application callback module.
PROJECT_REGISTERED
List of the names of all registered processes.
PROJECT_ENV
Configuration parameters used by the application.
PROJECT_APP_EXTRA_KEYS
Other keys you want to add to the application .app file. The variable content is written as-is to the .app file, so be sure to format valid Erlang terms. For example: PROJECT_APP_EXTRA_KEYS = {maxT, 10000}, {start_phases, [...]}.
LOCAL_DEPS
List of Erlang/OTP applications this project depends on, excluding erts, kernel and stdlib, or list of dependencies local to this repository (in APPS_DIR).
DEPS
List of applications this project depends on that need to be fetched by Erlang.mk.

There’s no need for quotes or anything. The relevant part of the Cowboy Makefile follows, if you need an example:

PROJECT = cowboy
PROJECT_DESCRIPTION = Small, fast, modular HTTP server.
PROJECT_VERSION = 2.0.0-pre.2
PROJECT_REGISTERED = cowboy_clock

LOCAL_DEPS = crypto
DEPS = cowlib ranch

Any space before and after the value is dropped.

Dependencies are covered in details in the next chapter.

6.3.2. Application environment

The PROJECT_ENV variable is used to set the application environment:

define PROJECT_ENV
[
  {chips, [currysauce,{mushypeas,false}]},
  {pizza, [{size,large},{toppings,[anchovies]}]}
]
endef

If you have a large set of environment variables, you may find it easier to use a separate file. Do this by including the following in your Makefile:

PROJECT_ENV_FILE = src/env.src
PROJECT_ENV = $(subst \n,$(newline),$(shell cat $(PROJECT_ENV_FILE) | sed -e 's/$$/\\n/;'))
ebin/$(PROJECT).app:: $(PROJECT_ENV_FILE)

The file has the same contents as the PROJECT_ENV variable:

[
  {chips, [currysauce,{mushypeas,false}]},
  {pizza, [{size,large},{toppings,[anchovies]}]}
]

6.3.3. Legacy method

The src/$(PROJECT).app.src file is a legacy method of building Erlang applications. It was introduced by the original rebar build tool, of which Erlang.mk owes a great deal as it is its main inspiration.

The .app.src file serves as a template to generate the .app file. Erlang.mk will take it, fill in the modules value dynamically, and save the result in ebin/$(PROJECT).app.

When using this method, Erlang.mk cannot fill the applications key from dependencies automatically, which means you need to add them to Erlang.mk and to the .app.src at the same time, duplicating the work.

If you really can’t live without the legacy method, for one reason or another, worry not; Erlang.mk will support it. And if you need to create a new project that uses this method, you just have to say so when bootstrapping:

$ make -f erlang.mk bootstrap-lib LEGACY=1

6.4. Automatic application resource file values

When building the application resource file, Erlang.mk may automatically add an id key with information about the Git commit (if using Git), or an empty string otherwise. It will only do this under specific conditions:

  • The application was built as a dependency of another, or
  • The legacy method was used, and the .app.src file contained {id, "git"}

This value is most useful when you need to help your users, as it allows you to know which version they run exactly by asking them to look in the file, or by running a simple command on their production server:

1> application:get_all_key(cowboy).
{ok,[{description,"Small, fast, modular HTTP server."},
     {id,"2.0.0-pre.2-25-g0ffde50-dirty"},

6.5. File formats

Erlang.mk supports a variety of different source file formats. The following formats are supported natively:

Extension Location Description Output

.erl

src/

Erlang source

ebin/*.beam

.core

src/

Core Erlang source

ebin/*.beam

.xrl

src/

Leex source

src/*.erl

.yrl

src/

Yecc source

src/*.erl

.asn1

asn1/

ASN.1 files

include/.hrl include/.asn1db src/*.erl

.mib

mibs/

SNMP MIB files

include/.hrl priv/mibs/.bin

Files are always searched recursively.

The build is ordered, so that files that generate Erlang source files are run before, and the resulting Erlang source files are then built normally.

In addition, Erlang.mk keeps track of header files (.hrl) as described at the end of this chapter. It can also compile C code, as described in the NIFs and port drivers chapter.

Erlang.mk also comes with plugins for the following formats:

Extension Location Description Output

.dtl

templates/

Django templates

ebin/*.beam

.proto

src/

Protocol buffers

ebin/*.beam

6.6. Compilation options

Erlang.mk provides a few variables that you can use to customize the build process and the resulting files.

6.6.1. ERLC_OPTS

ERLC_OPTS can be used to pass some options to erlc, the Erlang compiler. Erlang.mk does not restrict any option. Please refer to the erlc Manual for the full list.

By default, Erlang.mk will set the following options:

ERLC_OPTS = -Werror +debug_info +warn_export_vars +warn_shadow_vars +warn_obsolete_guard

In other words: warnings as errors, debug info (recommended) and enable warnings for exported variables, shadow variables and obsolete guard functions.

You can redefine this variable in your Makefile to change it completely, either before or after including Erlang.mk:

ERLC_OPTS = +debug_info

You can also filter out some options from the defaults Erlang.mk sets, by defining ERLC_OPTS after including Erlang.mk using the := operator.

include erlang.mk

ERLC_OPTS := $(filter-out -Werror,$(ERLC_OPTS))

6.6.2. ERLC_ASN1_OPTS

ERLC_ASN1_OPTS can be used to pass compiler options when compiling ASN.1 files. Please refer to the asn1ct manual for the full list.

By default, Erlang.mk will leave this empty.

You can redefine this variable in your Makefile. Please see the ERLC_OPTS section for instructions.

6.6.3. ERLC_EXCLUDE

ERLC_EXCLUDE can be used to exclude some modules from the compilation. It’s there for handling special cases, you should not normally need it.

To exclude a module, simply list it in the variable, either before or after including Erlang.mk:

ERLC_EXCLUDE = cowboy_http2

6.7. Cold and hot builds

The first time you run make, Erlang.mk will build everything.

The second time you run make, and all subsequent times, Erlang.mk will only rebuild what changed. Erlang.mk has been optimized for this use case, as it is the most common during development.

Erlang.mk figures out what changed by using the dependency tracking feature of Make. Make automatically rebuilds a target if one of its dependency has changed (for example if a header file has changed, all the source files that include it will be rebuilt), and Erlang.mk leverages this feature to cut down on rebuild times.

Note that this applies only to building; some other features of Erlang.mk will run every time they are called regardless of files changed.

6.8. Dependency tracking

Note

This section is about the dependency tracking between files inside your project, not application dependencies.

Erlang.mk keeps track of the dependencies between the different files in your project. This information is kept in the $(PROJECT).d file in your directory. It is generated if missing, and will be generated again after every file change, by default.

Dependency tracking is what allows Erlang.mk to know when to rebuild Erlang files when header files, behaviors or parse transforms have changed. Erlang.mk also automatically keeps track of which files should be compiled first, for example when you have behaviors used by other modules in your project.

If your project is stable, you may want to disable generating the dependency tracking file every time you compile. You can do this by adding the following line to your Makefile:

NO_MAKEDEP ?= 1

As you can see, the snippet above uses ?= instead of a simple equal sign. This is to allow you to temporarily override this value when you do make substantial changes to your project (including a new header file, new module with dependencies, etc.) and want to rebuild the dependency tracking file. You’ll be able to use the following command:

$ NO_MAKEDEP= make

Otherwise, make clean app will of course force the recompilation of your project.

Erlang.mk can also keep track of the source files generated by other means, for example if you generate code from a data file in your repository.

6.9. Generating Erlang source

Erlang.mk provides hooks at different stages of the build process. When your goal is to generate Erlang source files, you can add your own rules before or after the dependency tracking file is generated. To do this, you would add your hook before or after including the erlang.mk file.

The easiest way is after:

PROJECT = example

include erlang.mk

$(PROJECT).d:: src/generated_mod.erl

src/generated_mod.erl:: gen-mod.sh
    $(gen_verbose) ./gen-mod.sh $@

In this case we use $(gen_verbose) to hide the details of the build by default. Erlang.mk will simply say what file is it currently generating.

When using an external script to generate the Erlang source file, it is recommended to depend on that script, so that the source file gets generated again when the script gets modified.

If for whatever reason you prefer to hook before including Erlang.mk, don’t forget to set the .DEFAULT_GOAL variable, otherwise nothing will get built:

PROJECT = example

.DEFAULT_GOAL = all

$(PROJECT).d:: src/generated_mod.erl

include erlang.mk

src/generated_mod.erl:: gen-mod.sh
    $(gen_verbose) ./gen-mod.sh $@

6.10. Cleaning

Building typically involves creating a lot of new files. Some are reused in rebuilds, some are simply replaced. All can be removed safely.

Erlang.mk provides two commands to remove them: clean and distclean. clean removes all the intermediate files that were created as a result of building, including the BEAM files, the dependency tracking file and the generated documentation. distclean removes these and more, including the downloaded dependencies, Dialyzer’s PLT file and the generated release, putting your directory back to the state it was before you started working on it.

To clean:

$ make clean

Or distclean:

$ make distclean

That is the question.

Note that Erlang.mk will automatically clean some files as part of other targets, but it will never run distclean if you don’t explicitly use it.