Hacking Your First Package¶

The previous section essentially introduced Nix at the user level: How to install, search and remove packages. From this perspective, its usage does not fundamentally differ from another package manager. In this section, we will see that Nix becomes a powerful tool when it comes to software development.

More specifically, this section showcases nix-build and nix-shell usage. The former builds a package while the later enables entering into an interactive session containing the package’s build dependencies.

In this section, we will work on a demo project that we have extracted from the SimGrid simulator repository. In brief, this demo project simulates a Chord distributed hash table algorithm. The project is implemented in C++, uses SimGrid and Boost as dependencies, and requires the CMake build system and a C++ compiler. The project is available in the Demo project git repository.

A first Nix package definition¶

In Nix, packages are defined in a domain specific language soberly entitled the “Nix Expression Language”. This is a simple functional language whose syntax is a bit weird, but that is very efficient to define packages in the end. Here, we will present an explicit Nix package to introduce package definition in Nix. learnNixInYMinutes gives an example that quickly presents an overview of the Nix Expression Language. More information about the language are found on the Nix Expression Language page on NixOS’s wiki or on the official Nix Expression Language documentation.

To better fit Nix terminology, we will talk about derivations instead of packages here. A derivation is a function that describes a build process (precise definition: Nix’s documentation about derivations). The following file gives an example of a derivation.

chord_example.nix¶

{
  pkgs ? import (fetchTarball {
    url = "https://github.com/NixOS/nixpkgs/archive/4fe8d07066f6ea82cda2b0c9ae7aee59b2d241b3.tar.gz";
    sha256 = "sha256:06jzngg5jm1f81sc4xfskvvgjy5bblz51xpl788mnps1wrkykfhp";
  }) {}
}:
pkgs.stdenv.mkDerivation rec {
  pname = "chord";
  version = "0.1.0";

  src = pkgs.fetchgit {
    url = "https://gitlab.inria.fr/nix-tutorial/chord-tuto-nix-2022";
    rev = "069d2a5bfa4c4024063c25551d5201aeaf921cb3";
    sha256 = "sha256-MlqJOoMSRuYeG+jl8DFgcNnpEyeRgDCK2JlN9pOqBWA=";
  };

  buildInputs = [
    pkgs.simgrid
    pkgs.boost
    pkgs.cmake
  ];

  configurePhase = ''
    cmake .
  '';

  buildPhase = ''
    make
  '';

  installPhase = ''
    mkdir -p $out/bin
    mv chord $out/bin
  '';
}

The first lines (between the {} curly braces) define the inputs of the function. Here, the function has a single input: The pkgs variable. ? gives a default value to pkgs if the variable is not set at evaluation time. The default value provided for pkgs in this example is important, as it shows how to download a tarball. Here, pkgs is the top-level tree of Nixpkgs, which contains functions to help in building packages and a (big) set of packages. It is an alternative of using a channel, and is much more reproducible as the tarball can be fixed to a specific version as it is done here.

Note

The pkgs imported here is a snapshot of the unstable nixpkgs channel on the 4fe8d07066f6ea82cda2b0c9ae7aee59b2d241b3 commit. The nixpkgs tree of this commit can be traversed here.

For reproducibility sake, we also give the sha256 of the tarball we expect to fetch. The usual method is to pass lib.fakeSha256 (which evaluates to sha256:0000000000000000000000000000000000000000000000000000) to fetchTarball and try to build the package. Nix will then return an error indicating the actual sha256 of the tarball:

$ nix-build chord_example.nix
error: hash mismatch in file downloaded from 'https://github.com/NixOS/nixpkgs/archive/4fe8d07066f6ea82cda2b0c9ae7aee59b2d241b3.tar.gz':
         specified: sha256:0000000000000000000000000000000000000000000000000000
         got:       sha256:06jzngg5jm1f81sc4xfskvvgjy5bblz51xpl788mnps1wrkykfhp

Replace the dummy sha256 with the actual one in the fetchTarball function to build the package.

The rest of this example file defines a single derivation thanks to the mkDerivation function. mkDerivation takes a set as input and expects many attributes within it (see Sets in official NEL documentation).

First, the pname and version attributes are concatenated to form the package name (chord-0.1.0). Alternatively, we could have explicitly defined a name attribute instead.

The src attribute is mandatory and needs to point to the directory that contains the source code. In our case, we use the fetchurl function to download the archive that contains the source from the inria gitlab server (using the git commit defined in rev). fetchurl downloads the file, unpacks it if necessary, then ensures that the hash of the download content matches the expected sha256.

Note

Many other fetchers than fetchurl exist. The most common fetchers are listed on Nixpkgs’s documentation on fetchers.

The buildInputs attribute contains the list of packages required to build the derivation. The packages listed in this variable will be accessible from the build environment. In our case, you can see that cmake, simgrid and boost are needed to build the package.

Last but not least, the other attributes define some of the phases that build the package. Nix separates the build process in several phases, that are concatenated to form a build script. This seems strange at first but this is quite convenient, as usually only a small subpart of the package build process needs to be changed. Please refer to Nix’s documentation on build phases for more information about phases.

Note

In this example, overridding the phases is not required. This is only done for pedagogical purpose to make the phases explicit.

Nix changes its default behavior depending on the buildInputs. In this case, Nix does the default build script for CMake, which is essentially cmake && make && make install. Very curious readers may read Nixpkgs’ CMake setup-hook.sh script to figure out what’s under the hood.

Building the package (nix-build)¶

Now we can try to build this derivation with nix-build and see what happens. The first step is to copy or download the above chord_example.nix. Move into the folder containing the derivation and execute the following command:

nix-build chord_example.nix

This command should build the derivation in a pure environment where only the defined buildInputs are available. The resulting package resides in the nix store. If the package built successfully, nix-build creates a ./result link that points to the package.

As a result, the following command should display the help menu of the chord binary.

./result/bin/chord --help

Diving into environments (nix-shell)¶

Build Environment¶

The command nix-shell is a powerful tool for developers. In a nutshell, it starts a new shell into the build environment of a package. Starting from the previous chord_example.nix, you can run the following command to enter the build environment of the derivation.

nix-shell --pure chord_example.nix

Within the shell you have access to simgrid, boost and cmake, which are the dependencies specified into the buildInputs attribute. The --pure option asks nix-shell to start setting up the environment from clean environment variables. By default, it starts its setup from the current environment variables.

You can give a look at the environment variables to understand what Nix does to environment variables within a shell. In addition to standard environment variables such as $PATH, Nix sets various variables depending on which packages are set in buildInputs. Here, as cmake is used, Nix automatically sets many CMake-related environment variables such as $CMAKE_LIBRARY_PATH.

echo ${PATH} | tr ':' '\n' | sort -u
echo ${CMAKE_LIBRARY_PATH} | tr ':' '\n' | sort -u

To leave the shell, simply use Ctrl+D, or run:

exit

Now, we can try to compile the example project made for this tutorial. Leave the previous nix-shell if this is not already done. Then, clone the project with the following command:

git clone https://gitlab.inria.fr/nix-tutorial/chord-tuto-nix-2022.git /tmp/chord-tuto-nix-2022

Put the previous chord_example.nix into the cloned repository, then move into the repository:

cd /tmp/chord-tuto-nix-2022

Finally, enter into the build environment thanks to nix-shell chord_example.nix then run the following commands inside the shell.

mkdir build
cd build
cmake ..
make

Alternatively, you could use the --command option of nix-shell to run a command inside the desired environment. Delete the build directory (rm -rf ./build) then run the following command:

nix-shell chord_example.nix --command 'mkdir build && cd build && cmake .. && make'

Generic environment¶

Initially, nix-shell was designed to enter a package’s build environment for debugging purpose. However, nix-shell can also be used to enter an custom environment defined by the mkShell function.

shell.nix¶

# This shell defines a development environment for the Chord project.
{
  pkgs ? import (fetchTarball {
    url = "https://github.com/NixOS/nixpkgs/archive/4fe8d07066f6ea82cda2b0c9ae7aee59b2d241b3.tar.gz";
    sha256 = "sha256:06jzngg5jm1f81sc4xfskvvgjy5bblz51xpl788mnps1wrkykfhp";
  }) {}
}:
pkgs.mkShell rec {
   buildInputs = with pkgs; [
    cmake
    boost
    simgrid

    # debugging tools
    gdb
    valgrind
   ];
}

Once you have downloaded shell.nix, you can enter into the shell simply by using:

nix-shell shell.nix

Note

If no file is specified to nix-shell, it will look for shell.nix and default.nix (in this order). Thus, simply typing nix-shell would also work in our case.