Hacking Your First Package -------------------------- .. _section-description-1: The previous section essentially introduced Nix at the user level: How to install, search and remove packages. From this perspective, its usage does not fundamentally differ from another package manager. In this section, we will see that Nix becomes a powerful tool when it comes to software development. More specifically, this section showcases ``nix-build`` and ``nix-shell`` usage. The former builds a package while the later enables entering into an interactive session containing the package's build dependencies. In this section, we will work on a demo project that we have extracted from the SimGrid_ simulator repository. In brief, this demo project simulates a Chord_ distributed hash table algorithm. The project is implemented in C++, uses SimGrid_ and Boost_ as dependencies, and requires the CMake_ build system and a C++ compiler. The project is available in the `Demo project git repository`_. A first Nix package definition ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ .. To better understand what ``nix-shell`` does, lets first look at the definition of a packages: In Nix, packages are defined in a domain specific language soberly entitled the "Nix Expression Language". This is a simple functional language whose syntax is a bit *weird*, but that is very efficient to define packages in the end. Here, we will present an explicit Nix package to introduce package definition in Nix. learnNixInYMinutes_ gives an example that quickly presents an overview of the Nix Expression Language. More information about the language are found on the `Nix Expression Language page on NixOS's wiki`_ or on the `official Nix Expression Language documentation`_. To better fit Nix terminology, we will talk about *derivations* instead of *packages* here. A derivation is a function that describes a build process (precise definition: `Nix's documentation about derivations`_). The following file gives an example of a derivation. .. literalinclude:: ./examples/chord_example.nix :caption: :download:`chord_example.nix <./examples/chord_example.nix>` :language: nix The first lines (between the ``{}`` curly braces) define the inputs of the function. Here, the function has a single input: The ``pkgs`` variable. ``?`` gives a default value to ``pkgs`` if the variable is not set at evaluation time. The default value provided for ``pkgs`` in this example is important, as it shows how to download a tarball. Here, ``pkgs`` is the top-level tree of Nixpkgs, which contains functions to help in building packages and a (big) set of packages. It is an alternative of using a channel, and is much more reproducible as the tarball can be fixed to a specific version as it is done here. .. note:: The ``pkgs`` imported here is a snapshot of the `unstable` nixpkgs channel on the ``4fe8d07066f6ea82cda2b0c9ae7aee59b2d241b3`` commit. The nixpkgs tree of this commit can be traversed `here `_. For reproducibility sake, we also give the sha256 of the tarball we expect to fetch. The usual method is to pass ``lib.fakeSha256`` (which evaluates to ``sha256:0000000000000000000000000000000000000000000000000000``) to ``fetchTarball`` and try to build the package. Nix will then return an error indicating the actual ``sha256`` of the tarball: .. code-block:: bash $ nix-build chord_example.nix error: hash mismatch in file downloaded from 'https://github.com/NixOS/nixpkgs/archive/4fe8d07066f6ea82cda2b0c9ae7aee59b2d241b3.tar.gz': specified: sha256:0000000000000000000000000000000000000000000000000000 got: sha256:06jzngg5jm1f81sc4xfskvvgjy5bblz51xpl788mnps1wrkykfhp Replace the dummy `sha256` with the actual one in the ``fetchTarball`` function to build the package. The rest of this example file defines a single derivation thanks to the ``mkDerivation`` function. ``mkDerivation`` takes a *set* as input and expects many *attributes* within it (see `Sets in official NEL documentation`_). First, the ``pname`` and ``version`` attributes are concatenated to form the package ``name`` (``chord-0.1.0``). Alternatively, we could have explicitly defined a ``name`` attribute instead. The ``src`` *attribute* is mandatory and needs to point to the directory that contains the source code. In our case, we use the ``fetchurl`` function to download the archive that contains the source from the inria gitlab server (using the git commit defined in ``rev``). ``fetchurl`` downloads the file, unpacks it if necessary, then ensures that the hash of the download content matches the expected ``sha256``. .. note:: Many other fetchers than ``fetchurl`` exist. The most common fetchers are listed on `Nixpkgs's documentation on fetchers`_. The ``buildInputs`` *attribute* contains the list of packages required to build the derivation. The packages listed in this variable will be accessible from the build environment. In our case, you can see that ``cmake``, ``simgrid`` and ``boost`` are needed to build the package. Last but not least, the other *attributes* define some of the phases that build the package. Nix separates the build process in several *phases*, that are concatenated to form a build script. This seems strange at first but this is quite convenient, as usually only a small subpart of the package build process needs to be changed. Please refer to `Nix's documentation on build phases`_ for more information about phases. .. note:: In this example, overridding the phases is **not required**. This is only done for pedagogical purpose to make the phases explicit. Nix changes its default behavior depending on the ``buildInputs``. In this case, Nix does the default build script for CMake_, which is essentially ``cmake && make && make install``. Very curious readers may read `Nixpkgs' CMake setup-hook.sh script`_ to figure out what's under the hood. Building the package (nix-build) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Now we can try to build this derivation with ``nix-build`` and see what happens. The first step is to copy or download the above :download:`chord_example.nix <./examples/chord_example.nix>`. Move into the folder containing the derivation and execute the following command: .. literalinclude:: ./first-package/build-from-nix-build.bash :lines: 2 :language: bash This command should build the derivation in a pure environment where only the defined ``buildInputs`` are available. The resulting package resides in the nix store. If the package built successfully, ``nix-build`` creates a ``./result`` link that points to the package. As a result, the following command should display the help menu of the chord binary. .. code-block:: bash ./result/bin/chord --help .. That is to say, it will build the package build dependency if it is necessary and create an interactive session to log into. .. One can relate ``nix-shell`` to ``virtualenv`` or ``docker``. .. The example bellow enter into a new shell containing all dependencies necessary to build Simgrid. Diving into environments (nix-shell) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ .. Integrate Within a Development Workflow Build Environment +++++++++++++++++ The command ``nix-shell`` is a powerful tool for developers. In a nutshell, it starts a new shell into the build environment of a package. Starting from the previous :download:`chord_example.nix <./examples/chord_example.nix>`, you can run the following command to enter the build environment of the derivation. .. code-block:: bash nix-shell --pure chord_example.nix Within the shell you have access to `simgrid`, `boost` and `cmake`, which are the dependencies specified into the ``buildInputs`` attribute. The ``--pure`` option asks ``nix-shell`` to start setting up the environment from clean environment variables. By default, it starts its setup from the current environment variables. You can give a look at the environment variables to understand what Nix does to environment variables within a shell. In addition to standard environment variables such as ``$PATH``, Nix sets various variables depending on which packages are set in ``buildInputs``. Here, as ``cmake`` is used, Nix automatically sets many CMake-related environment variables such as ``$CMAKE_LIBRARY_PATH``. .. code-block:: bash echo ${PATH} | tr ':' '\n' | sort -u echo ${CMAKE_LIBRARY_PATH} | tr ':' '\n' | sort -u To leave the shell, simply use `Ctrl+D`, or run: .. code-block:: bash exit Now, we can try to compile the example project made for this tutorial. **Leave the previous nix-shell if this is not already done**. Then, clone the project with the following command: .. literalinclude:: ./first-package/retrieve-example-repo.bash :lines: 3 :language: bash Put the previous :download:`chord_example.nix <./examples/chord_example.nix>` into the cloned repository, then move into the repository: .. literalinclude:: ./first-package/move-into-example-repo.bash :lines: 2 :language: bash Finally, enter into the build environment thanks to ``nix-shell chord_example.nix`` then run the following commands inside the shell. .. code-block:: bash mkdir build cd build cmake .. make Alternatively, you could use the ``--command`` option of ``nix-shell`` to run a command inside the desired environment. Delete the build directory (``rm -rf ./build``) then run the following command: .. literalinclude:: ./first-package/build-from-shell-build-env.bash :lines: 2 :language: bash Generic environment +++++++++++++++++++ .. _section-description-2: .. The objective of this section is to show how one can leverage Nix, and .. more especially ``nix-shell`` , during the development phase of a .. software project. Initially, ``nix-shell`` was designed to enter a package's build environment for debugging purpose. However, ``nix-shell`` can also be used to enter an custom environment defined by the ``mkShell`` function. .. literalinclude:: ./examples/shell.nix :caption: :download:`shell.nix <./examples/shell.nix>` :language: nix Once you have downloaded :download:`shell.nix <./examples/shell.nix>`, you can enter into the shell simply by using: .. code:: bash nix-shell shell.nix .. note:: If no file is specified to ``nix-shell``, it will look for ``shell.nix`` and ``default.nix`` (in this order). Thus, simply typing ``nix-shell`` would also work in our case. .. _SimGrid: https://simgrid.frama.io/ .. _Chord: https://en.wikipedia.org/wiki/Chord_(peer-to-peer) .. _Boost: https://www.boost.org/ .. _CMake: https://cmake.org/ .. _Demo project git repository: https://gitlab.inria.fr/nix-tutorial/chord-tuto-nix-2022 .. _Nix Expression Language page on NixOS's wiki: https://nixos.wiki/wiki/Nix_Expression_Language .. _official Nix Expression Language documentation: https://nixos.org/nix/manual/#ch-expression-language .. _Sets in official NEL documentation: https://nixos.org/manual/nix/stable/expressions/language-values.html#sets .. _Nix's documentation about derivations: https://nixos.org/nix/manual/#ssec-derivation .. _Nix's documentation on build phases: https://nixos.org/nixpkgs/manual/#sec-stdenv-phases .. _Nixpkgs' CMake setup-hook.sh script: https://github.com/NixOS/nixpkgs/blob/4fe8d07066f6ea82cda2b0c9ae7aee59b2d241b3/pkgs/development/tools/build-managers/cmake/setup-hook.sh .. _Nixpkgs's documentation on fetchers: https://nixos.org/manual/nixpkgs/stable/#chap-pkgs-fetchers .. _learnNixInYMinutes: https://learnxinyminutes.com/docs/nix/