Working with notebooks

Notebooks can be valuable assets for experimenters, notably since they can bind results and their analysis together. This section shows how some notebook technologies can be used in conjunction with Nix, thus improving the notebook reproducibility.

Jupyter notebook

Jupyter is a popular web-based Python notebook, formerly called IPython Notebook. The example is stored in the notebook-jupyter directory of the Chord experiments git repository. The directory contains a Jupyter file and the following Nix shell.

notebook-jupyter/shell.nix

{
  tinypkgs ? import (fetchTarball {
    url = "https://gitlab.inria.fr/nix-tutorial/packages-repository/-/archive/master/packages-repository-8e43243635cd8f28c7213205b08c12f2ca2ac74d.tar.gz";
    sha256 = "sha256:09l2w3m1z0308zc476ci0bsz5amm536hj1n9xzpwcjm59jxkqpqa";
  }) {}
}:

with tinypkgs; # Put tinypkgs's attributes in the current scope.
with pkgs; # Same for pkgs.

mkShell {
  buildInputs = [
    chord

    # Defines a python + set of packages.
    (python3.withPackages (ps: with ps; with python3Packages; [
      jupyter
      ipython

      # Uncomment the following lines to make them available in the shell.
      # pandas
      # numpy
      # matplotlib
    ]))
  ];

  # Automatically run jupyter when entering the shell.
  shellHook = "jupyter notebook";
}

This shell points to the package repository we have done in section Experiment Packaging: Don’t Repeat Yourself and defines a single shell from which all desired packages are set. This example introduces a Python environment in Nix, which is here done thanks to python?.withPackages, which in short defines a python interpreter with a set of available packages. Please refer to NixOS’s wiki on Python and Nix’s official Python documentation for more information about using Python and Nix.

Running jupyter simply consists in running nix-shell in the notebook-jupyter directory. This should run a Jupyter server, which will also (in a non-pure shell) run your brower so you can interact with Jupyter.

R markdown

R markdown is a popular R notebook that can be used with or without RStudio. The example is stored in the notebook-rmarkdown directory of the Chord experiments git repository. This directory contains a R markdown file, a nix-shell and a runner script. More information on using R with Nix can be found on NixOS wiki’s page on R.

notebook-rmarkdown/shell.nix

{
  pkgs ? import (fetchTarball {
    url = "https://github.com/NixOS/nixpkgs/archive/4fe8d07066f6ea82cda2b0c9ae7aee59b2d241b3.tar.gz";
    sha256 = "sha256:06jzngg5jm1f81sc4xfskvvgjy5bblz51xpl788mnps1wrkykfhp";
  }) {}
}:

let
  r_pkgs = with pkgs.rPackages; [
    # rmarkdown-related packages.
    knitr
    rmarkdown
    tidyverse
    viridis
    # Rstudio-related packages.
    # servr
  ];
in
pkgs.mkShell {
  buildInputs = [
    pkgs.pandoc
    (pkgs.rWrapper.override {
      packages = r_pkgs;
    })

    # Uncomment to add RStudio in your environment.
    # (pkgs.rstudioWrapper.override {
    #  packages = r_pkgs;
    # })
  ];
}

notebook-rmarkdown/runner.R

#!/usr/bin/env nix-shell
#!nix-shell --pure -i Rscript
rmarkdown::render("experiment.Rmd")

As the runner uses a nix-shell shebang, generating the notebook output is done like this.

./runner.R

Emacs’s org-mode

org-mode is a popular GNU Emacs package that can be used as a notebook technology. The example is stored in the notebook-org-mode directory of the Chord experiments git repository. The directory contains an org-mode file, and a nix-shell. More information on using Emacs with Nix can be found on NixOS manual on Emacs.

notebook-org-mode/shell.nix

{
  pkgs ? import (fetchTarball {
    url = "https://github.com/NixOS/nixpkgs/archive/4fe8d07066f6ea82cda2b0c9ae7aee59b2d241b3.tar.gz";
    sha256 = "sha256:06jzngg5jm1f81sc4xfskvvgjy5bblz51xpl788mnps1wrkykfhp";
  }) {}
}:

let
  emacs_base = pkgs.emacs;
  emacsWithPackages = (pkgs.emacsPackagesFor emacs_base).emacsWithPackages;

  self = rec {
    my_emacs = emacsWithPackages (epkgs: with epkgs; [
      org
      org-ref
    ]);

    shell = pkgs.mkShell rec {
      name = "emacs-shell";
      buildInputs = [
        my_emacs
      ];
    };
  };
in
  self.shell

Emacs can be executed on the org-mode file with the following command.

nix-shell --pure shell.nix --command 'emacs -q ./experiment.org'

Once in Emacs, exporting the notebook can be done with the convenient C-c C-e h h Emacs shortcut.