Nix Release Notes

-------------------------------------------------------------------------------

Release 0.11 (December 31, 2007)

Nix 0.11 has many improvements over the previous stable release. The most
important improvement is secure multi-user support. It also features many
usability enhancements and language extensions, many of them prompted by NixOS,
the purely functional Linux distribution based on Nix. Here is an (incomplete)
list:

  * Secure multi-user support. A single Nix store can now be shared between
    multiple (possible untrusted) users. This is an important feature for
    NixOS, where it allows non-root users to install software. The old setuid
    method for sharing a store between multiple users has been removed. Details
    for setting up a multi-user store can be found in the manual.

  * The new command nix-copy-closure gives you an easy and efficient way to
    exchange software between machines. It copies the missing parts of the
    closure of a set of store path to or from a remote machine via ssh.

  * A new kind of string literal: strings between double single-quotes ('')
    have indentation "intelligently" removed. This allows large strings (such
    as shell scripts or configuration file fragments in NixOS) to cleanly
    follow the indentation of the surrounding expression. It also requires much
    less escaping, since '' is less common in most languages than ".

  * nix-env --set modifies the current generation of a profile so that it
    contains exactly the specified derivation, and nothing else. For example,
    nix-env -p /nix/var/nix/profiles/browser --set firefox lets the profile
    named browser contain just Firefox.

  * nix-env now maintains meta-information about installed packages in
    profiles. The meta-information is the contents of the meta attribute of
    derivations, such as description or homepage. The command nix-env -q --xml
    --meta shows all meta-information.

  * nix-env now uses the meta.priority attribute of derivations to resolve
    filename collisions between packages. Lower priority values denote a higher
    priority. For instance, the GCC wrapper package and the Binutils package in
    Nixpkgs both have a file bin/ld, so previously if you tried to install both
    you would get a collision. Now, on the other hand, the GCC wrapper declares
    a higher priority than Binutils, so the former's bin/ld is symlinked in the
    user environment.

  * nix-env -i / -u: instead of breaking package ties by version, break them by
    priority and version number. That is, if there are multiple packages with
    the same name, then pick the package with the highest priority, and only
    use the version if there are multiple packages with the same priority.

    This makes it possible to mark specific versions/variant in Nixpkgs more or
    less desirable than others. A typical example would be a beta version of
    some package (e.g., gcc-4.2.0rc1) which should not be installed even though
    it is the highest version, except when it is explicitly selected (e.g.,
    nix-env -i gcc-4.2.0rc1).

  * nix-env --set-flag allows meta attributes of installed packages to be
    modified. There are several attributes that can be usefully modified,
    because they affect the behaviour of nix-env or the user environment build
    script:

      o meta.priority can be changed to resolve filename clashes (see above).

      o meta.keep can be set to true to prevent the package from being upgraded
        or replaced. Useful if you want to hang on to an older version of a
        package.

      o meta.active can be set to false to "disable" the package. That is, no
        symlinks will be generated to the files of the package, but it remains
        part of the profile (so it won't be garbage-collected). Set it back to
        true to re-enable the package.

  * nix-env -q now has a flag --prebuilt-only (-b) that causes nix-env to show
    only those derivations whose output is already in the Nix store or that can
    be substituted (i.e., downloaded from somewhere). In other words, it shows
    the packages that can be installed "quickly", i.e., don't need to be built
    from source. The -b flag is also available in nix-env -i and nix-env -u to
    filter out derivations for which no pre-built binary is available.

  * The new option --argstr (in nix-env, nix-instantiate and nix-build) is like
    --arg, except that the value is a string. For example, --argstr system
    i686-linux is equivalent to --arg system \"i686-linux\" (note that --argstr
    prevents annoying quoting around shell arguments).

  * nix-store has a new operation --read-log (-l) paths that shows the build
    log of the given paths.

  * Nix now uses Berkeley DB 4.5. The database is upgraded automatically, but
    you should be careful not to use old versions of Nix that still use
    Berkeley DB 4.4.

  * The option --max-silent-time (corresponding to the configuration setting
    build-max-silent-time) allows you to set a timeout on builds - if a build
    produces no output on stdout or stderr for the given number of seconds, it
    is terminated. This is useful for recovering automatically from builds that
    are stuck in an infinite loop.

  * nix-channel: each subscribed channel is its own attribute in the top-level
    expression generated for the channel. This allows disambiguation (e.g.
    nix-env -i -A nixpkgs_unstable.firefox).

  * The substitutes table has been removed from the database. This makes
    operations such as nix-pull and nix-channel --update much, much faster.

  * nix-pull now supports bzip2-compressed manifests. This speeds up channels.

  * nix-prefetch-url now has a limited form of caching. This is used by
    nix-channel to prevent unnecessary downloads when the channel hasn't
    changed.

  * nix-prefetch-url now by default computes the SHA-256 hash of the file
    instead of the MD5 hash. In calls to fetchurl you should pass the sha256
    attribute instead of md5. You can pass either a hexadecimal or a base-32
    encoding of the hash.

  * Nix can now perform builds in an automatically generated "chroot". This
    prevents a builder from accessing stuff outside of the Nix store, and thus
    helps ensure purity. This is an experimental feature.

  * The new command nix-store --optimise reduces Nix store disk space usage by
    finding identical files in the store and hard-linking them to each other.
    It typically reduces the size of the store by something like 25-35%.

  * ~/.nix-defexpr can now be a directory, in which case the Nix expressions in
    that directory are combined into an attribute set, with the file names used
    as the names of the attributes. The command nix-env --import (which set the
    ~/.nix-defexpr symlink) is removed.

  * Derivations can specify the new special attribute allowedReferences to
    enforce that the references in the output of a derivation are a subset of a
    declared set of paths. For example, if allowedReferences is an empty list,
    then the output must not have any references. This is used in NixOS to
    check that generated files such as initial ramdisks for booting Linux don't
    have any dependencies.

  * The new attribute exportReferencesGraph allows builders access to the
    references graph of their inputs. This is used in NixOS for tasks such as
    generating ISO-9660 images that contain a Nix store populated with the
    closure of certain paths.

  * Fixed-output derivations (like fetchurl) can define the attribute
    impureEnvVars to allow external environment variables to be passed to
    builders. This is used in Nixpkgs to support proxy configuration, among
    other things.

  * Several new built-in functions: builtins.attrNames, builtins.filterSource,
    builtins.isAttrs, builtins.isFunction, builtins.listToAttrs,
    builtins.stringLength, builtins.sub, builtins.substring, throw,
    builtins.trace, builtins.readFile.

Release 0.10.1 (October 11, 2006)

This release fixes two somewhat obscure bugs that occur when evaluating Nix
expressions that are stored inside the Nix store (NIX-67). These do not affect
most users.

Release 0.10 (October 6, 2006)

Note

This version of Nix uses Berkeley DB 4.4 instead of 4.3. The database is
upgraded automatically, but you should be careful not to use old versions of
Nix that still use Berkeley DB 4.3. In particular, if you use a Nix installed
through Nix, you should run

$ nix-store --clear-substitutes

first.

Warning

Also, the database schema has changed slighted to fix a performance issue (see
below). When you run any Nix 0.10 command for the first time, the database will
be upgraded automatically. This is irreversible.

  * nix-env usability improvements:

      o An option --compare-versions (or -c) has been added to nix-env --query
        to allow you to compare installed versions of packages to available
        versions, or vice versa. An easy way to see if you are up to date with
        what's in your subscribed channels is nix-env -qc \*.

      o nix-env --query now takes as arguments a list of package names about
        which to show information, just like --install, etc.: for example,
        nix-env -q gcc. Note that to show all derivations, you need to specify
        \*.

      o nix-env -i pkgname will now install the highest available version of
        pkgname, rather than installing all available versions (which would
        probably give collisions) (NIX-31).

      o nix-env (-i|-u) --dry-run now shows exactly which missing paths will be
        built or substituted.

      o nix-env -qa --description shows human-readable descriptions of
        packages, provided that they have a meta.description attribute (which
        most packages in Nixpkgs don't have yet).

  * New language features:

      o Reference scanning (which happens after each build) is much faster and
        takes a constant amount of memory.

      o String interpolation. Expressions like

        "--with-freetype2-library=" + freetype + "/lib"

        can now be written as

        "--with-freetype2-library=${freetype}/lib"

        You can write arbitrary expressions within ${...}, not just
        identifiers.

      o Multi-line string literals.

      o String concatenations can now involve derivations, as in the example
        "--with-freetype2-library=" + freetype + "/lib". This was not
        previously possible because we need to register that a derivation that
        uses such a string is dependent on freetype. The evaluator now properly
        propagates this information. Consequently, the subpath operator (~) has
        been deprecated.

      o Default values of function arguments can now refer to other function
        arguments; that is, all arguments are in scope in the default values
        (NIX-45).

      o Lots of new built-in primitives, such as functions for list
        manipulation and integer arithmetic. See the manual for a complete
        list. All primops are now available in the set builtins, allowing one
        to test for the availability of primop in a backwards-compatible way.

      o Real let-expressions: let x = ...; ... z = ...; in ....

  * New commands nix-pack-closure and nix-unpack-closure than can be used to
    easily transfer a store path with all its dependencies to another machine.
    Very convenient whenever you have some package on your machine and you want
    to copy it somewhere else.

  * XML support:

      o nix-env -q --xml prints the installed or available packages in an XML
        representation for easy processing by other tools.

      o nix-instantiate --eval-only --xml prints an XML representation of the
        resulting term. (The new flag --strict forces `deep' evaluation of the
        result, i.e., list elements and attributes are evaluated recursively.)

      o In Nix expressions, the primop builtins.toXML converts a term to an XML
        representation. This is primarily useful for passing structured
        information to builders.

  * You can now unambigously specify which derivation to build or install in
    nix-env, nix-instantiate and nix-build using the --attr / -A flags, which
    takes an attribute name as argument. (Unlike symbolic package names such as
    subversion-1.4.0, attribute names in an attribute set are unique.) For
    instance, a quick way to perform a test build of a package in Nixpkgs is
    nix-build pkgs/top-level/all-packages.nix -A foo. nix-env -q --attr shows
    the attribute names corresponding to each derivation.

  * If the top-level Nix expression used by nix-env, nix-instantiate or
    nix-build evaluates to a function whose arguments all have default values,
    the function will be called automatically. Also, the new command-line
    switch --arg name value can be used to specify function arguments on the
    command line.

  * nix-install-package --url URL allows a package to be installed directly
    from the given URL.

  * Nix now works behind an HTTP proxy server; just set the standard
    environment variables http_proxy, https_proxy, ftp_proxy or all_proxy
    appropriately. Functions such as fetchurl in Nixpkgs also respect these
    variables.

  * nix-build -o symlink allows the symlink to the build result to be named
    something other than result.

  * Platform support:

      o Support for 64-bit platforms, provided a suitably patched ATerm library
        is used. Also, files larger than 2 GiB are now supported.

      o Added support for Cygwin (Windows, i686-cygwin), Mac OS X on Intel
        (i686-darwin) and Linux on PowerPC (powerpc-linux).

      o Users of SMP and multicore machines will appreciate that the number of
        builds to be performed in parallel can now be specified in the
        configuration file in the build-max-jobs setting.

  * Garbage collector improvements:

      o Open files (such as running programs) are now used as roots of the
        garbage collector. This prevents programs that have been uninstalled
        from being garbage collected while they are still running. The script
        that detects these additional runtime roots (find-runtime-roots.pl) is
        inherently system-specific, but it should work on Linux and on all
        platforms that have the lsof utility.

      o nix-store --gc (a.k.a. nix-collect-garbage) prints out the number of
        bytes freed on standard output. nix-store --gc --print-dead shows how
        many bytes would be freed by an actual garbage collection.

      o nix-collect-garbage -d removes all old generations of all profiles
        before calling the actual garbage collector (nix-store --gc). This is
        an easy way to get rid of all old packages in the Nix store.

      o nix-store now has an operation --delete to delete specific paths from
        the Nix store. It won't delete reachable (non-garbage) paths unless
        --ignore-liveness is specified.

  * Berkeley DB 4.4's process registry feature is used to recover from crashed
    Nix processes.

  * A performance issue has been fixed with the referer table, which stores the
    inverse of the references table (i.e., it tells you what store paths refer
    to a given path). Maintaining this table could take a quadratic amount of
    time, as well as a quadratic amount of Berkeley DB log file space (in
    particular when running the garbage collector) (NIX-23).

  * Nix now catches the TERM and HUP signals in addition to the INT signal. So
    you can now do a killall nix-store without triggering a database recovery.

  * bsdiff updated to version 4.3.

  * Substantial performance improvements in expression evaluation and nix-env
    -qa, all thanks to Valgrind. Memory use has been reduced by a factor 8 or
    so. Big speedup by memoisation of path hashing.

  * Lots of bug fixes, notably:

      o Make sure that the garbage collector can run succesfully when the disk
        is full (NIX-18).

      o nix-env now locks the profile to prevent races between concurrent
        nix-env operations on the same profile (NIX-7).

      o Removed misleading messages from nix-env -i (e.g., installing `foo'
        followed by uninstalling `foo') (NIX-17).

  * Nix source distributions are a lot smaller now since we no longer include a
    full copy of the Berkeley DB source distribution (but only the bits we
    need).

  * Header files are now installed so that external programs can use the Nix
    libraries.

Release 0.9.2 (September 21, 2005)

This bug fix release fixes two problems on Mac OS X:

  * If Nix was linked against statically linked versions of the ATerm or
    Berkeley DB library, there would be dynamic link errors at runtime.

  * nix-pull and nix-push intermittently failed due to race conditions
    involving pipes and child processes with error messages such as open2: open
    (GLOB(0x180b2e4), >&=9) failed: Bad file descriptor at /nix/bin/nix-pull
    line 77 (issue NIX-14).

Release 0.9.1 (September 20, 2005)

This bug fix release addresses a problem with the ATerm library when the
--with-aterm flag in configure was not used.

Release 0.9 (September 16, 2005)

NOTE: this version of Nix uses Berkeley DB 4.3 instead of 4.2. The database is
upgraded automatically, but you should be careful not to use old versions of
Nix that still use Berkeley DB 4.2. In particular, if you use a Nix installed
through Nix, you should run

$ nix-store --clear-substitutes

first.

  * Unpacking of patch sequences is much faster now since we no longer do
    redundant unpacking and repacking of intermediate paths.

  * Nix now uses Berkeley DB 4.3.

  * The derivation primitive is lazier. Attributes of dependent derivations can
    mutually refer to each other (as long as there are no data dependencies on
    the outPath and drvPath attributes computed by derivation).

    For example, the expression derivation attrs now evaluates to (essentially)

    attrs // {
      type = "derivation";
      outPath = derivation! attrs;
      drvPath = derivation! attrs;
    }

    where derivation! is a primop that does the actual derivation instantiation
    (i.e., it does what derivation used to do). The advantage is that it allows
    commands such as nix-env -qa and nix-env -i to be much faster since they no
    longer need to instantiate all derivations, just the name attribute.

    Also, it allows derivations to cyclically reference each other, for
    example,

    webServer = derivation {
      ...
      hostName = "svn.cs.uu.nl";
      services = [svnService];
    };

    svnService = derivation {
      ...
      hostName = webServer.hostName;
    };

    Previously, this would yield a black hole (infinite recursion).

  * nix-build now defaults to using ./default.nix if no Nix expression is
    specified.

  * nix-instantiate, when applied to a Nix expression that evaluates to a
    function, will call the function automatically if all its arguments have
    defaults.

  * Nix now uses libtool to build dynamic libraries. This reduces the size of
    executables.

  * A new list concatenation operator ++. For example, [1 2 3] ++ [4 5 6]
    evaluates to [1 2 3 4 5 6].

  * Some currently undocumented primops to support low-level build management
    using Nix (i.e., using Nix as a Make replacement). See the commit messages
    for r3578 and r3580.

  * Various bug fixes and performance improvements.

Release 0.8.1 (April 13, 2005)

This is a bug fix release.

  * Patch downloading was broken.

  * The garbage collector would not delete paths that had references from
    invalid (but substitutable) paths.

Release 0.8 (April 11, 2005)

NOTE: the hashing scheme in Nix 0.8 changed (as detailed below). As a result,
nix-pull manifests and channels built for Nix 0.7 and below will now work
anymore. However, the Nix expression language has not changed, so you can still
build from source. Also, existing user environments continue to work. Nix 0.8
will automatically upgrade the database schema of previous installations when
it is first run.

If you get the error message

you have an old-style manifest `/nix/var/nix/manifests/[...]'; please
delete it

you should delete previously downloaded manifests:

$ rm /nix/var/nix/manifests/*

If nix-channel gives the error message

manifest `http://catamaran.labs.cs.uu.nl/dist/nix/channels/[channel]/MANIFEST'
is too old (i.e., for Nix <= 0.7)

then you should unsubscribe from the offending channel (nix-channel --remove 
URL; leave out /MANIFEST), and subscribe to the same URL, with channels
replaced by channels-v3 (e.g., http://catamaran.labs.cs.uu.nl/dist/nix/
channels-v3/nixpkgs-unstable).

Nix 0.8 has the following improvements:

  * The cryptographic hashes used in store paths are now 160 bits long, but
    encoded in base-32 so that they are still only 32 characters long (e.g., /
    nix/store/csw87wag8bqlqk7ipllbwypb14xainap-atk-1.9.0). (This is actually a
    160 bit truncation of a SHA-256 hash.)

  * Big cleanups and simplifications of the basic store semantics. The notion
    of "closure store expressions" is gone (and so is the notion of
    "successors"); the file system references of a store path are now just
    stored in the database.

    For instance, given any store path, you can query its closure:

    $ nix-store -qR $(which firefox)
    ... lots of paths ...

    Also, Nix now remembers for each store path the derivation that built it
    (the "deriver"):

    $ nix-store -qR $(which firefox)
    /nix/store/4b0jx7vq80l9aqcnkszxhymsf1ffa5jd-firefox-1.0.1.drv

    So to see the build-time dependencies, you can do

    $ nix-store -qR $(nix-store -qd $(which firefox))

    or, in a nicer format:

    $ nix-store -q --tree $(nix-store -qd $(which firefox))

    File system references are also stored in reverse. For instance, you can
    query all paths that directly or indirectly use a certain Glibc:

    $ nix-store -q --referrers-closure \
        /nix/store/8lz9yc6zgmc0vlqmn2ipcpkjlmbi51vv-glibc-2.3.4

  * The concept of fixed-output derivations has been formalised. Previously,
    functions such as fetchurl in Nixpkgs used a hack (namely, explicitly
    specifying a store path hash) to prevent changes to, say, the URL of the
    file from propagating upwards through the dependency graph, causing
    rebuilds of everything. This can now be done cleanly by specifying the
    outputHash and outputHashAlgo attributes. Nix itself checks that the
    content of the output has the specified hash. (This is important for
    maintaining certain invariants necessary for future work on secure shared
    stores.)

  * One-click installation :-) It is now possible to install any top-level
    component in Nixpkgs directly, through the web - see, e.g., http://
    catamaran.labs.cs.uu.nl/dist/nixpkgs-0.8/. All you have to do is associate
    /nix/bin/nix-install-package with the MIME type application/nix-package (or
    the extension .nixpkg), and clicking on a package link will cause it to be
    installed, with all appropriate dependencies. If you just want to install
    some specific application, this is easier than subscribing to a channel.

  * nix-store -r PATHS now builds all the derivations PATHS in parallel.
    Previously it did them sequentially (though exploiting possible parallelism
    between subderivations). This is nice for build farms.

  * nix-channel has new operations --list and --remove.

  * New ways of installing components into user environments:

      o Copy from another user environment:

        $ nix-env -i --from-profile .../other-profile firefox

      o Install a store derivation directly (bypassing the Nix expression
        language entirely):

        $ nix-env -i /nix/store/z58v41v21xd3...-aterm-2.3.1.drv

        (This is used to implement nix-install-package, which is therefore
        immune to evolution in the Nix expression language.)

      o Install an already built store path directly:

        $ nix-env -i /nix/store/hsyj5pbn0d9i...-aterm-2.3.1

      o Install the result of a Nix expression specified as a command-line
        argument:

        $ nix-env -f .../i686-linux.nix -i -E 'x: x.firefoxWrapper'

        The difference with the normal installation mode is that -E does not
        use the name attributes of derivations. Therefore, this can be used to
        disambiguate multiple derivations with the same name.

  * A hash of the contents of a store path is now stored in the database after
    a succesful build. This allows you to check whether store paths have been
    tampered with: nix-store --verify --check-contents.

  * Implemented a concurrent garbage collector. It is now always safe to run
    the garbage collector, even if other Nix operations are happening
    simultaneously.

    However, there can still be GC races if you use nix-instantiate and
    nix-store --realise directly to build things. To prevent races, use the
    --add-root flag of those commands.

  * The garbage collector now finally deletes paths in the right order (i.e.,
    topologically sorted under the "references" relation), thus making it safe
    to interrupt the collector without risking a store that violates the
    closure invariant.

  * Likewise, the substitute mechanism now downloads files in the right order,
    thus preserving the closure invariant at all times.

  * The result of nix-build is now registered as a root of the garbage
    collector. If the ./result link is deleted, the GC root disappears
    automatically.

  * The behaviour of the garbage collector can be changed globally by setting
    options in /nix/etc/nix/nix.conf.

      o gc-keep-derivations specifies whether deriver links should be followed
        when searching for live paths.

      o gc-keep-outputs specifies whether outputs of derivations should be
        followed when searching for live paths.

      o env-keep-derivations specifies whether user environments should store
        the paths of derivations when they are added (thus keeping the
        derivations alive).

  * New nix-env query flags --drv-path and --out-path.

  * fetchurl allows SHA-1 and SHA-256 in addition to MD5. Just specify the
    attribute sha1 or sha256 instead of md5.

  * Manual updates.

Release 0.7 (January 12, 2005)

  * Binary patching. When upgrading components using pre-built binaries
    (through nix-pull / nix-channel), Nix can automatically download and apply
    binary patches to already installed components instead of full downloads.
    Patching is "smart": if there is a sequence of patches to an installed
    component, Nix will use it. Patches are currently generated automatically
    between Nixpkgs (pre-)releases.

  * Simplifications to the substitute mechanism.

  * Nix-pull now stores downloaded manifests in /nix/var/nix/manifests.

  * Metadata on files in the Nix store is canonicalised after builds: the
    last-modified timestamp is set to 0 (00:00:00 1/1/1970), the mode is set to
    0444 or 0555 (readable and possibly executable by all; setuid/setgid bits
    are dropped), and the group is set to the default. This ensures that the
    result of a build and an installation through a substitute is the same; and
    that timestamp dependencies are revealed.

Release 0.6 (November 14, 2004)

  * Rewrite of the normalisation engine.

      o Multiple builds can now be performed in parallel (option -j).

      o Distributed builds. Nix can now call a shell script to forward builds
        to Nix installations on remote machines, which may or may not be of the
        same platform type.

      o Option --fallback allows recovery from broken substitutes.

      o Option --keep-going causes building of other (unaffected) derivations
        to continue if one failed.

  * Improvements to the garbage collector (i.e., it should actually work now).

  * Setuid Nix installations allow a Nix store to be shared among multiple
    users.

  * Substitute registration is much faster now.

  * A utility nix-build to build a Nix expression and create a symlink to the
    result int the current directory; useful for testing Nix derivations.

  * Manual updates.

  * nix-env changes:

      o Derivations for other platforms are filtered out (which can be
        overriden using --system-filter).

      o --install by default now uninstall previous derivations with the same
        name.

      o --upgrade allows upgrading to a specific version.

      o New operation --delete-generations to remove profile generations
        (necessary for effective garbage collection).

      o Nicer output (sorted, columnised).

  * More sensible verbosity levels all around (builder output is now shown
    always, unless -Q is given).

  * Nix expression language changes:

      o New language construct: with E1; E2 brings all attributes defined in
        the attribute set E1 in scope in E2.

      o Added a map function.

      o Various new operators (e.g., string concatenation).

  * Expression evaluation is much faster.

  * An Emacs mode for editing Nix expressions (with syntax highlighting and
    indentation) has been added.

  * Many bug fixes.

Release 0.5 and earlier

Please refer to the Subversion commit log messages.

