Nov 16, 2023 - 6 min read

Generating documentation from zig build

Background

So a few days ago, I began following the second part of Robert Nystrom's Crafting Interpreters. The author is doing it in C, but I figured Zig would be the better tool because I get to enjoy some modern features like the build system while also having the low-level power of C. I took this project to also serve as a means to get more familiar with Zig. So, I have been forcing myself to go on whatever tangent ideas that occur to me. One of those ideas was a way to generate documentation on demand.

I had done the first part of the book in Rust, so I was very used to cargo and running cargo doc to generate my package's documentation. Zig has the -femit-docs flag (with zig build-exe) that triggers generation of documentation when building an executable. However, I couldn't find a way to do the same with the build system. The default build.zig file created when we run zig init-exe doesn't enable emitting docs. The one place where I found a way to enable it was apparently also outdated. To clarify, what I wanted was something similar to cargo doc where running something like zig build docs would generate the documentation for me.

For some reason1 I got stuck on this for days although I did stumble across some great articles along the way, and some nice folks on IRC who helped me figure this out. All in all, I feel like I have a better understanding of not just the build system but Zig in general because of this endeavour.

I started this blog promising myself that I'd document the things I learned. So, I figured I should write an article about it; summarising this little rabbit hole I just came out of.

The build binary

Couldn't I just pass the regular -femit-docs flag to zig build? Apparently not.

Executing zig build is actually a two-step process. First, zig builds a binary from the lib/build_runner.zig file. We can see that the file is supposed to be an executable, due to the presence of an entrypoint (pub fn main() !void)2.

Reading its contents, we can infer the arguments that the executable expects. Some of those arguments may even seem familiar to that of zig itself, but it's a completely different binary with its own set of arguments.

So, this binary combined with the build.zig file that we have defined is what's going to actually build our project. Two steps: first the build binary, then our binary.

Steps

The whole build system is modelled as a directed acyclic graph composed of Steps. The build function in our build.zig file is given a pointer to an instance of std.build.Build. Inside the function, we declaratively define a set of Steps that the build binary should execute. The build function doesn't actually execute anything, it only defines the steps and their dependencies.

The documentation for Step shows that every step has an id field which corresponds to the type of the step. A top level step is one that can be invoked by a name: zig build <name>. Remembering our goal - getting zig build docs to work - we probably should look into creating a top level step.

The Install step

Let's first look at what we have. Here's a minimal build.zig that we're starting with:

const std = @import("std");

pub fn build(b: *std.Build) void {
    const exe = b.addExecutable(.{
        .name = "myprogram",
        .root_source_file = .{ .path = "src/main.zig" },
        .target = b.standardTargetOptions(.{}),
        .optimize = b.standardOptimizeOption(.{}),
    });
    b.installArtifact(exe);
}

Running zig build -l with this file gives us the two default top-level steps: install and uninstall. The purpose of the Install step is to copy build artifacts to a prefix path. A build artifact is any file or directory that is created during the build process which isn't the primary output of the program. This includes things like object files, libraries, and executables. The prefix path is usually just ./zig-out/.

Defining a step by itself doesn't do anything. It's the dependencies it has that specify what to install.

In the example above, we call the addExecutable function. Here's its function signature:

fn addExecutable(b: *Build, options: ExecutableOptions) *Step.Compile

It returns a Step.Compile pointer. That function creates a step declaring intent for producing an executable. That pointer (stored as the exe constant) is then passed to installArtifact():

/// This creates a new step and adds it to the dependencies of the top-level install step, using all the default options.
fn installArtifact(self: *Build, artifact: *Step.Compile) void

It should be clear from the doc comment that the function adds a dependency to the install step. There are functions for installing a file or a directory as well: installFile(), installDirectory().

Creating a step

If you've used zig init-exe before, you should find that the default build.zig file it creates has several additional top-level steps defined. Those steps are defined with the step() function:

fn step(self: *Build, name: []const u8, description: []const u8) *Step

You should be able to guess by now about how I want to relate that information to getting zig build docs. Let's create ourselves a docs step:

pub fn build(b: *std.Build) void {
    ...
    const docs_step = b.step("docs", "Copy documentation to prefix path");
}

Here's what we have now:

❯ zig build -l
  install (default)            Copy build artifacts to prefix path
  uninstall                    Remove build artifacts from prefix path
  docs                         Copy documentation to prefix path

Again, this step by itself doesn't do anything. So let's dig deeper.

Triggering documentation generation

If we pass the --verbose flag to zig build, it prints us the command that the build binary is executing:

❯ zig build --verbose
/home/dev/bin/zig build-exe /home/dev/myprogram/src/main.zig
--cache-dir /home/dev/myprogram/zig-cache
--global-cache-dir /home/dev/.cache/zig
--name myprogram --listen=-

We now need to find a way to add the -femit-docs flag to that output. Alas, as much as I'd enjoy showing a gradual path to finding these things out, I'll have to eventually come to reality. Like all of what I've just explained, the actual path to learning these things isn't linear at all. It wasn't until someone nicked ifreund on Zig's IRC channel gave me the pointers did I figure this out.

Our exe constant is a pointer to Step.Compile which contains a member function:

// Returns the path to the generated documentation directory.
fn getEmittedDocs(self: *Compile) LazyPath

I've copied the comment from the source as-is. This is what made me stuck: I misunderstood the comment to mean that there's already a generated path and calling the function just returns a path to it. However, if we look at the source:

pub fn getEmittedDocs(self: *Compile) LazyPath {
    return self.getEmittedFileGeneric(&self.generated_docs);
}
fn getEmittedFileGeneric(self: *Compile, output_file: *?*GeneratedFile) LazyPath {
    if (output_file.*) |g| {
        return .{ .generated = g };
    }
    const arena = self.step.owner.allocator;
    const generated_file = arena.create(GeneratedFile) catch @panic("OOM");
    generated_file.* = .{ .step = &self.step };
    output_file.* = generated_file;
    return .{ .generated = generated_file };
}

Now, I am not comfortable in Zig enough to be certain about all of the code but here's what I can analyze:

  • The self.generated_docs field, being passed to getEmittedFileGeneric(), supposedly contains the final path to the generated documentation.
  • Inside getEmittedFileGeneric(), the first check ensures we don't generate files more than once.
  • The rest of the body creates or obtains a GeneratedFile and returns it as part of a LazyPath. All in all, we can infer that calling exe.getEmittedDocs() actually triggers the documentation generation and gives us the path.

Installing the documentation

We have a path to the documentation, but it won't be in our prefix path. As mentioned above, we need to define directories to be installed to the Install step. Let's update our build.zig file.

Now, there are two ways to go about doing that. The first one:

pub fn build(b: *std.Build) void {
    ...
    b.installDirectory(.{
        .source_dir = exe.getEmittedDocs(),
        .install_dir = .prefix,
        .install_subdir = "docs",
    });
}

Doing this would add a dependency to the top-level Install step so that the docs are installed every time the Install step is performed. However, we want to generate documentation on demand. Heck, we haven't even used the docs step we created. So, we have to use a more flexible approach.

Adding dependency to the docs step

The build system offers this function:

fn addInstallDirectory(self: *Build, options: InstallDirectoryOptions) *Step.InstallDir

This is similar to installDirectory() but returns us a Step rather than automatically adding the specified directory as a dependency to the Install step. Now we can choose which Step this InstallDir step should depend on. As you may have guessed, we're going to make it depend on our docs step.

pub fn build(b: *std.Build) void {
    ...
    const install_docs = b.addInstallDirectory(.{
        .source_dir = exe.getEmittedDocs(),
        .install_dir = .prefix,
        .install_subdir = "docs",
    });

    const docs_step = b.step("docs", "Copy documentation artifacts to prefix path");
    docs_step.dependOn(&install_docs.step);
}

And there you go! Having this as our build.zig gives us the zig build docs command. If we do a zig build --verbose docs, we should see the -femit-docs we wanted:

❯ zig build docs --verbose
/home/dev/bin/zig build-exe /home/dev/myprogram/src/main.zig -femit-docs
--cache-dir /home/dev/myprogram/zig-cache
--global-cache-dir /home/dev/.cache/zig
--name myprogram --listen=-

For convenience, here's the final version of our build.zig:

const std = @import("std");

pub fn build(b: *std.Build) void {
    const exe = b.addExecutable(.{
        .name = "myprogram",
        .root_source_file = .{ .path = "src/main.zig" },
        .target = b.standardTargetOptions(.{}),
        .optimize = b.standardOptimizeOption(.{}),
    });

    b.installArtifact(exe);

    const install_docs = b.addInstallDirectory(.{
        .source_dir = exe.getEmittedDocs(),
        .install_dir = .prefix,
        .install_subdir = "docs",
    });

    const docs_step = b.step("docs", "Copy documentation artifacts to prefix path");
    docs_step.dependOn(&install_docs.step);
}

Thanks for reading!

1

skill issue, obviously

2

you know C, right?

zig

1664 Words

2023-11-16