Python pdoc: recursively generating argparse help

I recently discovered1 pdoc.

  • It’s really fast
  • It generates API documentation from Markdown docstrings in your code
  • It comes with a webserver that regenerates the content when your files change on disk, and reloads the web browser

It’s biggest limitation is that it doesn’t support prose documentation at all3. If your code is an API, this is very natural. And, if your code is an API with a command-line script or two, you can use this method to document both.

Before taking them up on this for I learned how to automatically generate help for argparse command-line programs.

Generating text help from ArgumentParser objects

In pdoc, it loads your modules into a Python runtime. This is a limitation, because it means any module-level code gets executed when generating the docs, but it’s also a powerful feature, as it means your __doc__ attributes are dynamic. You can write code that runs at documentation generation time.

Here’s an example file with an argparse.ArgumentParser and a dynamic __doc__ string containing that parser’s help text:

import argparse
import textwrap

def get_argparse_help_string(
    name: str, parser: argparse.ArgumentParser, wrap: int = 80, wrap_indent: int = 8
) -> str:
    """Generate a docstring for an argparse parser that shows the help for the parser and all subparsers, recursively.

    Based on an idea from <>

    * `name`: The name of the program
    * `parser`: The parser
    * `wrap`: The number of characters to wrap the help text to (0 to disable)
    * `wrap_indent`: The number of characters to indent the wrapped text

    def get_parser_help_recursive(
        parser: argparse.ArgumentParser, cmd: str = "", root: bool = True
        docstring = ""
        if not root:
            docstring += "\n" + "_" * 72 + "\n\n"
        docstring += f"> {cmd} --help\n"
        docstring += parser.format_help()

        for action in parser._actions:
            if isinstance(action, argparse._SubParsersAction):
                for subcmd, subparser in action.choices.items():
                    docstring += get_parser_help_recursive(
                        subparser, f"{cmd} {subcmd}", root=False
        return docstring

    docstring = get_parser_help_recursive(parser, name)

    if wrap > 0:
        wrapped = []
        # From the textwrap docs:
        # > If replace_whitespace is false,
        # > newlines may appear in the middle of a line and cause strange output.
        # > For this reason, text should be split into paragraphs
        # > (using str.splitlines() or similar) which are wrapped separately.
        for line in docstring.splitlines():
            if line:
                wrapped += textwrap.wrap(
                    subsequent_indent=" " * wrap_indent,
                wrapped += [""]
        return "\n".join(wrapped)
        return docstring

def _make_parser() -> argparse.ArgumentParser:
    """Return the ArgumentParser for this program."""
    parser = argparse.ArgumentParser(description="A program that does something.")
        "--verbose", action="store_true", help="Increase verbosity of output"
    subparsers = parser.add_subparsers()
    subparser = subparsers.add_parser("subcommand", help="A subcommand")
    subparser.add_argument("--subarg", help="An argument for the subcommand")
    return parser

__doc__ = f"""
The command-line interface to our program.

## Command line help

The program's command-line help is reproduced here:

{get_argparse_help_string("program-name", _make_parser())}

This was adapted from this original version, with a couple of enhancements:

  • It recursively loads subcommands, and places a line (at wrap width) between each subcommand.
  • It shows the --help command for each subcommand, exactly as if a user had typed it on the command-line and then pasted the result into the documentation.
  • It properly wraps lines to a useful width. You’ll likely display this documentation in a Markdown code block, which doesn’t wrap text. If you have very long help, this can be annoying. This function wraps help to 80 characters by default, papering over a textwrap.wrap() wart that returns strange results by default.

As mentioned, pdoc doesn’t support prose documentation. The most natural place for the documentation this function generates is probably the file containing the ArgumentParser.

pdoc can now automatically generate help!


See this example pdoc result. pdoc generates static files, so this is just the literal result of running pdoc ./ -o generated in this directory.

Here’s a frame showing just the get_argparse_help_string.html file, without the sidebar, for a quick idea of what it looks like.

  1. I had previously found pdoc3, which is what I used for trappedbot documentation. However, pdoc is more recently updated, and pdoc3 improperly classifies @dataclass fields as class rather than instance properties. I also like pdoc’s default output styles better than pdoc3’s. ↩︎

  2. You can read progfiguration’s extremely alpha documentation, now written in Sphinx, if you are curious. ↩︎

  3. As they say in their documentation:

    Scope: pdoc main use case is API documentation. If you have substantially more complex documentation needs, we recommend using Sphinx!

    I decided to take them up on this2 for progfiguration, which now uses Sphinx. I was hoping I could get by with API-only documentation, but decided that I really did need significant prose documentation as well. ↩︎


Comments are hosted on this site and powered by Remark42 (thanks!).

Webmentions are hosted on remote sites and syndicated via (thanks!).