InVEST® Plugins: Developer’s Guide

What is a plugin?

Conceptually, an InVEST plugin is an ecosystem services model. Like the core InVEST models, it takes in data of various formats (usually including some geospatial data), processes that data, and produces output files that contain the results. Unlike the core models, a plugin is not “official”, i.e., not reviewed or maintained by NatCap. Plugins may be developed, used, and distributed totally independently of the natcap/invest repo and the Natural Capital Alliance.

In a technical sense, an InVEST plugin is a python package that conforms to the natcap.invest plugin API. This makes it possible to run the plugin from the InVEST workbench and the invest command line tool. The plugin can execute any arbitrary code when it runs. Commonly the ecosystem services model logic will be implemented in the python package, but it could also invoke another software tool - for example, if your model is already implemented in another language, you could develop the plugin as a python wrapper for it.

Why make a plugin?

A plugin can be run in the InVEST workbench, which provides a graphical interface where a user can enter the model inputs, run the model, watch its progress, and access the results. All the necessary information to display the model in the workbench is pulled from the plugin python package - no frontend development needed. This is handy when resources are too limited to develop a separate GUI for a project. It is a major benefit for developers to be able to focus on their model and not worry about maintaining a desktop application or distributing it across multiple operating systems.

The data validation component of InVEST is also very useful for projects that don’t have enough resources to develop this independently. The plugin API requires that data inputs are rigorously specified. Before running a model, InVEST validates that the provided data meets all of the requirements, and provides helpful feedback if it does not. This prevents a lot of trouble with invalid data.

Even if resources were unlimited, we think there is value in having a shared interface for ecosystem services models. Seeing different ecosystem service models, or different versions of the same model, side-by-side in the workbench facilitates running them together and comparing them.

The plugin API is a useful framework in which to think of developing a model. This framework is helpful when tackling the task of turning a “model” (which may exist in the form of mathematical equations, scripts, or other software) into a well-documented, reusable, distributable software tool. Implementing the plugin API requires attention to many details that are easily overlooked when writing a basic script. Going through the process of developing a model into a plugin will help to catch bugs and identify assumptions that may exist in your math or your code.

Example plugins

The NatCap team maintains a small number of InVEST plugins. The following are some examples you may find useful as you develop your own plugin:

InVEST Demo Plugin (referenced in more detail in later sections of this guide)
InVEST GCM Downscaling
Sediment Delivery Ratio with USLE C Raster Input

In addition, you may wish to consult the source code of the core InVEST models for examples of model code.

How to develop a plugin

Note

This guide is written for python developers. If you are unfamiliar with python packaging, the Python Packaging User Guide is a helpful resource.

At its most basic, a plugin is a python package. Begin by creating a directory with a simple python package structure:

invest-demo-plugin/
|- pyproject.toml
|- src/
   |- invest_demo_plugin/
      |- __init__.py
      |- foo.py

The model code is in src/invest_demo_plugin/foo.py. pyproject.toml is a standard configuration file that defines the package. A full-fledged plugin will contain many other optional files as well, like a readme, license, test suite, and sample data, but these are not required.

Writing the `pyproject.toml`

The pyproject.toml contains standard information used to build your python package, as well as custom configuration for other software that interacts with the package. InVEST looks for metadata about your package in the pyproject.toml. Configuration specific to InVEST is defined in the [tool.natcap.invest] block. See the Python Packaging User Guide for more general information on pyproject.toml.

[build-system]
requires = ["setuptools>=64", "wheel", "setuptools-scm>=8"]
build-backend = "setuptools.build_meta"

[project]
name = "invest-demo-plugin"
# Allow setuptools_scm to derive the version dynamically from the git metadata
# Using setuptools_scm is not required, but we highly recommend it
dynamic = ["version"]  

# All direct python dependencies should be listed here. If a package is
# available from conda-forge, it can also be listed under the conda_dependencies.
dependencies = [
  "pygeoprocessing",
  "taskgraph",
  "natcap.invest>=3.18.0"
]

[tool.setuptools_scm]
# Use default configuration. A warning is issued if this section is absent.

[tool.natcap.invest]
# This is the plugin API version that this plugin uses.
# invest uses this to determine whether it can run this plugin.
# "v1" is the current and only version at this time
api_version = "v1"

# This is the importable package name of the plugin. It should match the name of the 
# package directory. It MUST begin with "invest". It is included here so that 
# natcap.invest knows how to import the plugin. 
# This is distinct from the project.name defined above (these names are often the same, 
# but don't have to be). Unlike the project.name, this name should not contain hyphens 
# or other special characters because it will be used in an import statement.
package_name = "invest_demo_plugin"

# List of conda dependencies to install into the plugin's conda environment. 
# Usually this includes python itself and any non-python dependencies
# like the GDAL binaries. Packages listed here will be installed into the
# plugin environment prior to the other dependencies, so you may wish to list
# packages here that are difficult to install with pip.
#
# Only the conda-forge channel is supported. These dependencies MUST be available on 
# conda-forge. We have disabled the default channel due to Anaconda's licensing.
#
# Python is installed into the plugin environment by default, but you may 
# include it here if you wish to specify a particular version
conda_dependencies = [
  "python==3.13",  
  "gdal<3.11",  # missing libgdal.so issue happened with gdal 3.11
  "geotiff>=1.7.3",  # earlier versions don't support space in CONDA_PREFIX
  "pygeoprocessing",  # install from conda-forge rather than building wheel with pip
  "natcap.invest>=3.18.0"  # install from conda-forge rather than building wheel with pip
]

[tool.setuptools.package-data]
# Use this to include any non-python files in your package distribution,
# such as data required by your package, or a non-python executable
foo = ["data/foo"]

Note

When InVEST installs a plugin, it uses micromamba to create an isolated environment in which to run the plugin. This environment contains the dependencies that the plugin needs, and prevents dependency conflicts with other plugins.

If you are familiar with conda, note that micromamba is very similar. It can install the same packages and has a similar API.

A plugin will likely depend on some other python packages like numpy or pandas. It may also depend on some other software that is not available as a pure python package, like GDAL. This is why we use micromamba instead of a python-specific package manager like pip - micromamba can manage both python and non-python dependencies (and python itself).

Writing the main module

The plugin python package must have the attributes MODEL_SPEC, execute, and validate:

`MODEL_SPEC`

An instance of natcap.invest.spec.ModelSpec(). This object stores key information about the model, its inputs, and its outputs. See the API Reference for the specifics on instantiating this object. Note that the model_id should be globally unique, i.e. different from the model_id of any core InVEST model or any other plugin. To help ensure uniqueness among other plugins, it is a good idea for the model_id to match the python package name.

Here is an example MODEL_SPEC taken from the demo plugin. It describes a model that takes in three inputs: a raster file, an integer multiplication factor, and a workspace directory in which to produce the results. The model produces a raster file output which is the result of multiplying the input raster pixelwise by the multiplication factor.

from natcap.invest import spec

MODEL_SPEC = spec.ModelSpec(
    model_id="demo",
    model_title="Demo Plugin",
    userguide='',
    input_field_order=[
        ['workspace_dir'],
        ['raster_path', 'factor']],
    inputs=[
        spec.WORKSPACE,
        spec.N_WORKERS,
        spec.SingleBandRasterInput(
            id="raster_path",
            name="Input Raster",
            data_type=float,
            units=None
        ),
        spec.IntegerInput(
            id="factor",
            name="Multiplication Factor"
        )
    ],
    outputs=[
        spec.SingleBandRasterOutput(
            id="result",
            path="result.tif",
            about="Raster multiplied by factor",
            data_type=float,
            units=None
        )
    ]
)

`execute`

This function executes the model. When a user runs the model, this function is invoked with the inputs that the user provided. When this function returns, the model run is complete.

Arguments: args (dictionary). Maps input ids (matching the id of each Input in MODEL_SPEC.inputs) to their values to run the model on.

Returns: Dictionary mapping output IDs to the absolute file paths where those outputs were created. All invest models use natcap.invest.file_registry.FileRegistry to manage and track file paths throughout the model. Returning the registry attribute of the FileRegistry satisfies this requirement.

Here is an example implementation of execute corresponding to the example MODEL_SPEC above. It multiplies a raster pixelwise by an integer value, and writes out the result to a new raster file:

MODEL_SPEC = ... # see above

def execute(args):
    args, file_registry, task_graph = MODEL_SPEC.setup(args)
    task_graph.add_task(
        func=multiply_op,
        kwargs={
            'raster_path': args['raster_path'],
            'factor': int(args['factor']),
            'target_path': file_registry['result']
        },
        target_path_list=[file_registry['result']],
        task_name='multiply raster by factor')
    task_graph.close()
    task_graph.join()
    return file_registry.registry

Note

All core InVEST models use taskgraph to organize the steps of execution. This is optional, but taskgraph has several benefits including avoided recomputation, distributing tasks over multiple CPUs, and logically organizing the model as a workflow of tasks that process data. See the InVEST source code for many examples of using taskgraph.

`validate`

This function validates the model inputs. Its purpose is to identify problems with the user’s data before running the model, and give helpful feedback so the problems can be fixed. When a user enters data into the workbench UI, validate is called and its output is used to provide instant feedback (for instance, highlighting problematic inputs in red). The “Run” button will be disabled until all inputs validate successfully and validate returns [].

Arguments: args (dictionary). Maps input ids (matching the id of each Input in MODEL_SPEC.inputs) to their values to run the model on.

Returns: A list of tuples where the first element of the tuple is an iterable of keys affected by the error in question and the second element of the tuple is the string message of the error. If no validation errors were found, an empty list is returned.

The following implementation of validate will suffice for most models:

from natcap.invest import validation

@validation.invest_validator
def validate(args):
    return validation.validate(args, MODEL_SPEC)

validation.validate performs pre-defined validation for each input type based on its properties. See the validate method of each Input class to see exactly what checks are performed. If you need to validate properties of the input data that are not covered by the pre-defined checks, you may add on to this basic validate function.

Specifying model inputs and outputs

Model inputs are specified in the inputs attribute of the MODEL_SPEC. Many different types of model inputs are supported, including numbers, CSVs, raster and vector files, etc. Each input in inputs is an instance of a subclass of natcap.invest.spec.Input() that represents the data type. Choose the most appropriate Input type available in spec. You may also subclass from natcap.invest.spec.Input() if you wish to create a custom type. Each Input type should define an id (which uniquely identifies the input within the model), a name (user-facing input name, displayed in the workbench), and an about (user-facing description of the input), in addition to the properties specific to the input type.

User-provided values for all input types are ultimately passed to the execute function as strings or numbers. For instance, all file-based types will accept a path string.

Model outputs are specified in the outputs attribute of the MODEL_SPEC. All InVEST model outputs are files - there are no plain number or string outputs. Choose the most appropriate Output type available in spec. You may also subclass from natcap.invest.spec.Output() if you wish to create a custom type.

Note

Common inputs: All core InVEST models include the inputs workspace_dir (a target directory where all model results are written), results_suffix (a suffix appended to all model results, which may be used to differentiate model runs in the same workspace), and n_workers (passed to taskgraph to configure the number of CPUs used). Standard specs for these inputs are provided in natcap.invest.spec().

The n_workers input is normally hidden from display in core InVEST models (using the hidden attribute). n_workers is configured as a workbench-level setting, rather than a model-level input. The workbench adds the n_workers value from its settings to the args passed to the model. If you do not use taskgraph, you are free to ignore this value.

Specifying units

Some input and output types have a units attribute representing the units of measurement of the data. We use pint to manage units. In pint, all unit objects must derive from the same UnitRegistry in order to be used together. Therefore, you should reference natcap.invest’s shared unit registry, natcap.invest.spec.u(). Example: spec.u.meter ** 3 (cubic meters).

Nested data

Certain input and output types contain multiple types of data (such as columns in a CSV, or fields in a vector).

CSVInput() and CSVOutput(): The columns attribute is an iterable of Inputs or Outputs that represent the data stored in each column of the CSV. The id of each Input/Output must match the column header.
VectorInput() and VectorOutput(): The fields attribute is an iterable of Inputs or Outputs that represent the data stored in each field of the Vector. The id of each Input/Output must match the field name.
DirectoryInput(): The contents attribute is an iterable of Inputs that represent the file contents of the directory. The id of each Input must match the file name.

Example:

CSVInput(
    id="biophysical_table_path",
    name="biophysical table",
    about="Table of crop coefficients for each LULC class.",
    columns=[
        IntegerInput(
            id="lulc_code",
            about="Land use/land cover code"
        ),
        NumberInput(
            id="kc_factor",
            about="Crop coefficient for each land use/land cover class",
            units=None
        )
    ],
    index_col="lulc_code"
)

`init.py`

If you are following the project layout described above, and demonstrated in the demo plugin repo, MODEL_SPEC, execute, and validate will be properties of the foo submodule. You must make them available at the level of the invest_plugin package by importing them into __init__.py. This is demonstrated in the demo plugin repo.

Writing a reporter module

Some InVEST models include a “reporter” that creates visual summaries of the model results. This is an optional part of a model or plugin, but natcap.invest includes some utilities and templates to make it convenient to develop a report for your plugin. If you follow this example, a report html file will be generated whenever the plugin is run from the Workbench or using the invest command-line-interface.

A file structure for a plugin with a reporter could look like this:

invest-demo-plugin/
|- pyproject.toml
|- src/
   |- invest_demo_plugin/
      |- __init__.py
      |- foo.py
      |- reporter.py
      |- templates/
         |- report.html

reporter.py will be responsible for generating the report. Reference it in your plugin’s ModelSpec:

MODEL_SPEC = spec.ModelSpec(
    model_id="demo",
    model_title="Demo Plugin",
    reporter='invest_demo_plugin.reporter',

invest_demo_plugin.reporter must contain a function named report with this signature:

def report(file_registry, args_dict, model_spec, target_html_filepath):
    """Generate an HTML summary of model results.

    Args:
        file_registry (dict): The ``natcap.invest.FileRegistry.registry``
            that was returned by the model's ``execute`` method.
        args_dict (dict): The arguments that were passed to the model's
            ``execute`` method.
        model_spec (natcap.invest.spec.ModelSpec): the model's ``MODEL_SPEC``.
        target_html_filepath (str): path to an HTML file to be generated by
            this function.

    Returns:
        ``None``
    """

In order to generate the html document, the reporter can use jinja2 templates. Create a jinja2 html template for the report, such as “src/invest_demo_plugin/templates/report.html” and then render it from the report function:

from natcap.invest.reports import jinja_env
...
env = jinja2.Environment(
        loader=jinja2.PackageLoader('invest_demo_plugin', 'templates'),
        autoescape=jinja2.select_autoescape(),
        undefined=jinja2.StrictUndefined)
    template = env.get_template('report.html')

    with open(target_html_filepath, 'w', encoding='utf-8') as target_file:
        target_file.write(template.render(
            ...
            invest_reports_env=jinja_env
        ))

In order to use templates and macros provided with natcap.invest.reports pass natcap.invest.reports.jinja_env to your report template, and then your template can use those resources like this:

{% extends invest_reports_env.get_template('base.html') %}

{% block content %}

  {{ super() }}

  {% from invest_reports_env.get_template('caption.html') import caption %}
  {% from invest_reports_env.get_template('raster-plot-img.html') import raster_plot_img %}
  {% from invest_reports_env.get_template('content-grid.html') import content_grid %}

  {{ accordion_section(
    section_header,
    content_grid([
      (raster_plot_img(img_src, section_header), 100),
      (caption(img_caption, definition_list=True), 100)
    ])
  )}}

{% endblock content %}

How InVEST interacts with plugins

At the python level, when natcap.invest is imported, it searches for other installed python packages that look like plugins. Plugins are identified by having a package name beginning with invest and having the expected attributes described above. All identified plugin packages are recorded in natcap.invest.models and become available to natcap.invest just like the core models.

At the workbench level, plugins are installed through the “Manage Plugins” window. The user provides the plugin source code (as either a git URL or a local path), and the workbench follows this process to install it:

If installing from a git URL, download the pyproject.toml.
Parse metadata from the pyproject.toml.
Create a micromamba environment for the plugin to run in. The environment contains python and git by default, plus any dependencies specified in pyproject.toml in tool.natcap.invest.conda_dependencies.
Use pip to install the plugin into the environment created in step 3.
Import the plugin and access metadata from the MODEL_SPEC.
Save the plugin information to the workbench settings store.

The settings store is where the workbench tracks what plugins are installed. When a user launches a plugin, a new natcap.invest server is launched from the plugin’s environment. This server runs on a different port than the natcap.invest server that serves core models. All model-specific requests related to running the plugin are sent to that port.

Note

The workbench uniquely identifies plugins by a hash of a combination of the model_id from the MODEL_SPEC and the package.version from the pyproject.toml. Therefore it is possible to have multiple versions of the same plugin simultaneously installed. At the python level, only one plugin with the same package name can be used at a time.

Testing Your Plugin

Automated tests help ensure your plugin continues to work as expected as you develop it. See the InVEST repo for examples of model tests.

InVEST® Plugins: Developer’s Guide

What is a plugin?

Why make a plugin?

Example plugins

How to develop a plugin

Writing the pyproject.toml

Writing the main module

MODEL_SPEC

execute

validate

Specifying model inputs and outputs

Specifying units

Nested data

__init__.py

Writing a reporter module

How InVEST interacts with plugins

Testing Your Plugin

Writing the `pyproject.toml`

`MODEL_SPEC`

`execute`

`validate`

`init.py`