Testing Package

The natcap.invest.testing package defines core testing routines and functionality.

Rationale

While the python standard library’s unittest package provides valuable resources for testing, GIS applications such as the various InVEST models output GIS data that require more in-depth testing to verify equality. For cases such as this, natcap.invest.testing provides a GISTest class that provides assertions for common data formats.

Writing Tests with natcap.invest.testing

The easiest way to take advantage of the functionality in natcap.invest.testing is to use the GISTest class whenever you write a TestCase class for your model. Doing so will grant you access to the GIS assertions provided by GISTest.

This example is relatively simplistic, since there will often be many more assertions you may need to make to be able to test your model effectively:

import natcap.invest.testing
import natcap.invest.example_model

class ExampleTest(natcap.invest.testing.GISTest):
    def test_some_model(self):
        example_args = {
            'workspace_dir': './workspace',
            'arg_1': 'foo',
            'arg_2': 'bar',
        }
        natcap.invest.example_model.execute(example_args)

        # example GISTest assertion
        self.assertRastersEqual('workspace/raster_1.tif',
            'regression_data/raster_1.tif')

natcap.invest.testing.GISTest

class natcap.invest.testing.GISTest(methodName='runTest')

Bases: unittest.case.TestCase

A test class with an emphasis on testing GIS outputs.

The GISTest class provides many functions for asserting the equality of various GIS files. This is particularly useful for GIS tool outputs, when we wish to assert the accuracy of very detailed outputs.

GISTest is a subclass of unittest.TestCase, so all members that exist in unittest.TestCase also exist here. Read the python documentation on unittest for more information about these test fixtures and their usage. The important thing to note is that GISTest merely provides more assertions for the more specialized testing and assertions that GIS outputs require.

Example usage of GISTest:

import natcap.invest.testing

class ModelTest(natcap.invest.testing.GISTest):
    def test_some_function(self):
        # perform your tests here.

Note that to take advantage of these additional assertions, you need only to create a subclass of GISTest in your test file to gain access to the GISTest assertions.

assertArchives(archive_1_uri, archive_2_uri)

Compare the contents of two archived workspaces against each other.

Takes two archived workspaces, each generated from build_regression_archives(), unzips them and compares the resulting workspaces against each other.

Parameters:
  • archive_1_uri (string) – a URI to a .tar.gz workspace archive
  • archive_2_uri (string) – a URI to a .tar.gz workspace archive
Raises:

AssertionError – Raised when the two workspaces are found to be different.
Returns:

Nothing.
assertCSVEqual(aUri, bUri)

Tests if csv files a and b are ‘almost equal’ to each other on a per cell basis. Numeric cells are asserted to be equal out to 7 decimal places. Other cell types are asserted to be equal.

Parameters:
  • aUri (string) – a URI to a csv file
  • bUri (string) – a URI to a csv file
Raises:

AssertionError – Raised when the two CSV files are found to be different.
Returns:

Nothing.
assertFiles(file_1_uri, file_2_uri)

Assert two files are equal.

If the extension of the provided file is recognized, the relevant filetype-specific function is called and a more detailed check of the file can be done. If the extension is not recognized, the MD5sums of the two files are compared instead.

Known extensions: .json, .tif, .shp, .csv, .txt., .html

Parameters:
  • file_1_uri (string) – a string URI to a file on disk.
  • file_2_uru (string) – a string URI to a file on disk.
Raises:

AssertionError – Raised when one of the input files does not exist, when the extensions of the input files differ, or if the two files are found to differ.
Returns:

Nothing.
assertJSON(json_1_uri, json_2_uri)

Assert two JSON files against each other.

The two JSON files provided will be opened, read, and their contents will be asserted to be equal. If the two are found to be different, the diff of the two files will be printed.

Parameters:
  • json_1_uri (string) – a uri to a JSON file.
  • json_2_uri (string) – a uri to a JSON file.
Raises:

AssertionError – Raised when the two JSON objects differ.
Returns:

Nothing.
assertMD5(uri, regression_hash)

Assert the MD5sum of a file against a regression MD5sum.

This method is a convenience method that uses natcap.invest.testing.get_hash() to determine the MD5sum of the file located at uri. It is functionally equivalent to calling:

self.assertEqual(get_hash(uri), '<some md5sum>')

Regression MD5sums can be calculated for you by using natcap.invest.testing.get_hash() or a system-level md5sum program.

Parameters:
  • uri (string) – a string URI to the file to be tested.
  • regression_hash (string) –
Raises:

AssertionError – Raised when the MD5sum of the file at uri differs from the provided regression md5sum hash.
Returns:

Nothing.
assertMatrixes(matrix_a, matrix_b, decimal=6)

Tests if the input numpy matrices are equal up to decimal places.

This is a convenience function that wraps up required functionality in numpy.testing.

Parameters:
  • matrix_a (numpy.ndarray) – a numpy matrix
  • matrix_b (numpy.ndarray) – a numpy matrix
  • decimal (int) – an integer of the desired precision.
Raises:

AssertionError – Raised when the two matrices are determined to be different.
Returns:

Nothing.
assertRastersEqual(a_uri, b_uri)

Tests if datasets a and b are ‘almost equal’ to each other on a per pixel basis

This assertion method asserts the equality of these raster characteristics:

System Message: ERROR/3 (/home/docs/checkouts/readthedocs.org/user_builds/invest/envs/3.3.1/local/lib/python2.7/site-packages/natcap/invest/testing/__init__.py:docstring of natcap.invest.testing.GISTest.assertRastersEqual, line 6)

Unexpected indentation.
  • Raster height and width
  • The number of layers in the raster
  • Each pixel value, out to a precision of 7 decimal places if the pixel value is a float.
Parameters:
  • a_uri (string) – a URI to a GDAL dataset
  • b_uri (string) – a URI to a GDAL dataset
Returns:

Nothing.
Raises:
  • IOError – Raised when one of the input files is not found on disk.
  • AssertionError – Raised when the two rasters are found to be not equal to each other.
assertTextEqual(text_1_uri, text_2_uri)

Assert that two text files are equal

This comparison is done line-by-line.

Parameters:
  • text_1_uri (string) – a python string uri to a text file. Considered the file to be tested.
  • text_2_uri (string) – a python string uri to a text file. Considered the regression file.
Raises:

AssertionError – Raised when a line differs in the two files.
Returns:

Nothing.
assertVectorsEqual(aUri, bUri)

Tests if vector datasources are equal to each other.

This assertion method asserts the equality of these vector characteristics:

System Message: ERROR/3 (/home/docs/checkouts/readthedocs.org/user_builds/invest/envs/3.3.1/local/lib/python2.7/site-packages/natcap/invest/testing/__init__.py:docstring of natcap.invest.testing.GISTest.assertVectorsEqual, line 5)

Unexpected indentation.
  • Number of layers in the vector
  • Number of features in each layer
  • Feature geometry type
  • Number of fields in each feature
  • Name of each field
  • Field values for each feature
Parameters:
  • aUri (string) – a URI to an OGR vector
  • bUri (string) – a URI to an OGR vector
Raises:
  • IOError – Raised if one of the input files is not found on disk.
  • AssertionError – Raised if the vectors are not found to be equal to one another.
Returns
Nothing.
assertWorkspace(archive_1_folder, archive_2_folder, glob_exclude='')

Check the contents of two folders against each other.

This method iterates through the contents of each workspace folder and verifies that all files exist in both folders. If this passes, then each file is compared against each other using GISTest.assertFiles().

If one of these workspaces includes files that are known to be different between model runs (such as logs, or other files that include timestamps), you may wish to specify a glob pattern matching those filenames and passing it to glob_exclude.

Parameters:
  • archive_1_folder (string) – a uri to a folder on disk
  • archive_2_folder (string) – a uri to a folder on disk
  • glob_exclude (string) – a string in glob format representing files to ignore
Raises:

AssertionError – Raised when the two folders are found to have different contents.
Returns:

Nothing.