natcap.invest.testing package¶
Submodules¶
natcap.invest.testing.autocomplete module¶
natcap.invest.testing.data_storage module¶
A module for InVEST test-related data storage.
-
natcap.invest.testing.data_storage.archive_uri(name=None)¶
-
natcap.invest.testing.data_storage.collect_parameters(parameters, archive_uri)¶ Collect an InVEST model’s arguments into a dictionary and archive all the input data.
parameters - a dictionary of arguments archive_uri - a URI to the target archive.
Returns nothing.
-
natcap.invest.testing.data_storage.extract_archive(workspace_dir, archive_uri)¶ Extract a .tar.gzipped file to the given workspace.
workspace_dir - the folder to which the archive should be extracted archive_uri - the uri to the target archive
Returns nothing.
-
natcap.invest.testing.data_storage.extract_parameters_archive(workspace_dir, archive_uri, input_folder=None)¶ Extract the target archive to the target workspace folder.
workspace_dir - a uri to a folder on disk. Must be an empty folder. archive_uri - a uri to an archive to be unzipped on disk. Archive must
be in .tar.gz format.- input_folder=None - either a URI to a folder on disk or None. If None,
- temporary folder will be created and then erased using the atexit register.
Returns a dictionary of the model’s parameters for this run.
-
natcap.invest.testing.data_storage.format_dictionary(input_dict, types_lookup={})¶ Recurse through the input dictionary and return a formatted dictionary.
As each element is encountered, the correct function to use is looked up in the types_lookup input. If a type is not found, we assume that the element should be returned verbatim.
input_dict - a dictionary to process types_lookup - a dictionary mapping types to functions. These functions
must take a single parameter of the type that is the key. These functions must return a formatted version of the input parameter.Returns a formatted dictionary.
-
natcap.invest.testing.data_storage.is_multi_file(filename)¶ Check if the filename given is a file with multiple parts to it, such as an ESRI shapefile or an ArcInfo Binary Grid.
-
natcap.invest.testing.data_storage.make_random_dir(workspace, seed_string, prefix, make_dir=True)¶
-
natcap.invest.testing.data_storage.make_raster_dir(workspace, seed_string, make_dir=True)¶
-
natcap.invest.testing.data_storage.make_vector_dir(workspace, seed_string, make_dir=True)¶
natcap.invest.testing.test_writing module¶
-
natcap.invest.testing.test_writing.add_test_to_class(file_uri, test_class_name, test_func_name, in_archive_uri, out_archive_uri, module)¶ Add a test function to an existing test file. The test added is a regression test using the natcap.invest.testing.regression archive decorator.
file_uri - URI to the test file to modify. test_class_name - string. The test class name to modify. If the test class
already exists, the test function will be added to the test class. If not, the new class will be created.- test_func_name - string. The name of the test function to write. If a
- test function by this name already exists in the target class, the function will not be written.
in_archive_uri - URI to the input archive. out_archive_uri - URI to the output archive. module - string module, whose execute function will be run in the test
(e.g. ‘natcap.invest.pollination.pollination’)WARNING: The input test file is overwritten with the new test file.
Returns nothing.
-
natcap.invest.testing.test_writing.class_has_test(test_file_uri, test_class_name, test_func_name)¶ Check that a python test file contains the given class and function.
test_file_uri - a URI to a python file containing test classes. test_class_name - a string, the class name we’re looking for. test_func_name - a string, the test function name we’re looking for.
This function should be located within the target test class.Returns True if the function is found within the class, False otherwise.
-
natcap.invest.testing.test_writing.file_has_class(test_file_uri, test_class_name)¶ Check that a python test file contains a class.
test_file_uri - a URI to a python file containing test classes. test_class_name - a string, the class name we’re looking for.
Returns True if the class is found, False otherwise.
Module contents¶
The natcap.invest.testing package defines core testing routines and functionality.
Rationale¶
While the python standard library’s unittest package provides valuable
resources for testing, GIS applications such as the various InVEST models
output GIS data that require more in-depth testing to verify equality. For
cases such as this, natcap.invest.testing provides a GISTest class that
provides assertions for common data formats.
Writing Tests with natcap.invest.testing¶
The easiest way to take advantage of the functionality in natcap.invest.testing
is to use the GISTest class whenever you write a TestCase class for your
model. Doing so will grant you access to the GIS assertions provided by
GISTest.
This example is relatively simplistic, since there will often be many more assertions you may need to make to be able to test your model effectively:
import natcap.invest.testing
import natcap.invest.example_model
class ExampleTest(natcap.invest.testing.GISTest):
def test_some_model(self):
example_args = {
'workspace_dir': './workspace',
'arg_1': 'foo',
'arg_2': 'bar',
}
natcap.invest.example_model.execute(example_args)
# example GISTest assertion
self.assertRastersEqual('workspace/raster_1.tif',
'regression_data/raster_1.tif')
-
class
natcap.invest.testing.GISTest(methodName='runTest')¶ Bases:
unittest.case.TestCaseA test class with an emphasis on testing GIS outputs.
The
GISTestclass provides many functions for asserting the equality of various GIS files. This is particularly useful for GIS tool outputs, when we wish to assert the accuracy of very detailed outputs.GISTestis a subclass ofunittest.TestCase, so all members that exist inunittest.TestCasealso exist here. Read the python documentation onunittestfor more information about these test fixtures and their usage. The important thing to note is thatGISTestmerely provides more assertions for the more specialized testing and assertions that GIS outputs require.Example usage of
GISTest:import natcap.invest.testing class ModelTest(natcap.invest.testing.GISTest): def test_some_function(self): # perform your tests here.Note that to take advantage of these additional assertions, you need only to create a subclass of
GISTestin your test file to gain access to theGISTestassertions.-
assertArchives(archive_1_uri, archive_2_uri)¶ Compare the contents of two archived workspaces against each other.
Takes two archived workspaces, each generated from
build_regression_archives(), unzips them and compares the resulting workspaces against each other.Parameters: - archive_1_uri (string) – a URI to a .tar.gz workspace archive
- archive_2_uri (string) – a URI to a .tar.gz workspace archive
Raises: AssertionError– Raised when the two workspaces are found to be different.Returns: Nothing.
-
assertCSVEqual(aUri, bUri)¶ Tests if csv files a and b are ‘almost equal’ to each other on a per cell basis. Numeric cells are asserted to be equal out to 7 decimal places. Other cell types are asserted to be equal.
Parameters: - aUri (string) – a URI to a csv file
- bUri (string) – a URI to a csv file
Raises: AssertionError– Raised when the two CSV files are found to be different.Returns: Nothing.
-
assertFiles(file_1_uri, file_2_uri)¶ Assert two files are equal.
If the extension of the provided file is recognized, the relevant filetype-specific function is called and a more detailed check of the file can be done. If the extension is not recognized, the MD5sums of the two files are compared instead.
Known extensions:
.json,.tif,.shp,.csv,.txt.,.htmlParameters: - file_1_uri (string) – a string URI to a file on disk.
- file_2_uru (string) – a string URI to a file on disk.
Raises: AssertionError– Raised when one of the input files does not exist, when the extensions of the input files differ, or if the two files are found to differ.Returns: Nothing.
-
assertJSON(json_1_uri, json_2_uri)¶ Assert two JSON files against each other.
The two JSON files provided will be opened, read, and their contents will be asserted to be equal. If the two are found to be different, the diff of the two files will be printed.
Parameters: - json_1_uri (string) – a uri to a JSON file.
- json_2_uri (string) – a uri to a JSON file.
Raises: AssertionError– Raised when the two JSON objects differ.Returns: Nothing.
-
assertMD5(uri, regression_hash)¶ Assert the MD5sum of a file against a regression MD5sum.
This method is a convenience method that uses
natcap.invest.testing.get_hash()to determine the MD5sum of the file located at uri. It is functionally equivalent to calling:self.assertEqual(get_hash(uri), '<some md5sum>')
Regression MD5sums can be calculated for you by using
natcap.invest.testing.get_hash()or a system-level md5sum program.Parameters: - uri (string) – a string URI to the file to be tested.
- regression_hash (string) –
Raises: AssertionError– Raised when the MD5sum of the file at uri differs from the provided regression md5sum hash.Returns: Nothing.
-
assertMatrixes(matrix_a, matrix_b, decimal=6)¶ Tests if the input numpy matrices are equal up to decimal places.
This is a convenience function that wraps up required functionality in
numpy.testing.Parameters: - matrix_a (numpy.ndarray) – a numpy matrix
- matrix_b (numpy.ndarray) – a numpy matrix
- decimal (int) – an integer of the desired precision.
Raises: AssertionError– Raised when the two matrices are determined to be different.Returns: Nothing.
-
assertRastersEqual(a_uri, b_uri)¶ Tests if datasets a and b are ‘almost equal’ to each other on a per pixel basis
This assertion method asserts the equality of these raster characteristics:
- Raster height and width
- The number of layers in the raster
- Each pixel value, out to a precision of 7 decimal places if the pixel value is a float.
Parameters: - a_uri (string) – a URI to a GDAL dataset
- b_uri (string) – a URI to a GDAL dataset
Returns: Nothing.
Raises: IOError– Raised when one of the input files is not found on disk.AssertionError– Raised when the two rasters are found to be not equal to each other.
-
assertTextEqual(text_1_uri, text_2_uri)¶ Assert that two text files are equal
This comparison is done line-by-line.
Parameters: - text_1_uri (string) – a python string uri to a text file. Considered the file to be tested.
- text_2_uri (string) – a python string uri to a text file. Considered the regression file.
Raises: AssertionError– Raised when a line differs in the two files.Returns: Nothing.
-
assertVectorsEqual(aUri, bUri)¶ Tests if vector datasources are equal to each other.
This assertion method asserts the equality of these vector characteristics:
- Number of layers in the vector
- Number of features in each layer
- Feature geometry type
- Number of fields in each feature
- Name of each field
- Field values for each feature
Parameters: - aUri (string) – a URI to an OGR vector
- bUri (string) – a URI to an OGR vector
Raises: IOError– Raised if one of the input files is not found on disk.AssertionError– Raised if the vectors are not found to be equal to one another.
- Returns
- Nothing.
-
assertWorkspace(archive_1_folder, archive_2_folder, glob_exclude='')¶ Check the contents of two folders against each other.
This method iterates through the contents of each workspace folder and verifies that all files exist in both folders. If this passes, then each file is compared against each other using
GISTest.assertFiles().If one of these workspaces includes files that are known to be different between model runs (such as logs, or other files that include timestamps), you may wish to specify a glob pattern matching those filenames and passing it to glob_exclude.
Parameters: - archive_1_folder (string) – a uri to a folder on disk
- archive_2_folder (string) – a uri to a folder on disk
- glob_exclude (string) – a string in glob format representing files to ignore
Raises: AssertionError– Raised when the two folders are found to have different contents.Returns: Nothing.
-
-
natcap.invest.testing.build_regression_archives(file_uri, input_archive_uri, output_archive_uri)¶ Build regression archives for a target model run.
With a properly formatted JSON configuration file at file_uri, all input files and parameters are collected and compressed into a single gzip. Then, the target model is executed and the output workspace is zipped up into another gzip. These could then be used for regression testing, such as with the
natcap.invest.testing.regressiondecorator.Example configuration file contents (serialized to JSON):
{ "model": "natcap.invest.pollination.pollination", "arguments": { # the full set of model arguments here } }
Example function usage:
import natcap.invest.testing file_uri = "/path/to/config.json" input_archive_uri = "/path/to/archived_inputs.tar.gz" output_archive_uri = "/path/to/archived_outputs.tar.gz" natcap.invest.testing.build_regression_archives(file_uri, input_archive_uri, output_archive_uri)
Parameters: - file_uri (string) – a URI to a json file on disk containing the
- configuration options. (above) –
- input_archive_uri (string) – the URI to where the gzip archive
- inputs should be saved once it is created. (of) –
- output_archive_uri (string) – the URI to where the gzip output
- of output should be saved once it is created. (archive) –
Returns: Nothing.
-
natcap.invest.testing.get_hash(uri)¶ Get the MD5 hash for a single file. The file is read in a memory-efficient fashion.
Parameters: uri (string) – a string uri to the file to be tested. Returns: An md5sum of the input file
-
natcap.invest.testing.regression(input_archive, workspace_archive)¶ Decorator to unzip input data, run the regression test and compare the outputs against the outputs on file.
Example usage with a test case:
import natcap.invest.testing @natcap.invest.testing.regression('/data/input.tar.gz', /data/output.tar.gz') def test_workspaces(self): model.execute(self.args)Parameters: - input_archive (string) – The path to a .tar.gz archive with the input data.
- workspace_archive (string) – The path to a .tar.gz archive with the workspace to assert.
Returns: Composed function with regression testing.
-
natcap.invest.testing.save_workspace(new_workspace)¶ Decorator to save a workspace to a new location.
If new_workspace already exists on disk, it will be recursively removed.
Example usage with a test case:
import natcap.invest.testing @natcap.invest.testing.save_workspace('/path/to/workspace') def test_workspaces(self): model.execute(self.args)
Note
- Target workspace folder must be saved to
self.workspace_dir This decorator is only designed to work with test functions from subclasses of
unittest.TestCasesuch asnatcap.invest.testing.GISTest.
- Target workspace folder must be saved to
- If
new_workspaceexists, it will be removed. So be careful where you save things.
- If
Parameters: new_workspace (string) – a URI to the where the workspace should be copied. Returns: A composed test case function which will execute and then save your workspace to the specified location.