natcap.invest.validation

Common validation utilities for InVEST models.

natcap.invest.validation.CHECK_ALL_KEYS = None

A flag to pass to the validation context manager indicating that all keys should be checked.

natcap.invest.validation.check_boolean(value, **kwargs)

Validate a boolean value.

If the value provided is not a python boolean, an error message is returned.

Parameters:

value – The value to evaluate.

Returns:

A string error message if an error was found. None otherwise.

natcap.invest.validation.check_csv(filepath, **kwargs)

Validate a table.

Parameters:

filepath (string) – The string filepath to the table.

Returns:

A string error message if an error was found. None otherwise.

natcap.invest.validation.check_directory(dirpath, must_exist=True, permissions='rx', **kwargs)

Validate a directory.

Parameters:

  • dirpath (string) – The directory path to validate.

  • must_exist=True (bool) – If True, the directory at dirpath must already exist on the filesystem.

  • permissions='rx' (string) – A string that includes the lowercase characters r, w and/or x indicating required permissions for this folder . See check_permissions for details.

Returns:

A string error message if an error was found. None otherwise.

natcap.invest.validation.check_file(filepath, permissions='r', **kwargs)

Validate a single file.

Parameters:

  • filepath (string) – The filepath to validate.

  • permissions='r' (string) – A string that includes the lowercase characters r, w and/or x indicating required permissions for this file. See check_permissions for details.

Returns:

A string error message if an error was found. None otherwise.

natcap.invest.validation.check_freestyle_string(value, regexp=None, **kwargs)

Validate an arbitrary string.

Parameters:

  • value – The value to check. Must be able to be cast to a string.

  • regexp=None (string) – a string interpreted as a regular expression.

Returns:

A string error message if an error was found. None otherwise.

natcap.invest.validation.check_headers(expected_headers, actual_headers, header_type='header')

Validate that expected headers are in a list of actual headers.

  • Each expected header should be found exactly once.

  • Actual headers may contain extra headers that are not expected.

  • Headers are converted to lowercase before matching.

Parameters:

  • expected_headers (list[str]) – A list of headers that are expected to exist in actual_headers.

  • actual_headers (list[str]) – A list of actual headers to validate against expected_headers.

  • header_type (str) – A string to use in the error message to refer to the header (typically one of ‘column’, ‘row’, ‘field’)

Returns:

None, if validation passes; or a string describing the problem, if a validation rule is broken.

natcap.invest.validation.check_integer(value, **kwargs)

Validate an integer.

Parameters:

value – A python value. This should be able to be cast to an int.

Returns:

A string error message if an error was found. None otherwise.

natcap.invest.validation.check_number(value, expression=None, **kwargs)

Validate numbers.

Parameters:

  • value – A python value. This should be able to be cast to a float.

  • expression=None (string) – A string expression to be evaluated with the intent of determining that the value is within a specific range. The expression must contain the string value, which will represent the user-provided value (after it has been cast to a float). Example expression: "(value >= 0) & (value <= 1)".

Returns:

A string error message if an error was found. None otherwise.

natcap.invest.validation.check_option_string(value, options, **kwargs)

Validate that a string is in a set of options.

Parameters:

  • value – The value to test. Will be cast to a string before comparing against the allowed options.

  • options (dict) – option spec to validate against.

Returns:

A string error message if value is not in options. None otherwise.

natcap.invest.validation.check_percent(value, **kwargs)

Validate a percent (a proportion expressed as a value from 0 to 100).

Parameters:

value – A python value. This should be able to be cast to a float.

Returns:

A string error message if an error was found. None otherwise.

natcap.invest.validation.check_permissions(path, permissions)

Validate permissions on a filesystem object.

This function uses os.access to determine permissions access.

Parameters:

  • path (string) – The path to examine for permissions.

  • permissions (string) – a string including the characters r, w and/or x (lowercase), indicating read, write, and execute permissions (respectively) that the filesystem object at path must have.

Returns:

A string error message if an error was found. None otherwise.

natcap.invest.validation.check_raster(filepath, projected=False, projection_units=None, **kwargs)

Validate a GDAL Raster on disk.

Parameters:

  • filepath (string) – The path to the raster on disk. The file must exist and be readable.

  • projected=False (bool) – Whether the spatial reference must be projected in linear units.

  • projection_units=None (pint.Units) – The required linear units of the projection. If None, the projection units will not be checked.

Returns:

A string error message if an error was found. None otherwise.

natcap.invest.validation.check_ratio(value, **kwargs)

Validate a ratio (a proportion expressed as a value from 0 to 1).

Parameters:

value – A python value. This should be able to be cast to a float.

Returns:

A string error message if an error was found. None otherwise.

natcap.invest.validation.check_spatial_overlap(spatial_filepaths_list, different_projections_ok=False)

Check that the given spatial files spatially overlap.

Parameters:

  • spatial_filepaths_list (list) – A list of files that can be opened with GDAL. Must be on the local filesystem.

  • different_projections_ok=False (bool) – Whether it’s OK for the input spatial files to have different projections. If True, all projections will be converted to WGS84 before overlap is checked.

Returns:

A string error message if an error is found. None otherwise.

natcap.invest.validation.check_vector(filepath, geometries, fields=None, projected=False, projection_units=None, **kwargs)

Validate a GDAL vector on disk.

Note

If the provided vector has multiple layers, only the first layer will be checked.

Parameters:

  • filepath (string) – The path to the vector on disk. The file must exist and be readable.

  • geometries (set) – Set of geometry type(s) that are allowed. Options are ‘POINT’, ‘LINESTRING’, ‘POLYGON’, ‘MULTIPOINT’, ‘MULTILINESTRING’, and ‘MULTIPOLYGON’.

  • fields=None (dict) – A dictionary spec of field names that the vector is expected to have. See the docstring of check_headers for details on validation rules.

  • projected=False (bool) – Whether the spatial reference must be projected in linear units. If None, the projection will not be checked.

  • projection_units=None (pint.Units) – The required linear units of the projection. If None, the projection units will not be checked.

Returns:

A string error message if an error was found. None otherwise.

natcap.invest.validation.get_headers_to_validate(spec)

Get header names to validate from a row/column/field spec dictionary.

This module only validates row/column/field names that are static and always required. If ‘required’ is anything besides True, or if the name contains brackets indicating it’s user-defined, it is not returned.

Parameters:

spec (dict) – a row/column/field spec dictionary that maps row/column/ field names to specs for them

Returns:

list of expected header names to validate against

natcap.invest.validation.get_invalid_keys(validation_warnings)

Get the invalid keys from a validation warnings list.

Parameters:

validation_warnings (list) – A list of two-tuples where the first item is an iterable of string args keys affected and the second item is a string error message.

Returns:

A set of the string args keys found across all of the first elements in the validation tuples.

natcap.invest.validation.get_sufficient_keys(args)

Determine which keys in args are sufficient.

A sufficient key is one that is:

  1. Present within args

  2. Does not have a value of '' or None.

Parameters:

args (dict) – An args dict of string keys to serializeable values.

Returns:

A set of keys from args that are sufficient.

natcap.invest.validation.get_validated_dataframe(csv_path, columns=None, rows=None, index_col=None, read_csv_kwargs={}, **kwargs)

Read a CSV into a dataframe that is guaranteed to match the spec.

natcap.invest.validation.invest_validator(validate_func)

Decorator to enforce characteristics of validation inputs and outputs.

Attributes of inputs and outputs that are enforced are:

  • args parameter to validate must be a dict

  • limit_to parameter to validate must be either None or a string (str or unicode) that exists in the args dict.

  • All keys in args must be strings

  • Decorated validate func must return a list of 2-tuples, where each 2-tuple conforms to these rules:

    • The first element of the 2-tuple is an iterable of strings. It is an error for the first element to be a string.

    • The second element of the 2-tuple is a string error message.

In addition, this validates the n_workers argument if it’s included.

Raises:

AssertionError when an invalid format is found. –

Example:

from natcap.invest import validation
@validation.invest_validator
def validate(args, limit_to=None):
    # do your validation here
natcap.invest.validation.load_fields_from_vector(filepath, layer_id=0)

Load fieldnames from a given vector.

Parameters:

  • filepath (string) – The path to a GDAL-compatible vector on disk.

  • layer_id=0 (string or int) – The identifier for the layer to use.

Returns:

A list of string fieldnames within the target layer.

natcap.invest.validation.timeout(func, *args, timeout=5, **kwargs)

Stop a function after a given amount of time.

Parameters:

  • func (function) – function to apply the timeout to

  • args – arguments to pass to the function

  • timeout (number) – how many seconds to allow the function to run. Defaults to 5.

Returns:

A string warning message if the thread completed in time and returned warnings, None otherwise.

Raises:

RuntimeWarning` if the thread does not complete in time –

natcap.invest.validation.validate(args, spec, spatial_overlap_opts=None)

Validate an args dict against a model spec.

Validates an arguments dictionary according to the rules laid out in spec. If spatial_overlap_opts is also provided, valid spatial inputs will be checked for spatial overlap.

Parameters:

  • args (dict) – The InVEST model args dict to validate.

  • spec (dict) – The InVEST model spec dict to validate against.

  • spatial_overlap_opts=None (dict) – A dict. If provided, the key

  • present ("spatial_keys" is required to be a list of keys that may be) – in the args dict and (if provided in args) will be checked for overlap with all other keys in this list. If the key "reference_key" is also present in this dict, the bounding boxes of each of the files represented by spatial_overlap_opts["spatial_keys"] will be transformed to the SRS of the dataset at this key.

Returns:

A list of tuples where the first element of the tuple is an iterable of keys affected by the error in question and the second element of the tuple is the string message of the error. If no validation errors were found, an empty list is returned.