natcap.invest.utils

InVEST specific code utils.

class natcap.invest.utils.ThreadFilter(thread_name)

Bases: Filter

Filters out log messages issued by the given thread.

Any log messages generated by a thread with the name matching the threadname provided to the constructor will be excluded.

filter(record)

Filter the given log record.

Parameters:

record (log record) – The log record to filter.

Returns:

True if the record should be included, false if not.

natcap.invest.utils.build_file_registry(base_file_path_list, file_suffix)

Combine file suffixes with key names, base filenames, and directories.

Parameters:

  • base_file_tuple_list (list) – a list of (dict, path) tuples where the dictionaries have a ‘file_key’: ‘basefilename’ pair, or ‘file_key’: list of ‘basefilename’s. ‘path’ indicates the file directory path to prepend to the basefile name.

  • file_suffix (string) – a string to append to every filename, can be empty string

Returns:

dictionary of ‘file_keys’ from the dictionaries in base_file_tuple_list mapping to full file paths with suffixes or lists of file paths with suffixes depending on the original type of the ‘basefilename’ pair.

Raises:

  • ValueError if there are duplicate file keys or duplicate file paths. –

  • ValueError if a path is not a string or a list of strings. –

natcap.invest.utils.capture_gdal_logging()

Context manager for logging GDAL errors with python logging.

GDAL error messages are logged via python’s logging system, at a severity that corresponds to a log level in logging. Error messages are logged with the osgeo.gdal logger.

Parameters:

None

Returns:

None

natcap.invest.utils.create_coordinate_transformer(base_ref, target_ref, osr_axis_mapping_strategy=0)

Create a spatial reference coordinate transformation function.

Parameters:

  • base_ref (osr spatial reference) – A defined spatial reference to transform FROM

  • target_ref (osr spatial reference) – A defined spatial reference to transform TO

  • osr_axis_mapping_strategy (int) – OSR axis mapping strategy for SpatialReference objects. Defaults to utils.DEFAULT_OSR_AXIS_MAPPING_STRATEGY. This parameter should not be changed unless you know what you are doing.

Returns:

An OSR Coordinate Transformation object

natcap.invest.utils.expand_path(path, base_path)

Check if a path is relative, and if so, expand it using the base path.

Parameters:

  • path (string) – path to check and expand if necessary

  • base_path (string) – path to expand the first path relative to

Returns:

path as an absolute path

natcap.invest.utils.has_utf8_bom(textfile_path)

Determine if the text file has a UTF-8 byte-order marker.

Parameters:

textfile_path (str) – The path to a file on disk.

Returns:

A bool indicating whether the textfile has a BOM. If True, a BOM is present.

natcap.invest.utils.log_to_file(logfile, exclude_threads=None, logging_level=0, log_fmt='%(asctime)s (%(name)s) %(module)s.%(funcName)s(%(lineno)d) %(levelname)s %(message)s', date_fmt=None)

Log all messages within this context to a file.

Parameters:

  • logfile (string) – The path to where the logfile will be written. If there is already a file at this location, it will be overwritten.

  • exclude_threads=None (list) – If None, logging from all threads will be included in the log. If a list, it must be a list of string thread names that should be excluded from logging in this file.

  • logging_level=logging.NOTSET (int) – The logging threshold. Log messages with a level less than this will be automatically excluded from the logfile. The default value (logging.NOTSET) will cause all logging to be captured.

  • log_fmt=LOG_FMT (string) – The logging format string to use. If not provided, utils.LOG_FMT will be used.

  • date_fmt (string) – The logging date format string to use. If not provided, ISO8601 format will be used.

Yields:

handler –

An instance of logging.FileHandler that

represents the file that is being written to.

Returns:

None

natcap.invest.utils.make_directories(directory_list)

Create directories in directory_list if they do not already exist.

natcap.invest.utils.make_suffix_string(args, suffix_key)

Make an InVEST appropriate suffix string.

Creates an InVEST appropriate suffix string given the args dictionary and suffix key. In general, prepends an ‘_’ when necessary and generates an empty string when necessary.

Parameters:

  • args (dict) – the classic InVEST model parameter dictionary that is passed to execute.

  • suffix_key (string) – the key used to index the base suffix.

Returns:

If suffix_key is not in args, or args[‘suffix_key’] is “”

return “”,

If args[‘suffix_key’] starts with ‘_’ return args[‘suffix_key’]

else return ‘_’+`args[‘suffix_key’]`

natcap.invest.utils.matches_format_string(test_string, format_string)

Assert that a given string matches a given format string.

This means that the given test string could be derived from the given format string by replacing replacement fields with any text. For example, the string ‘Value “foo” is invalid.’ matches the format string ‘Value “{value}” is invalid.’

Parameters:

  • test_string (str) – string to test.

  • format_string (str) – format string, which may contain curly-brace delimited replacement fields

Returns:

True if test_string matches format_string, False if not.

natcap.invest.utils.mean_pixel_size_and_area(pixel_size_tuple)

Convert to mean and raise Exception if they are not close.

Parameter:
pixel_size_tuple (tuple): a 2 tuple indicating the x/y size of a

pixel.

Returns:

tuple of (mean absolute average of pixel_size, area of pixel size)

Raises:

ValueError if the dimensions of pixel_size_tuple are not almost – square.

natcap.invest.utils.prepare_workspace(workspace, name, logging_level=0, exclude_threads=None)

Prepare the workspace.

natcap.invest.utils.read_csv_to_dataframe(path, **kwargs)

Return a dataframe representation of the CSV.

Wrapper around pandas.read_csv that performs some common data cleaning. Column names are lowercased and whitespace is stripped off. Empty rows and columns are dropped. Sets custom defaults for some kwargs passed to pandas.read_csv, which you can override with kwargs:

  • sep=None: lets the Python engine infer the separator

  • engine=’python’: The ‘python’ engine supports the sep=None option.

  • encoding=’utf-8-sig’: ‘utf-8-sig’ handles UTF-8 with or without BOM.

  • index_col=False: force pandas not to index by any column, useful in

    case of trailing separators

Parameters:

  • path (str) – path to a CSV file

  • **kwargs – additional kwargs will be passed to pandas.read_csv

Returns:

pandas.DataFrame with the contents of the given CSV

natcap.invest.utils.reclassify_raster(raster_path_band, value_map, target_raster_path, target_datatype, target_nodata, error_details)

A wrapper function for calling pygeoprocessing.reclassify_raster.

This wrapper function is helpful when added as a TaskGraph.task so a better error message can be provided to the users if a pygeoprocessing.ReclassificationMissingValuesError is raised.

Parameters:

  • raster_path_band (tuple) – a tuple including file path to a raster and the band index to operate over. ex: (path, band_index)

  • value_map (dictionary) – a dictionary of values of {source_value: dest_value, …} where source_value’s type is the same as the values in base_raster_path at band band_index. Must contain at least one value.

  • target_raster_path (string) – target raster output path; overwritten if it exists

  • target_datatype (gdal type) – the numerical type for the target raster

  • target_nodata (numerical type) – the nodata value for the target raster Must be the same type as target_datatype

  • error_details (dict) –

    a dictionary with key value pairs that provide more context for a raised pygeoprocessing.ReclassificationMissingValuesError. keys must be {‘raster_name’, ‘column_name’, ‘table_name’}. Values each key represent:

    ’raster_name’ - string for the raster name being reclassified ‘column_name’ - name of the table column that value_map dictionary keys came from. ‘table_name’ - table name that value_map came from.

Returns:

None

Raises:

  • ValueError –

    • if values_required is True and a pixel value from raster_path_band is not a key in value_map.

  • TypeError –

    • if there is a None or NA key in value_map.

natcap.invest.utils.sandbox_tempdir(suffix='', prefix='tmp', dir=None)

Create a temporary directory for this context and clean it up on exit.

Parameters are identical to those for tempfile.mkdtemp().

When the context manager exits, the created temporary directory is recursively removed.

Parameters:

  • suffix='' (string) – a suffix for the name of the directory.

  • prefix='tmp' (string) – the prefix to use for the directory name.

  • dir=None (string or None) – If a string, a directory that should be the parent directory of the new temporary directory. If None, tempfile will determine the appropriate tempdir to use as the parent folder.

Yields:

sandbox (string) – The path to the new folder on disk.

Returns:

None