InVEST: Integrated Valuation of Ecosystem Services and Tradeoffs¶
Release: 3.7.0.post0+d20190509
InVEST (Integrated Valuation of Ecosystem Services and Tradeoffs) is a family of tools for quantifying the values of natural capital in clear, credible, and practical ways. In promising a return (of societal benefits) on investments in nature, the scientific community needs to deliver knowledge and tools to quantify and forecast this return. InVEST enables decision-makers to quantify the importance of natural capital, to assess the tradeoffs associated with alternative choices, and to integrate conservation and human development.
Older versions of InVEST ran as script tools in the ArcGIS ArcToolBox environment, but have almost all been ported over to a purely open-source python environment.
InVEST is licensed under a permissive, modified BSD license.
- For more information, see:
- InVEST on bitbucket
- The latest InVEST User’s Guide
- The Natural Capital Project website.
Getting Started¶
Installing InVEST¶
Note
The natcap.invest
python package is currently only supported in Python
2.7. Other versions of python may be supported at a later date.
Warning
Python 2.7.11 or later is required to be able to use the InVEST Recreation model on Windows.
Binary Dependencies¶
InVEST itself depends only on python packages, but many of these package dependencies depend on low-level libraries or have complex build processes. In recent history, some of these packages (notably, numpy and scipy) have started to release precompiled binary packages of their own, removing the need to install these packages through a system package manager. Others, however, remain easiest to install through a package manager.
Linux¶
Linux users have it easy, as almost every package required to use
natcap.invest is available in the package repositories. The provided
commands will install only the libararies and binaries that are needed, allowing
pip
to install the rest.
Ubuntu & Debian¶
Attention
The package versions in the debian:stable repositories often lag far behind the latest releases. It may be necessary to install a later version of a libarary from a different package repository, or else build the library from source.
$ sudo apt-get install python-dev python-setuptools python-gdal python-rtree python-shapely python-matplotlib python-qt4
Fedora¶
$ sudo yum install python-devel python-setuptools gdal-python python-rtree python-shapely python-matplotlib PyQt4
Mac OS X¶
The easiest way to install binary packages on Mac OS X is through a package manager such as Homebrew:
$ brew install gdal spatialindex pyqt matplotlib
The GDAL, PyQt and matplotlib packages include their respective python packages.
The others will allow their corresponding python packages to be compiled
against these binaries via pip
.
Windows¶
While many packages are available for Windows on the Python Package Index, some may need to be fetched from a different source. Many are available from Christogh Gohlke’s unofficial build page: http://www.lfd.uci.edu/~gohlke/pythonlibs/
PyQt4 installers can also be downloaded from the Riverbank Computing website.
Python Dependencies¶
Dependencies for natcap.invest
are listed in requirements.txt
:
#
# Any lines with "# pip-only" at the end will be processed by
# scripts/convert-requirements-to-conda-yml.py as though it can only be found
# on pip.
GDAL>=2.0,<3.0
Pyro4==4.41 # pip-only
pandas>=0.22.0
numpy>=1.11.0,!=1.16.0
Rtree>=0.8.2
scipy>=0.16.1
Shapely>=1.6.4
six
pygeoprocessing>=1.6.1
taskgraph[niced_processes]>=0.8.2
psutil<5.6.0
chardet>=3.0.4
matplotlib
xlrd>=1.2.0
Optional Qt User Interface¶
InVEST’s user interface is built with PyQt. Because of the hefty binary
requirement of Qt and the relative difficulty of installing PyQt, these
dependencies will not be installed with the standard
pip install natcap.invest
. Several of these dependencies are available
as extras, however, and can be installed via pip:
$ pip install natcap.invest[ui]
These extras do not include a distribution of PyQt, so you will need to install PyQt in an appropriate way on your system. PyQt4 is not currently available from the Python Package Index, but other sources and package managers allow for straightforward installation on Windows, Mac OS X, and Linux.
The InVEST user interface uses a wrapper layer to support both PyQt4 and PyQt5, one of which must be installed on your system for the UI to be able to run. If both are installed, PyQt5 is preferred, but you can force the UI to use PyQt4 by defining an environment variable before launching the UI:
$ QT_API=pyqt4 invest pollination
Installing from Source¶
Note
Windows users will find best compilation results by using the MSVC compiler, which can be downloaded from the Microsoft website. See the python wiki page on compilation under Windows for more information.
Assuming you have a C/C++ compiler installed and configured for your system, and dependencies installed, the easiest way to install InVEST as a python package is:
$ pip install natcap.invest
If you are working within virtual environments, there is a documented issue
with namespaces
in setuptools that may cause problems when importing packages within the
natcap
namespace. The current workaround is to use these extra pip flags:
$ pip install natcap.invest --egg --no-binary :all:
Installing the latest development version¶
Pre-built binaries for Windows¶
Pre-built installers and wheels of development versions of natcap.invest
for 32-bit Windows python installations are available from
http://data.naturalcapitalproject.org/invest-releases/#dev, along with other
distributions of InVEST. Once downloaded, wheels can be installed locally via
pip:
> pip install .\natcap.invest-3.3.0.post89+nfc4a8d4de776-cp27-none-win32.whl
Installing from our source tree¶
The latest development version of InVEST can be installed from our Mercurial source tree:
$ pip install hg+https://bitbucket.org/natcap/invest@develop
The InVEST CLI¶
Installing¶
The invest
cli application is installed with the natcap.invest
python
package. See Installing InVEST
Usage¶
To run an InVEST model from the command-line, use the invest
cli single
entry point:
$ invest --help
usage: invest [-h] [--version] [-v | --debug] [--list] [-l] [-d [DATASTACK]]
[-w [WORKSPACE]] [-q] [-y] [-n]
[model]
Integrated Valuation of Ecosystem Services and Tradeoffs. InVEST (Integrated
Valuation of Ecosystem Services and Tradeoffs) is a family of tools for
quantifying the values of natural capital in clear, credible, and practical
ways. In promising a return (of societal benefits) on investments in nature,
the scientific community needs to deliver knowledge and tools to quantify and
forecast this return. InVEST enables decision-makers to quantify the
importance of natural capital, to assess the tradeoffs associated with
alternative choices, and to integrate conservation and human development.
Older versions of InVEST ran as script tools in the ArcGIS ArcToolBox
environment, but have almost all been ported over to a purely open-source
python environment.
positional arguments:
model The model/tool to run. Use --list to show available
models/tools. Identifiable model prefixes may also be
used. Alternatively,specify "launcher" to reveal a
model launcher window.
optional arguments:
-h, --help show this help message and exit
--version show program's version number and exit
-v, --verbose Increase verbosity. Affects how much is printed to the
console and (if running in headless mode) how much is
written to the logfile.
--debug Enable debug logging. Alias for -vvvvv
--list List available models
-l, --headless Attempt to run InVEST without its GUI.
-d [DATASTACK], --datastack [DATASTACK]
Run the specified model with this datastack
-w [WORKSPACE], --workspace [WORKSPACE]
The workspace in which outputs will be saved
gui options:
These options are ignored if running in headless mode
-q, --quickrun Run the target model without validating and quit with
a nonzero exit status if an exception is encountered
headless options:
-y, --overwrite Overwrite the workspace without prompting for
confirmation
-n, --no-validate Do not validate inputs before running the model.
To list the available models:
$ invest --list
To launch a model:
$ invest <modelname>
Changelog¶
Unreleased Changes¶
3.7.0 (2019-05-09)¶
- Refactoring Coastal Vulnerability (CV) model. CV now uses TaskGraph and Pygeoprocessing >=1.6.1. The model is now largely vector-based instead of raster-based. Fewer input datasets are required for the same functionality. Runtime in sycnhronous mode is similar to previous versions, but runtime can be reduced with multiprocessing. CV also supports avoided recomputation for successive runs in the same workspace, even if a different file suffix is used. Output vector files are in CSV and geopackage formats.
- Model User Interface ‘Report an Issue’ link points to our new community.naturalcapitalproject.org
- Correcting an issue with the Coastal Blue Carbon preprocessor where using misaligned landcover rasters would cause an exception to be raised.
- Correcting an issue with RouteDEM where runs of the tool with Flow Direction
enabled would cause the tool to crash if
n_workers > 0
. - Correcting an issue with Habitat Quality’s error checking where nodata values in landcover rasters were not being taken into account.
- Valuation is now an optional component of the InVEST Scenic Quality model.
- Fixing a bug in the percentiles algorithm used by Scenic Quality that would result in incorrect visual quality outputs.
- Carbon Model and Crop Production models no longer crash if user-input rasters do not have a nodata value defined. In this case these models treat all pixel values as valid data.
- Adding bitbucket pipelines and AppVeyor build configurations.
- Refactoring Recreation Model client to use taskgraph and the latest pygeoprocessing. Avoided re-computation from taskgraph means that successive model runs with the same AOI and gridding option can re-use PUD results and avoid server communication entirely. Successive runs with the same predictor data will re-use intermediate geoprocessing results. Multiprocessing offered by taskgraph means server-side PUD calculations and client-side predictor data processing can happen in parallel. Some output filenames have changed.
- Upgrading to SDR to use new PyGeoprocessing multiflow routing, DEM pit filling, contiguous stream extraction, and TaskGraph integration. This also includes a new TaskGraph feature that avoids recomputation by copying results from previous runs so long as the expected result would be identical. To use this feature, users must execute successive runs of SDR in the same workspace but use a different file suffix. This is useful when users need to do a parameter study or run scenarios with otherwise minor changes to inputs.
- Refactoring Habitat Risk Assessment (HRA) Model to use TaskGraph >= 0.8.2 and Pygeoprocessing >= 1.6.1. The HRA Proprocessor is removed and its previous functionality was simplified and merged into the HRA model itself. The model will no longer generate HTML plots and tables.
- Adding a software update notification button, dialog, and a link to the download page on the User Interface when a new InVEST version is available.
- Migrating the subversion sample and test data repositories to Git LFS repositories on BitBucket. Update the repository URL and fetch commands on Makefile accordingly.
- Fixing a bug in Habitat Quality UI where the absence of the required half_saturation_constant variable did not raise an exception.
- Adding encoding=’utf-8-sig’ to pandas.read_csv() to support utils.build_lookup_from_csv() to read CSV files encoded with UTF-8 BOM (byte-order mark) properly.
3.6.0 (2019-01-30)¶
- Correcting an issue with the InVEST Carbon Storage and Sequestration model where filepaths containing non-ASCII characters would cause the model’s report generation to crash. The output report is now a UTF-8 document.
- Refactoring RouteDEM to use taskgraph and the latest pygeoprocessing
(
>=1.5.0
). RouteDEM now fills hydrological sinks and users have the option to use either of the D8 or Multiple Flow Direction (MFD) routing algorithms. - Adding a new input to the InVEST Settings window to allow users to customize
the value that should be used for the
n_workers
parameter in taskgraph-enabled models. This change involves removing the “Number of Parallel Workers” input from the model inputs pane for some models in favor of this new location. The default value for this setting is-1
, indicating synchronous (non-threaded, non-multiprocessing) execution of tasks. - Removing Scenario Generator: Rule-based model.
- Fixing a bug in Hydropower model where watershed aggregations would be incorrect if a watershed is partially covering nodata raster values. Nodata values are now ignored in zonal statistics. Numerical results change very slightly in the case where a watershed only includes a few nodata pixels.
- Adding TaskGraph functionality to GLOBIO model.
- Adding some TaskGraph functionality to Scenario Generator: Proximity.
- Fixing an issue with the InVEST Fisheries model that would prevent the model from batch-processing a directory of population tables. The model will now process these files as expected.
- Reimplementing Crop Production models using taskgraph.
- Fixing an issue with Crop Production Regression’s result_table.csv where the ‘production_modeled’ and ‘<nutrient>_modeled’ values calculated for each crop were done so using the same crop raster (e.g. wheat, soybean, and barley values were all based on soybean data).
- Hydropower subwatershed results now include all the same metrics as the watershed results, with the exception of economic valuation metrics.
- Reimplementing the Hydropower model using taskgraph.
- Reimplementing the Carbon model using taskgraph.
- Fixing an issue with Coastal Blue Carbon validation to allow column names to ignore case.
- Updating core carbon forest edge regression data coefficient to drop impossible negative coefficients.
- Fixing an issue with the Scenario Generator: Proximity model that would raise an exception if no AOI were passed in even though the AOI is optional.
- Removing Overlap Analysis and Overlap Analysis: Management Zones.
- Removing Habitat Suitability.
- Added comprehensive error checking to hydropower model to test for the VERY common errors of missing biophysical, demand, and valuation coefficients in their respective tables.
- Fixing an issue with Hydropower Water Yield (“Annual Water Yield”) where valuation would never be triggered when running the model through the User Interface. And a related issue where the model would crash if a valuation table was provided but a demand table was not. The UI no longer validates that config.
- Fixing an issue with how logging is captured when a model is run through the InVEST User Interface. Now, logging from any thread started by the executor thread will be written to the log file, which we expect to aid in debugging.
- Fixing an issue with Scenic Quality where viewpoints outside of the AOI were not being properly excluded. Viewpoints are now excluded correctly.
- The crop production model has been refactored to drop the “aggregate ID” concept when summarizing results across an aggregate polygon. The model now uses the polygon FIDs internally and externally when producing the result summary table.
- Correcting the rating instructions in the criteria rating instructions on how the data quality (DQ) and weight should be rated in the HRA Preprocessor. A DQ score of 1 should represent better data quality whereas the score of 3 is worse data quality. A weight score of 1 is more important, whereas that of 3 is less important.
- Fixing a case where a zero discount rate and rate of change in the carbon model would cause a divide by zero error.
3.5.0 (2018-08-14)¶
- Bumped pygeoprocessing requirement to
pygeoprocessing>=1.2.3
. - Bumped taskgraph requirement to
taskgraph>=0.6.1
. - Reimplemented the InVEST Scenic Quality model. This new version removes the ‘population’ and ‘overlap’ postprocessing steps, updates the available valuation functions and greatly improves the runtime and memory-efficiency of the model. See the InVEST User’s Guide chapter for more information.
- Updated Recreation server’s database to include metadata from photos taken from 2005-2017 (previous range was 2005-2014). The new range is reflected in the UI.
- Fixed an issue with the InVEST binary build where binaries on Windows would crash with an error saying Python27.dll could not be loaded.
- Fixed an issue in the Rule-Based Scenario Generator UI where vector column names from override and constraint layers were not being loaded. This bug caused the field ‘UNKNOWN’ to be passed to the model, causing an error.
- Fixed an issue with the InVEST UI (all models), where attempting to drag-and-drop a directory onto a model input would cause the application to crash.
- Coastal Vulnerability UI now specifies a number of reasonable defaults for some numeric inputs.
- Fixed an issue with the Fisheries UI where alpha and beta parameter inputs were incorrectly disabled for the Ricker recruitment function.
- InVEST now uses a Makefile to automate the build processes. GNU Make is
required to use the Makefile. See
README.rst
for instructions on building InVEST. This replaces the oldpavement.py
build entrypoint, which has been removed. - Fixed an issue with the InVEST UI (all models), where attempting to drag-and-drop a directory onto a model input would cause the application to crash.
- Fixed an issue with Forest Carbon Edge Effect where the UI layer was always causing the model to run with only the aboveground carbon pool
- Added functionality to the InVEST UI so that
Dropdown
inputs can now map dropdown values to different output values. - Fixed an issue in the Crop Production Percentile model that would treat the optional AOI vector field as a filename and crash on a run if it were empty.
- Fixing an issue in the Pollination Model that would cause occasional crashes due to a missing dependent task; it had previously been patched by setting taskgraph to operate in single thread mode. This restores multithreading in the pollination model.
- Fixed an issue in the water yield / hydropower model that would skip calculation of water demand tables when “water scarcity” was enabled.
- Fixed an issue in the model data of the crop production model where some crops were using incorrect climate bin rasters. Since the error was in the data and not the code, users will need to download the most recent version of InVEST’s crop model data during the installation step to get the fix.
3.4.4 (2018-03-26)¶
- InVEST now requires GDAL 2.0.0 and has been tested up to GDAL 2.2.3. Any API users of InVEST will need to use GDAL version >= 2.0. When upgrading GDAL we noticed slight numerical differences in our test suite in both numerical raster differences, geometry transforms, and occasionally a single pixel difference when using gdal.RasterizeLayer. Each of these differences in the InVEST test suite is within a reasonable numerical tolerance and we have updated our regression test suite appropriately. Users comparing runs between previous versions of InVEST may also notice reasonable numerical differences between runs.
- Added a UI keyboard shortcut for showing documentation. On Mac OSX, this will be Command-?. On Windows, GNOME and KDE, this will be F1.
- Patching an issue in NDR that was using the nitrogen subsurface retention efficiency for both nitrogen and phosphorous.
- Fixed an issue with the Seasonal Water Yield model that incorrectly required a rain events table when the climate zone mode was in use.
- Fixed a broken link to local and online user documentation from the Seasonal Water Yield model from the model’s user interface.
3.4.3 (2018-03-26)¶
- Fixed a critical issue in the carbon model UI that would incorrectly state the user needed a “REDD Priority Raster” when none was required.
- Fixed an issue in annual water yield model that required subwatersheds even though it is an optional field.
- Fixed an issue in wind energy UI that was incorrectly validating most of the inputs.
3.4.2 (2017-12-15)¶
- Fixed a cross-platform issue with the UI where logfiles could not be dropped onto UI windows.
- Model arguments loaded from logfiles are now cast to their correct literal value. This addresses an issue where some models containing boolean inputs could not have their parameters loaded from logfiles.
- Fixed an issue where the Pollination Model’s UI required a farm polygon. It should have been optional and now it is.
- Fixing an issue with the documentation and forums links on the InVEST model windows. The links now correctly link to the documentation page or forums as needed.
- Fixing an issue with the
FileSystemRunDialog
where pressing the ‘X’ button in the corner of the window would close the window, but not reset its state. The window’s state is now reset whenever the window is closed (and the window cannot be closed when the model is running)
3.4.1 (2017-12-11)¶
- In the Coastal Blue Carbon model, the
interest_rate
parameter has been renamed toinflation_rate
. - Fixed issues with sample parameter sets for InVEST Habitat Quality, Habitat Risk Assessment, Coastal Blue Carbon, and Coastal Blue Carbon Preprocessors. All sample parameter sets now have the correct paths to the model’s input files, and correctly note the name of the model that they apply to.
- Added better error checking to the SDR model for missing ws_id and invalid ws_id values such as None or some non-integer value. Also added tests for the SDR validation module.
3.4.0 (2017-12-03)¶
- Fixed an issue with most InVEST models where the suffix was not being reflected in the output filenames. This was due to a bug in the InVEST UI, where the suffix args key was assumed to be
'suffix'
. Instances ofInVESTModel
now accept a keyword argument to defined the suffix args key. - Fixed an issue/bug in Seasonal Water Yield that would occur when a user provided a datastack that had nodata values overlapping with valid DEM locations. Previously this would generate an NaN for various biophysical values at that pixel and cascade it downslope. Now any question of nodata on a valid DEM pixel is treated as “0”. This will make serious visual artifacts on the output, but should help users pinpoint the source of bad data rather than crash.
- Refactored all but routing components of SDR to use PyGeoprocessing 0.5.0 and laid a consistent raster floating point type of ‘float32’. This will cause numerically insignificant differences between older versions of SDR and this one. But differences are well within the tolerance of the overall error of the model and expected error rate of data. Advantages are smaller disk footprint per run, cleaner and more maintainable design, and a slight performance increase.
- Bug fixed in SDR that would align the output raster stack to match with the landcover pixel stack even though the rest of the rasters are scaled and clipped to the DEM.
- When loading parameters from a datastack, parameter set or logfile, the UI will check that the model that created the file being loaded matches the name of the model that is currently running. If there is a mismatch, a dialog is presented for the user to confirm or cancel the loading of parameters. Logfiles from IUI (which do not have clearly-recorded modelname or InVEST version information) can still have their arguments parsed, but the resulting model name and InVEST version will be set to
"UNKNOWN"
. - Data Stack files (
*.invest.json
,*.invest.tar.gz
) can now be dragged and dropped on an InVEST model window, which will prompt the UI to load that parameter set. - Spatial inputs to Coastal Blue Carbon are now aligned as part of the model. This resolves a longstanding issue with the model where inputs would need to perfectly overlap (even down to pixel indices), or else the model would yield strange results.
- The InVEST UI now contains a submenu for opening a recently-opened datastack. This submenu is automatically populated with the 10 most recently-opened datastacks for the current model.
- Removed vendored
natcap.invest.dbfpy
subpackage. - Removed deprecated
natcap.invest.fileio
module. - Removed
natcap.invest.iui
UI subpackage in favor of a new UI framework found atnatcap.invest.ui
. This new UI features a greatly improved API, good test coverage, support for Qt4 and Qt5, and includes updates to all InVEST models to support validation of model arguments from a python script, independent of the UI. - Updated core model of seasonal water yield to allow for negative L_avail.
- Updated RouteDEM to allow for file suffixes, finer control over what DEM routing algorithms to run, and removal of the multiple stepped stream threshold classification.
- Redesign/refactor of pollination model. Long term bugs in the model are resolved, managed pollinators added, and many simplifications to the end user’s experience. The updated user’s guide chapter is available here: http://data.naturalcapitalproject.org/nightly-build/invest-users-guide/html/croppollination.html
- Scenario Generator - Rule Based now has an optional input to define a seed. This input is used to seed the random shuffling of parcels that have equal priorities.
- InVEST on mac is now distributed as a single application bundle, allowing InVEST to run as expected on mac OSX Sierra. Individual models are selected and launched from a new launcher window.
- The InVEST CLI now has a GUI model launcher:
$ invest launcher
- Updated the Coastal Blue Carbon model to improve handling of blank lines in input CSV tables and improve memory efficiency of the current implementation.
- Improved the readability of a cryptic error message in Coastal Vulnerability that is normally raised when the depth threshold is too high or the exposure proportion is too low to detect any shoreline segments.
- Adding InVEST HTML documentation to the Mac disk image distribution.
- Upgrading dependency of PyGeoprocessing to 0.3.3. This fixes a memory leak associated with any model that aggregates rasters over complicated overlapping polygons.
- Adding sample data to Blue Carbon model that were missing.
- Deprecating the InVEST Marine Water Quality model. This also removes InVEST’s dependancy on the pyamg package which has been removed from REQUIREMENTS.TXT.
- Deprecating the ArcGIS-based Coastal Protection model and ArcGIS-based data-preprocessing scripts. The toolbox and scripts may still be found at https://bitbucket.org/natcap/invest.arcgis.
- Fixing an issue in the carbon edge effect model that caused output values in the shapefile to be rounded to the nearest integer.
- Fixing issue in SDR model that would occasionally cause users to see errors about field widths in the output shapefile generation.
- Updated the erodibility sample raster that ships with InVEST for the SDR model. The old version was in US units, in this version we convert to SI units as the model requires, and clipped the raster to the extents of the other stack to save disk space.
3.3.3 (2017-02-06)¶
- Fixed an issue in the UI where the carbon model wouldn’t accept negative numbers in the price increase of carbon.
- RouteDEM no longer produces a “tiled_dem.tif” file since that functionality is being deprecated in PyGeoprocessing.
- Fixing an issue in SDR where the optional drainage layer would not be used in most of the SDR biophysical calculations.
- Refactoring so water yield pixels with Kc and et0 equal to be 0 now yields a 0.0 value of water yield on that pixel rather than nodata.
- Light optimization refactor of wind energy model that improves runtimes in some cases by a factor of 2-3.
- Performance optimizations to HRA that improve runtimes by approximately 30%.
- Fixed a broken UI link to Seasonal Water Yield’s user’s guide.
- Fixed an issue with DelineateIT that caused ArcGIS users to see both the watershed and inverse watershed polygons when viewing the output of the tool.
- Upgrading dependency to PyGeoprocessing 0.3.2.
- Fixed an issue with SDR that caused the LS factor to be an order of magnitue too high in areas where the slope was greater than 9%. In our sample case this caused sediment export estimates to be about 6% too high, but in cases where analyses are run over steep slopes the error would have been greater.
paver check
now warns if thePYTHONHOME
environment variable is set.- API docs now correctly reflect installation steps needed for python development headers on linux.
- Fixed a side effect in the InVEST user interface that would cause
tempfile.tempdir
to be set and then not be reset after a model run is finished. - The InVEST user interface will now record GDAL/OGR log messages in the log messages window and in the logfile written to the workspace.
- Updated branding and usability of the InVEST installer for Windows, and the Mac Disk Image (.dmg).
3.3.2 (2016-10-17)¶
- Partial test coverage for HRA model.
- Full test coverage for Overlap Analysis model.
- Full test coverage for Finfish Aquaculture.
- Full test coverage for DelineateIT.
- Full test coverage for RouteDEM.
- Fixed an issue in Habitat Quality where an error in the sample table or malformed threat raster names would display a confusing message to the user.
- Full test coverage for scenario generator proximity model.
- Patching an issue in seasonal water yield that causes an int overflow error if the user provides a floating point landcover map and the nodata value is outside of the range of an int64.
- Full test coverage for the fisheries model.
- Patched an issue that would cause the Seasonal Water Edge model to crash when the curve number was 100.
- Patching a critical issue with forest carbon edge that would give incorrect results for edge distance effects.
- Patching a minor issue with forest carbon edge that would cause the model to crash if only one interpolation point were selected.
- Full test coverage for pollination model.
- Removed “farms aggregation” functionality from the InVEST pollination model.
- Full test coverage for the marine water quality model.
- Full test coverage for GLOBIO model.
- Full test coverage for carbon forest edge model.
- Upgraded SciPy dependancy to 0.16.1.
- Patched bug in NDR that would cause a phosphorus density to be reported per pixel rather than total amount of phosporous in a pixel.
- Corrected an issue with the uses of buffers in the euclidean risk function of Habitat Risk Assessment. (issue #3564)
- Complete code coverage tests for Habitat Quality model.
- Corrected an issue with the
Fisheries_Inputs.csv
sample table used by Overlap Analysis. (issue #3548) - Major modifications to Terrestrial Carbon model to include removing the harvested wood product pool, uncertainty analysis, and updated efficient raster calculations for performance.
- Fixed an issue in GLOBIO that would cause model runs to crash if the AOI marked as optional was not present.
- Removed the deprecated and incomplete Nearshore Wave and Erosion model (
natcap.invest.nearshore_wave_and_erosion
). - Removed the deprecated Timber model (
natcap.invest.timber
). - Fixed an issue where seasonal water yield would raise a divide by zero error if a watershed polygon didn’t cover a valid data region. Now sets aggregation quantity to zero and reports a warning in the log.
natcap.invest.utils.build_file_registry
now raises aValueError
if a path is not a string or list of strings.- Fixed issues in NDR that would indicate invalid values were being processed during runtimes by skipping the invalid calculations in the first place rather than calculating them and discarding after the fact.
- Complete code coverage tests for NDR model.
- Minor (~10% speedup) performance improvements to NDR.
- Added functionality to recreation model so that the monthly_table.csv file now receives a file suffix if one is provided by the user.
- Fixed an issue in SDR where the m exponent was calculated incorrectly in many situations resulting in an error of about 1% in total export.
- Fixed an issue in SDR that reported runtime overflow errors during normal processing even though the model completed without other errors.
3.3.1 (2016-06-13)¶
- Refactored API documentation for readability, organization by relevant topics, and to allow docs to build on invest.readthedocs.io,
- Installation of
natcap.invest
now requiresnatcap.versioner
. If this is not available on the system at runtime, setuptools will make it available at runtime. - InVEST Windows installer now includes HISTORY.rst as the changelog instead of the old
InVEST_Updates_<version>
files. - Habitat suitability model is generalized and released as an API only accessible model. It can be found at
natcap.invest.habitat_suitability.execute
. This model replaces the oyster habitat suitability model. - The refactor of this model requires an upgrade to
numpy >= 1.11.0
.
- The refactor of this model requires an upgrade to
- Habitat suitability model is generalized and released as an API only accessible model. It can be found at
- Fixed a crash in the InVEST CLI where calling
invest
without a parameter would raise an exception on linux-based systems. (Issue #3528) - Patched an issue in Seasonal Water Yield model where a nodata value in the landcover map that was equal to
MAX_INT
would cause an overflow error/crash. - InVEST NSIS installer will now optionally install the Microsoft Visual C++ 2008 redistributable on Windows 7 or earlier. This addresses a known issue on Windows 7 systems when importing GDAL binaries (Issue #3515). Users opting to install this redistributable agree to abide by the terms and conditions therein.
- Removed the deprecated subpackage
natcap.invest.optimization
. - Updated the InVEST license to legally define the Natural Capital Project.
- Corrected an issue in Coastal Vulnerability where an output shapefile was being recreated for each row, and where field values were not being stored correctly.
- Updated Scenario Generator model to add basic testing, file registry support, PEP8 and PEP257 compliance, and to fix several bugs.
- Updated Crop Production model to add a simplified UI, faster runtime, and more testing.
3.3.0 (2016-03-14)¶
- Refactored Wind Energy model to use a CSV input for wind data instead of a Binary file.
- Redesigned InVEST recreation model for a single input streamlined interface, advanced analytics, and refactored outputs. While the model is still based on “photo user days” old model runs are not backward compatable with the new model or interface. See the Recreation Model user’s guide chapter for details.
- The refactor of this model requires an upgrade to
GDAL >=1.11.0 <2.0
andnumpy >= 1.10.2
.
- The refactor of this model requires an upgrade to
- Removed nutrient retention (water purification) model from InVEST suite and replaced it with the nutrient delivery ratio (NDR) model. NDR has been available in development relseases, but has now officially been added to the set of Windows Start Menu models and the “under development” tag in its users guide has been removed. See the InVEST user’s guide for details between the differences and advantages of NDR over the old nutrient model.
- Modified NDR by adding a required “Runoff Proxy” raster to the inputs. This allows the model to vary the relative intensity of nutrient runoff based on varying precipitation variability.
- Fixed a bug in the Area Change rule of the Rule-Based Scenario Generator, where units were being converted incorrectly. (Issue #3472) Thanks to Fosco Vesely for this fix.
- InVEST Seasonal Water Yield model released.
- InVEST Forest Carbon Edge Effect model released.
- InVEST Scenario Generator: Proximity Based model released and renamed the previous “Scenario Generator” to “Scenario Generator: Rule Based”.
- Implemented a blockwise exponential decay kernel generation function, which is now used in the Pollination and Habitat Quality models.
- GLOBIO now uses an intensification parameter and not a map to average all agriculture across the GLOBIO 8 and 9 classes.
- GLOBIO outputs modified so core outputs are in workspace and intermediate outputs are in a subdirectory called ‘intermediate_outputs’.
- Fixed a crash with the NDR model that could occur if the DEM and landcover maps were different resolutions.
- Refactored all the InVEST model user interfaces so that Workspace defaults to the user’s home “Documents” directory.
- Fixed an HRA bug where stessors with a buffer of zero were being buffered by 1 pixel
- HRA enhancement which creates a common raster to burn all input shapefiles onto, ensuring consistent alignment.
- Fixed an issue in SDR model where a landcover map that was smaller than the DEM would create extraneous “0” valued cells.
- New HRA feature which allows for “NA” values to be entered into the “Ratings” column for a habitat / stressor pair in the Criteria Ratings CSV. If ALL ratings are set to NA, the habitat / stressor will be treated as having no interaction. This means in the model, that there will be no overlap between the two sources. All rows parameters with an NA rating will not be used in calculating results.
- Refactored Coastal Blue Carbon model for greater speed, maintainability and clearer documentation.
- Habitat Quality bug fix when given land cover rasters with different pixel sizes than threat rasters. Model would use the wrong pixel distance for the convolution kernel.
- Light refactor of Timber model. Now using CSV input attribute file instead of DBF file.
- Fixed clipping bug in Wave Energy model that was not properly clipping polygons correctly. Found when using global data.
- Made the following changes / updates to the coastal vulnerability model:
- Fixed a bug in the model where the geomorphology ranks were not always being used correctly.
- Removed the HTML summary results output and replaced with a link to a dashboard that helps visualize and interpret CV results.
- Added a point shapefile output: ‘outputs/coastal_exposure.shp’ that is a shapefile representation of the corresponding CSV table.
- The model UI now requires the ‘Relief’ input. No longer optional.
- CSV outputs and Shapefile outputs based on rasters now have x, y coorinates of the center of the pixel instead of top left of the pixel.
- Turning setuptools’ zip_safe to False for consistency across the Natcap Namespace.
- GLOBIO no longer requires user to specify a keyfield in the AOI.
- New feature to GLOBIO to summarize MSA by AOI.
- New feature to GLOBIO to use a user defined MSA parameter table to do the MSA thresholds for infrastructure, connectivity, and landuse type
- Documentation to the GLOBIO code base including the large docstring for ‘execute’.
3.2.0 (2015-05-31)¶
InVEST 3.2.0 is a major release with the addition of several experimental models and tools as well as an upgrade to the PyGeoprocessing core:
- Upgrade to PyGeoprocessing v0.3.0a1 for miscelaneous performance improvements to InVEST’s core geoprocessing routines.
- An alpha unstable build of the InVEST crop production model is released with partial documentation and sample data.
- A beta build of the InVEST fisheries model is released with documentation and sample data.
- An alpha unstable build of the nutrient delivery ratio (NDR) model is available directly under InVEST’s instalation directory at
invest-x86/invest_ndr.exe
; eventually this model will replace InVEST’s current “Nutrient” model. It is currently undocumented and unsupported but inputs are similar to that of InVEST’s SDR model. - An alpha unstable build of InVEST’s implementation of GLOBIO is available directly under InVEST’s instalation directory at
invest-x86/invest_globio.exe
. It is currently undocumented but sample data are provided. - DelinateIT, a watershed delination tool based on PyGeoprocessing’s d-infinity flow algorithm is released as a standalone tool in the InVEST repository with documentation and sample data.
- Miscelaneous performance patches and bug fixes.
3.1.3 (2015-04-23)¶
InVEST 3.1.3 is a hotfix release patching a memory blocking issue resolved in PyGeoprocessing version 0.2.1. Users might have experienced slow runtimes on SDR or other routed models.
3.1.2 (2015-04-15)¶
InVEST 3.1.2 is a minor release patching issues mostly related to the freshwater routing models and signed GDAL Byte datasets.
- Patching an issue where some projections were not regognized and InVEST reported an UnprojectedError.
- Updates to logging that make it easier to capture logging messages when scripting InVEST.
- Shortened water yield user interface height so it doesn’t waste whitespace.
- Update PyGeoprocessing dependency to version 0.2.0.
- Fixed an InVEST wide issue related to bugs stemming from the use of signed byte raster inputs that resulted in nonsensical outputs or KeyErrors.
- Minor performance updates to carbon model.
- Fixed an issue where DEMS with 32 bit ints and INT_MAX as the nodata value nodata value incorrectly treated the nodata value in the raster as a very large DEM value ultimately resulting in rasters that did not drain correctly and empty flow accumulation rasters.
- Fixed an issue where some reservoirs whose edges were clipped to the edge of the watershed created large plateaus with no drain except off the edge of the defined raster. Added a second pass in the plateau drainage algorithm to test for these cases and drains them to an adjacent nodata area if they occur.
- Fixed an issue in the Fisheries model where the Results Suffix input was invariably initializing to an empty string.
- Fixed an issue in the Blue Carbon model that prevented the report from being generated in the outputs file.
3.1.1 (2015-03-13)¶
InVEST 3.1.1 is a major performance and memory bug patch to the InVEST toolsuite. We recommend all users upgrade to this version.
- Fixed an issue surrounding reports of SDR or Nutrient model outputs of zero values, nodata holes, excessive runtimes, or out of memory errors. Some of those problems happened to be related to interesting DEMs that would break the flat drainage algorithm we have inside RouteDEM that adjusted the heights of those regions to drain away from higher edges and toward lower edges, and then pass the height adjusted dem to the InVEST model to do all its model specific calculations. Unfortunately this solution was not amenable to some degenerate DEM cases and we have now adjusted the algorithm to treat each plateau in the DEM as its own separate region that is processed independently from the other regions. This decreases memory use so we never effectively run out of memory at a minor hit to overall runtime. We also now adjust the flow direction directly instead of adjust the dem itself. This saves us from having to modify the DEM and potentially get it into a state where a drained plateau would be higher than its original pixel neighbors that used to drain into it.
There are side effects that result in sometimes large changes to un calibrated runs of SDR or nutrient. These are related to slightly different flow directions across the landscape and a bug fix on the distance to stream calculation.
- InVEST geoprocessing now uses the PyGeoprocessing package (v0.1.4) rather than the built in functionality that used to be in InVEST. This will not affect end users of InVEST but may be of interest to users who script InVEST calls who want a standalone Python processing package for raster stack math and hydrological routing. The project is hosted at https://bitbucket.org/richpsharp/pygeoprocessing.
- Fixed an marine water quality issue where users could input AOIs that were unprojected, but output pixel sizes were specified in meters. Really the output pixel size should be in the units of the polygon and are now specified as such. Additionally an exception is raised if the pixel size is too small to generate a numerical solution that is no longer a deep scipy error.
- Added a suffix parameter to the timber and marine water quality models that append a user defined string to the output files; consistent with most of the other InVEST models.
- Fixed a user interface issue where sometimes the InVEST model run would not open a windows explorer to the user’s workspace. Instead it would open to C:User[..]My Documents. This would often happen if there were spaces in the the workspace name or “/” characters in the path.
- Fixed an error across all InVEST models where a specific combination of rasters of different cell sizes and alignments and unsigned data types could create errors in internal interpolation of the raster stacks. Often these would appear as ‘KeyError: 0’ across a variety of contexts. Usually the ‘0’ was an erroneous value introduced by a faulty interpolation scheme.
- Fixed a MemoryError that could occur in the pollination and habitat quality models when the the base landcover map was large and the biophysical properties table allowed the effect to be on the order of that map. Now can use any raster or range values with only a minor hit to runtime performance.
- Fixed a serious bug in the plateau resolution algorithm that occurred on DEMs with large plateau areas greater than 10x10 in size. The underlying 32 bit floating point value used to record small height offsets did not have a large enough precision to differentiate between some offsets thus creating an undefined flow direction and holes in the flow accumulation algorithm.
- Minor performance improvements in the routing core, in some cases decreasing runtimes by 30%.
- Fixed a minor issue in DEM resolution that occurred when a perfect plateau was encountered. Rather that offset the height so the plateau would drain, it kept the plateau at the original height. This occurred because the uphill offset was nonexistent so the algorithm assumed no plateau resolution was needed. Perfect plateaus now drain correctly. In practice this kind of DEM was encountered in areas with large bodies of water where the remote sensing algorithm would classify the center of a lake 1 meter higher than the rest of the lake.
- Fixed a serious routing issue where divergent flow directions were not getting accumulated 50% of the time. Related to a division speed optimization that fell back on C-style modulus which differs from Python.
- InVEST SDR model thresholded slopes in terms of radians, not percent thus clipping the slope tightly between 0.001 and 1%. The model now only has a lower threshold of 0.00005% for the IC_0 factor, and no other thresholds. We believe this was an artifact left over from an earlier design of the model.
- Fixed a potential memory inefficiency in Wave Energy Model when computing the percentile rasters. Implemented a new memory efficient percentile algorithm and updated the outputs to reflect the new open source framework of the model. Now outputting csv files that describe the ranges and meaning of the percentile raster outputs.
- Fixed a bug in Habitat Quality where the future output “quality_out_f.tif” was not reflecting the habitat value given in the sensitivity table for the specified landcover types.
3.1.0 (2014-11-19)¶
InVEST 3.1.0 (http://www.naturalcapitalproject.org/download.html) is a major software and science milestone that includes an overhauled sedimentation model, long awaited fixes to exponential decay routines in habitat quality and pollination, and a massive update to the underlying hydrological routing routines. The updated sediment model, called SDR (sediment delivery ratio), is part of our continuing effort to improve the science and capabilities of the InVEST tool suite. The SDR model inputs are backwards comparable with the InVEST 3.0.1 sediment model with two additional global calibration parameters and removed the need for the retention efficiency parameter in the biophysical table; most users can run SDR directly with the data they have prepared for previous versions. The biophysical differences between the models are described in a section within the SDR user’s guide and represent a superior representation of the hydrological connectivity of the watershed, biophysical parameters that are independent of cell size, and a more accurate representation of sediment retention on the landscape. Other InVEST improvements to include standard bug fixes, performance improvements, and usability features which in part are described below:
- InVEST Sediment Model has been replaced with the InVEST Sediment Delivery Ratio model. See the SDR user’s guide chapter for the difference between the two.
- Fixed an issue in the pollination model where the exponential decay function decreased too quickly.
- Fixed an issue in the habitat quality model where the exponential decay function decreased too quickly and added back linear decay as an option.
- Fixed an InVEST wide issue where some input rasters that were signed bytes did not correctly map to their negative nodata values.
- Hydropower input rasters have been normalized to the LULC size so sampling error is the same for all the input watersheds.
- Adding a check to make sure that input biophysical parameters to the water yield model do not exceed invalid scientific ranges.
- Added a check on nutrient retention in case the upstream water yield was less than 1 so that the log value did not go negative. In that case we clamp upstream water yield to 0.
- A KeyError issue in hydropower was resolved that occurred when the input rasters were at such a coarse resolution that at least one pixel was completely contained in each watershed. Now a value of -9999 will be reported for watersheds that don’t contain any valid data.
- An early version of the monthly water yield model that was erroneously included in was in the installer; it was removed in this version.
- Python scripts necessary for running the ArcGIS version of Coastal Protection were missing. They’ve since been added back to the distribution.
- Raster calculations are now processed by raster block sizes. Improvements in raster reads and writes.
- Fixed an issue in the routing core where some wide DEMs would cause out of memory errors.
- Scenario generator marked as stable.
- Fixed bug in HRA where raster extents of shapefiles were not properly encapsulating the whole AOI.
- Fixed bug in HRA where any number of habitats over 4 would compress the output plots. Now extends the figure so that all plots are correctly scaled.
- Fixed a bug in HRA where the AOI attribute ‘name’ could not be an int. Should now accept any type.
- Fixed bug in HRA which re-wrote the labels if it was run immediately without closing the UI.
- Fixed nodata masking bug in Water Yield when raster extents were less than that covered by the watershed.
- Removed hydropower calibration parameter form water yield model.
- Models that had suffixes used to only allow alphanumeric characters. Now all suffix types are allowed.
- A bug in the core platform that would occasionally cause routing errors on irregularly pixel sized rasters was fixed. This often had the effect that the user would see broken streams and/or nodata values scattered through sediment or nutrient results.
- Wind Energy:
- Added new framework for valuation component. Can now input a yearly price table that spans the lifetime of the wind farm. Also if no price table is made, can specify a price for energy and an annual rate of change.
- Added new memory efficient distance transform functionality
- Added ability to leave out ‘landing points’ in ‘grid connection points’ input. If not landing points are found, it will calculate wind farm directly to grid point distances
- Error message added in Wave Energy if clip shape has no intersection
- Fixed an issue where the data type of the nodata value in a raster might be different than the values in the raster. This was common in the case of 64 bit floating point values as nodata when the underlying raster was 32 bit. Now nodata values are cast to the underlying types which improves the reliability of many of the InVEST models.
3.0.1 (2014-05-19)¶
- Blue Carbon model released.
- HRA UI now properly reflects that the Resolution of Analysis is in meters, not meters squared, and thus will be applied as a side length for a raster pixel.
- HRA now accepts CSVs for ratings scoring that are semicolon separated as well as comma separated.
- Fixed a minor bug in InVEST’s geoprocessing aggregate core that now consistently outputs correct zonal stats from the underlying pixel level hydro outputs which affects the water yield, sediment, and nutrient models.
- Added compression to InVEST output geotiff files. In most cases this reduces output disk usage by a factor of 5.
- Fixed an issue where CSVs in the sediment model weren’t open in universal line read mode.
- Fixed an issue where approximating whether pixel edges were the same size was not doing an approximately equal function.
- Fixed an issue that made the CV model crash when the coastline computed from the landmass didn’t align perfectly with that defined in the geomorphology layer.
- Fixed an issue in the CV model where the intensity of local wave exposure was very low, and yielded zero local wave power for the majority of coastal segments.
- Fixed an issue where the CV model crashes if a coastal segment is at the edge of the shore exposure raster.
- Fixed the exposure of segments surrounded by land that appeared as exposed when their depth was zero.
- Fixed an issue in the CV model where the natural habitat values less than 5 were one unit too low, leading to negative habitat values in some cases.
- Fixed an exponent issue in the CV model where the coastal vulnerability index was raised to a power that was too high.
- Fixed a bug in the Scenic Quality model that prevented it from starting, as well as a number of other issues.
- Updated the pollination model to conform with the latest InVEST geoprocessing standards, resulting in an approximately 33% speedup.
- Improved the UI’s ability to remember the last folder visited, and to have all file and folder selection dialogs have access to this information.
- Fixed an issue in Marine Water Quality where the UV points were supposed to be optional, but instead raised an exception when not passed in.
3.0.0 (2014-03-23)¶
The 3.0.0 release of InVEST represents a shift away from the ArcGIS to the InVEST standalone computational platform. The only exception to this shift is the marine coastal protection tier 1 model which is still supported in an ArcGIS toolbox and has no InVEST 3.0 standalone at the moment. Specific changes are detailed below
- A standalone version of the aesthetic quality model has been developed and packaged along with this release. The standalone outperforms the ArcGIS equivalent and includes a valuation component. See the user’s guide for details.
- The core water routing algorithms for the sediment and nutrient models have been overhauled. The routing algorithms now correctly adjust flow in plateau regions, address a bug that would sometimes not route large sections of a DEM, and has been optimized for both run time and memory performance. In most cases the core d-infinity flow accumulation algorithm out performs TauDEM. We have also packaged a simple interface to these algorithms in a standalone tool called RouteDEM; the functions can also be referenced from the scripting API in the invest_natcap.routing package.
- The sediment and nutrient models are now at a production level release. We no longer support the ArcGIS equivalent of these models.
- The sediment model has had its outputs simplified with major changes including the removal of the ‘pixel mean’ outputs, a direct output of the pixel level export and retention maps, and a single output shapefile whose attribute table contains aggregations of sediment output values. Additionally all inputs to the sediment biophysical table including p, c, and retention coefficients are now expressed as a proportion between 0 and 1; the ArcGIS model had previously required those inputs were integer values between 0 and 1000. See the “Interpreting Results” section of sediment model for full details on the outputs.
- The nutrient model has had a similar overhaul to the sediment model including a simplified output structure with many key outputs contained in the attribute table of the shapefile. Retention coefficients are also expressed in proportions between 0 and 1. See the “Interpreting Results” section of nutrient model for full details on the outputs.
- Fixed a bug in Habitat Risk Assessment where the HRA module would incorrectly error if a criteria with a 0 score (meant to be removed from the assessment) had a 0 data quality or weight.
- Fixed a bug in Habitat Risk Assessment where the average E/C/Risk values across the given subregion were evaluating to negative numbers.
- Fixed a bug in Overlap Analysis where Human Use Hubs would error if run without inter-activity weighting, and Intra-Activity weighting would error if run without Human Use Hubs.
- The runtime performance of the hydropower water yield model has been improved.
- Released InVEST’s implementation of the D-infinity flow algorithm in a tool called RouteDEM available from the start menu.
- Unstable version of blue carbon available.
- Unstable version of scenario generator available.
- Numerous other minor bug fixes and performance enhacnements.
2.6.0 (2013-12-16)¶
The 2.6.0 release of InVEST removes most of the old InVEST models from the Arc toolbox in favor of the new InVEST standalone models. While we have been developing standalone equivalents for the InVEST Arc models since version 2.3.0, this is the first release in which we removed support for the deprecated ArcGIS versions after an internal review of correctness, performance, and stability on the standalones. Additionally, this is one of the last milestones before the InVEST 3.0.0 release later next year which will transition InVEST models away from strict ArcGIS dependence to a standalone form.
Specifically, support for the following models have been moved from the ArcGIS toolbox to their Windows based standalones: (1) hydropower/water yield, (2) finfish aquaculture, (3) coastal protection tier 0/coastal vulnerability, (4) wave energy, (5) carbon, (6) habitat quality/biodiversity, (7) pollination, (8) timber, and (9) overlap analysis. Additionally, documentation references to ArcGIS for those models have been replaced with instructions for launching standalone InVEST models from the Windows start menu.
This release also addresses minor bugs, documentation updates, performance tweaks, and new functionality to the toolset, including:
- A Google doc to provide guidance for scripting the InVEST standalone models: https://docs.google.com/document/d/158WKiSHQ3dBX9C3Kc99HUBic0nzZ3MqW3CmwQgvAqGo/edit?usp=sharing
- Fixed a bug in the sample data that defined Kc as a number between 0 and 1000 instead of a number between 0 and 1.
- Link to report an issue now takes user to the online forums rather than an email address.
- Changed InVEST Sediment model standalone so that retention values are now between 0 and 1 instead of 0 and 100.
- Fixed a bug in Biodiversity where if no suffix were entered output filenames would have a trailing underscore (_) behind them.
- Added documentation to the water purification/nutrient retention model documentation about the standalone outputs since they differ from the ArcGIS version of the model.
- Fixed an issue where the model would try to move the logfile to the workspace after the model run was complete and Windows would erroneously report that the move failed.
- Removed the separation between marine and freshwater terrestrial models in the user’s guide. Now just a list of models.
- Changed the name of InVEST “Biodiversity” model to “Habitat Quality” in the module names, start menu, user’s guide, and sample data folders.
- Minor bug fixes, performance enhancements, and better error reporting in the internal infrastructure.
- HRA risk in the unstable standalone is calculated differently from the last release. If there is no spatial overlap within a cell, there is automatically a risk of 0. This also applies to the E and C intermediate files for a given pairing. If there is no spatial overlap, E and C will be 0 where there is only habitat. However, we still create a recovery potential raster which has habitat- specific risk values, even without spatial overlap of a stressor. HRA shapefile outputs for high, medium, low risk areas are now calculated using a user-defined maximum number of overlapping stressors, rather than all potential stressors. In the HTML subregion averaged output, we now attribute what portion of risk to a habitat comes from each habitat-stressor pairing. Any pairings which don’t overlap will have an automatic risk of 0.
- Major changes to Water Yield : Reservoir Hydropower Production. Changes include an alternative equation for calculating Actual Evapotranspiration (AET) for non-vegetated land cover types including wetlands. This allows for a more accurate representation of processes on land covers such as urban, water, wetlands, where root depth values aren’t applicable. To differentiate between the two equations a column ‘LULC_veg’ has been added to the Biophysical table in Hydropower/input/biophysical_table.csv. In this column a 1 indicates vegetated and 0 indicates non-vegetated.
- The output structure and outputs have also change in Water Yield : Reservoir Hydropower Production. There is now a folder ‘output’ that contains all output files including a sub directory ‘per_pixel’ which has three pixel raster outputs. The subwatershed results are only calculated for the water yield portion and those results can be found as a shapefile, ‘subwatershed_results.shp’, and CSV file, ‘subwatershed_results.csv’. The watershed results can be found in similar files: watershed_results.shp and watershed_results.csv. These two files for the watershed outputs will aggregate the Scarcity and Valuation results as well.
- The evapotranspiration coefficients for crops, Kc, has been changed to a decimal input value in the biophysical table. These values used to be multiplied by 1000 so that they were in integer format, that pre processing step is no longer necessary.
- Changing support from richsharp@stanford.edu to the user support forums at http://ncp-yamato.stanford.edu/natcapforums.
2.5.6 (2013-09-06)¶
The 2.5.6 release of InVEST that addresses minor bugs, performance tweaks, and new functionality of the InVEST standalone models. Including:
- Change the changed the Carbon biophysical table to use code field name from LULC to lucode so it is consistent with the InVEST water yield biophysical table.
- Added Monte Carlo uncertainty analysis and documentation to finfish aquaculture model.
- Replaced sample data in overlap analysis that was causing the model to crash.
- Updates to the overlap analysis user’s guide.
- Added preprocessing toolkit available under C:{InVEST install directory}utils
- Biodiversity Model now exits gracefully if a threat raster is not found in the input folder.
- Wind Energy now uses linear (bilinear because its over 2D space?) interpolation.
- Wind Energy has been refactored to current API.
- Potential Evapotranspiration input has been properly named to Reference Evapotranspiration.
- PET_mn for Water Yield is now Ref Evapotranspiration times Kc (evapotranspiration coefficient).
- The soil depth field has been renamed ‘depth to root restricting layer’ in both the hydropower and nutrient retention models.
- ETK column in biophysical table for Water Yield is now Kc.
- Added help text to Timber model.
- Changed the behavior of nutrient retention to return nodata values when the mean runoff index is zero.
- Fixed an issue where the hydropower model didn’t use the suffix inputs.
- Fixed a bug in Biodiversity that did not allow for numerals in the threat names and rasters.
- Updated routing algorithm to use a modern algorithm for plateau direction resolution.
- Fixed an issue in HRA where individual risk pixels weren’t being calculated correctly.
- HRA will now properly detect in the preprocessed CSVs when criteria or entire habitat-stressor pairs are not desired within an assessment.
- Added an infrastructure feature so that temporary files are created in the user’s workspace rather than at the system level folder. Â This lets users work in a secondary workspace on a USB attached hard drive and use the space of that drive, rather than the primary operating system drive.
2.5.5 (2013-08-06)¶
The 2.5.5 release of InVEST that addresses minor bugs, performance tweaks, and new functionality of the InVEST standalone models. Including:
- Production level release of the 3.0 Coastal Vulnerability model.
- This upgrades the InVEST 2.5.4 version of the beta standalone CV to a full release with full users guide. This version of the CV model should be used in all cases over its ArcGIS equivalent.
- Production level release of the Habitat Risk Assessment model.
- This release upgrades the InVEST 2.5.4 beta version of the standalone habitat risk assessment model. It should be used in all cases over its ArcGIS equivalent.
- Uncertainty analysis in Carbon model (beta)
- Added functionality to assess uncertainty in sequestration and emissions given known uncertainty in carbon pool stocks. Users can now specify standard deviations of carbon pools with normal distributions as well as desired uncertainty levels. New outputs include masks for regions which both sequester and emit carbon with a high probability of confidence. Please see the “Uncertainty Analysis” section of the carbon user’s guide chapter for more information.
- REDD+ Scenario Analysis in Carbon model (beta)
- Additional functionality to assist users evaluating REDD and REDD+ scenarios in the carbon model. The uncertainty analysis functionality can also be used with these scenarios. Please see the “REDD Scenario Analysis” section of the carbon user’s guide chapter for more information.
- Uncertainty analysis in Finfish Aquaculture model (beta)
- Additionally functionality to account for uncertainty in alpha and beta growth parameters as well as histogram plots showing the distribution of harvest weights and net present value. Uncertainty analysis is performed through Monte Carlo runs that normally sample the growth parameters.
- Streamlined Nutrient Retention model functionality
- The nutrient retention module no longer requires users to explicitly run the water yield model. The model now seamlessly runs water yield during execution.
- Beta release of the recreation model
- The recreation is available for beta use with limited documentation.
- Full release of the wind energy model
- Removing the ‘beta’ designation on the wind energy model.
Known Issues:
- Flow routing in the standalone sediment and nutrient models has a bug that prevents routing in some (not all) landscapes. This bug is related to resolving d-infinity flow directions across flat areas. We are implementing the solution in Garbrecht and Martx (1997). In the meanwhile the sediment and nutrient models are still marked as beta until this issue is resolved.
2.5.4 (2013-06-07)¶
This is a minor release of InVEST that addresses numerous minor bugs and performance tweaks in the InVEST 3.0 models. Including:
- Refactor of Wave Energy Model:
- Combining the Biophysical and Valuation modules into one.
- Adding new data for the North Sea and Australia
- Fixed a bug where elevation values that were equal to or greater than zero were being used in calculations.
- Fixed memory issues when dealing with large datasets.
- Updated core functions to remove any use of depracated functions
Performance updates to the carbon model.
- Nodata masking fix for rarity raster in Biodiversity Model.
- When computing rarity from a base landuse raster and current or future landuse raster, the intersection of the two was not being properly taken.
Fixes to the flow routing algorithms in the sediment and nutrient retention models in cases where stream layers were burned in by ArcGIS hydro tools. In those cases streams were at the same elevation and caused routing issues.
Fixed an issue that affected several InVEST models that occured when watershed polygons were too small to cover a pixel. Excessively small watersheds are now handled correctly
Arc model deprecation. We are deprecating the following ArcGIS versions of our InVEST models in the sense we recommend ALL users use the InVEST standalones over the ArcGIS versions, and the existing ArcGIS versions of these models will be removed entirely in the next release.
- Timber
- Carbon
- Pollination
- Biodiversity
- Finfish Aquaculture
Known Issues:
- Flow routing in the standalone sediment and nutrient models has a bug that prevents routing in several landscapes. We’re not certain of the nature of the bug at the moment, but we will fix by the next release. Thus, sediment and nutrient models are marked as (beta) since in some cases the DEM routes correctly.
2.5.3 (2013-03-21)¶
This is a minor release of InVEST that fixes an issue with the HRA model that caused ArcGIS versions of the model to fail when calculating habitat maps for risk hotspots. This upgrade is strongly recommended for users of InVEST 2.5.1 or 2.5.2.
2.5.2 (2013-03-17)¶
This is a minor release of InVEST that fixes an issue with the HRA sample data that caused ArcGIS versions of the model to fail on the training data. There is no need to upgrade for most users unless you are doing InVEST training.
2.5.1 (2013-03-12)¶
This is a minor release of InVEST that does not add any new models, but does add additional functionality, stability, and increased performance to one of the InVEST 3.0 standalones:
- Pollination 3.0 Beta:
- Fixed a bug where Windows users of InVEST could run the model, but most raster outputs were filled with nodata values.
Additionally, this minor release fixes a bug in the InVEST user interface where collapsible containers became entirely non-interactive.
2.5.0 (2013-03-08)¶
This a major release of InVEST that includes new standalone versions (ArcGIS is not required) our models as well as additional functionality, stability, and increased performance to many of the existing models. This release is timed to support our group’s annual training event at Stanford University. We expect to release InVEST 2.5.1 a couple of weeks after to address any software issues that arise during the training. See the release notes below for details of the release, and please contact richsharp@stanford.edu for any issues relating to software:
- new Sediment 3.0 Beta:
- This is a standalone model that executes an order of magnitude faster than the original ArcGIS model, but may have memory issues with larger datasets. This fix is scheduled for the 2.5.1 release of InVEST.
- Uses a d-infinity flow algorithm (ArcGIS version uses D8).
- Includes a more accurate LS factor.
- Outputs are now summarized by polygon rather than rasterized polygons. Users can view results directly as a table rather than sampling a GIS raster.
- new Nutrient 3.0 Beta:
- This is a standalone model that executes an order of magnitude faster than the original ArcGIS model, but may have memory issues with larger datasets. This fix is scheduled for the 2.5.1 release of InVEST.
- Uses a d-infinity flow algorithm (ArcGIS version uses D8).
- Includes a more accurate LS factor.
- Outputs are now summarized by polygon rather than rasterized polygons. Users can view results directly as a table rather than sampling a GIS raster.
- new Wind Energy:
- A new offshore wind energy model. This is a standalone-only model available under the windows start menu.
- new Recreation Alpha:
- This is a working demo of our soon to be released future land and near shore recreation model. The model itself is incomplete and should only be used as a demo or by NatCap partners that know what they’re doing.
- new Habitat Risk Assessment 3.0 Alpha:
- This is a working demo of our soon to be released 3.0 version of habitat risk assessment. The model itself is incomplete and should only be used as a demo or by NatCap partners that know what they’re doing. Users that need to use the habitat risk assessment should use the ArcGIS version of this model.
- Improvements to the InVEST 2.x ArcGIS-based toolset:
- Bug fixes to the ArcGIS based Coastal Protection toolset.
- Removed support for the ArcGIS invest_VERSION.mxd map. We expect to transition the InVEST toolset exclusive standalone tools in a few months. In preparation of this we are starting to deprecate parts of our old ArcGIS toolset including this ArcMap document. The InVEST ArcToolbox is still available in C:InVEST_2_5_0invest_250.tbx.
- Known issues:
- The InVEST 3.0 standalones generate open source GeoTiffs as outputs rather than the proprietary ESRI Grid format. ArcGIS 9.3.1 occasionally displays these rasters incorrectly. We have found that these layers can be visualized in ArcGIS 9.3.1 by following convoluted steps: Right Click on the layer and select Properties; click on the Symbology tab; select Stretch, agree to calculate a histogram (this will create an .aux file that Arc can use for visualization), click “Ok”, remove the raster from the layer list, then add it back. As an alternative, we suggest using an open source GIS Desktop Tool like Quantum GIS or ArcGIS version 10.0 or greater.
- The InVEST 3.0 carbon model will generate inaccurate sequestration results if the extents of the current and future maps don’t align. This will be fixed in InVEST 2.5.1; in the meanwhile a workaround is to clip both LULCs so they have identical overlaps.
- A user reported an unstable run of InVEST 3.0 water yield. We are not certain what is causing the issue, but we do have a fix that will go out in InVEST 2.5.1.
- At the moment the InVEST standalones do not run on Windows XP. This appears to be related to an incompatibility between Windows XP and GDAL, the an open source gis library we use to create and read GIS data. At the moment we are uncertain if we will be able to fix this bug in future releases, but will pass along more information in the future.
2.4.5 (2013-02-01)¶
This is a minor release of InVEST that does not add any new models, but does add additional functionality, stability, and increased performance to many of the InVEST 3.0 standalones:
- Pollination 3.0 Beta:
- Greatly improved memory efficiency over previous versions of this model.
- 3.0 Beta Pollination Biophysical and Valuation have been merged into a single tool, run through a unified user interface.
- Slightly improved runtime through the use of newer core InVEST GIS libraries.
- Optional ability to weight different species individually. This feature adds a column to the Guilds table that allows the user to specify a relative weight for each species, which will be used before combining all species supply rasters.
- Optional ability to aggregate pollinator abundances at specific points provided by an optional points shapefile input.
- Bugfix: non-agricultural pixels are set to a value of 0.0 to indicate no value on the farm value output raster.
- Bugfix: sup_val_<beename>_<scenario>.tif rasters are now saved to the intermediate folder inside the user’s workspace instead of the output folder.
- Carbon Biophysical 3.0 Beta:
- Tweaked the user interface to require the user to provide a future LULC raster when the ‘Calculate Sequestration’ checkbox is checked.
- Fixed a bug that restricted naming of harvest layers. Harvest layers are now selected simply by taking the first available layer.
- Better memory efficiency in hydropower model.
- Better support for unicode filepaths in all 3.0 Beta user interfaces.
- Improved state saving and retrieval when loading up previous-run parameters in all 3.0 Beta user interfaces.
- All 3.0 Beta tools now report elapsed time on completion of a model.
- All 3.0 Beta tools now provide disk space usage reports on completion of a model.
- All 3.0 Beta tools now report arguments at the top of each logfile.
- Biodiversity 3.0 Beta: The half-saturation constant is now allowed to be a positive floating-point number.
- Timber 3.0 Beta: Validation has been added to the user interface for this tool for all tabular and shapefile inputs.
- Fixed some typos in Equation 1 in the Finfish Aquaculture user’s guide.
- Fixed a bug where start menu items were not getting deleted during an InVEST uninstall.
- Added a feature so that if the user selects to download datasets but the datasets don’t successfully download the installation alerts the user and continues normally.
- Fixed a typo with tau in aquaculture guide, originally said 0.8, really 0.08.
- Improvements to the InVEST 2.x ArcGIS-based toolset:
- Minor bugfix to Coastal Vulnerability, where an internal unit of measurements was off by a couple digits in the Fetch Calculator.
- Minor fixes to various helper tools used in InVEST 2.x models.
- Outputs for Hargreaves are now saved as geoTIFFs.
- Thornwaite allows more flexible entering of hours of sunlight.
2.4.4 (2012-10-24)¶
- Fixes memory errors experienced by some users in the Carbon Valuation 3.0 Beta model.
- Minor improvements to logging in the InVEST User Interface
- Fixes an issue importing packages for some officially-unreleased InVEST models.
2.4.3 (2012-10-19)¶
- Fixed a minor issue with hydropower output vaulation rasters whose statistics were not pre-calculated. This would cause the range in ArcGIS to show ther rasters at -3e38 to 3e38.
- The InVEST installer now saves a log of the installation process to InVEST_<version>install_log.txt
- Fixed an issue with Carbon 3.0 where carbon output values were incorrectly calculated.
- Added a feature to Carbon 3.0 were total carbon stored and sequestered is output as part of the running log.
- Fixed an issue in Carbon 3.0 that would occur when users had text representations of floating point numbers in the carbon pool dbf input file.
- Added a feature to all InVEST 3.0 models to list disk usage before and after each run and in most cases report a low free space error if relevant.
2.4.2 (2012-10-15)¶
- Fixed an issue with the ArcMap document where the paths to default data were not saved as relative paths. This caused the default data in the document to not be found by ArcGIS.
- Introduced some more memory-efficient processing for Biodiversity 3.0 Beta. This fixes an out-of-memory issue encountered by some users when using very large raster datasets as inputs.
2.4.1 (2012-10-08)¶
- Fixed a compatibility issue with ArcGIS 9.3 where the ArcMap and ArcToolbox were unable to be opened by Arc 9.3.
2.4.0 (2012-10-05)¶
Changes in InVEST 2.4.0
General:
This is a major release which releases two additional beta versions of the InVEST models in the InVEST 3.0 framework. Additionally, this release introduces start menu shortcuts for all available InVEST 3.0 beta models. Existing InVEST 2.x models can still be found in the included Arc toolbox.
Existing InVEST models migrated to the 3.0 framework in this release include:
- Biodiversity 3.0 Beta
- Minor bug fixes and usability enhancements
- Runtime decreased by a factor of 210
- Overlap Analysis 3.0 Beta
- In most cases runtime decreased by at least a factor of 15
- Minor bug fixes and usability enhancements
- Split into two separate tools:
- Overlap Analysis outputs rasters with individually-weighted pixels
- Overlap Analysis: Management Zones produces a shapefile output.
- Updated table format for input activity CSVs
- Removed the “grid the seascape” step
Updates to ArcGIS models:
- Coastal vulnerability
- Removed the “structures” option
- Minor bug fixes and usability enhancements
- Coastal protection (erosion protection)
- Incorporated economic valuation option
- Minor bug fixes and usability enhancements
Additionally there are a handful of minor fixes and feature enhancements:
- InVEST 3.0 Beta standalones (identified by a new InVEST icon) may be run from the Start Menu (on windows navigate to Start Menu -> All Programs -> InVEST 2.4.0
- Bug fixes for the calculation of raster statistics.
- InVEST 3.0 wave energy no longer requires an AOI for global runs, but encounters memory issues on machines with less than 4GB of RAM. This is a known issue that will be fixed in a minor release.
- Minor fixes to several chapters in the user’s guide.
- Minor bug fix to the 3.0 Carbon model: harvest maps are no longer required inputs.
- Other minor bug fixes and runtime performance tweaks in the 3.0 framework.
- Improved installer allows users to remove InVEST from the Windows Add/Remove programs menu.
- Fixed a visualization bug with wave energy where output rasters did not have the min/max/stdev calculations on them. This made the default visualization in arc be a gray blob.
2.3.0 (2012-08-02)¶
Changes in InVEST 2.3.0
General:
This is a major release which releases several beta versions of the InVEST models in the InVEST 3.0 framework. These models run as standalones, but a GIS platform is needed to edit and view the data inputs and outputs. Until InVEST 3.0 is released the original ArcGIS based versions of these tools will remain the release.
Existing InVEST models migrated to the 3.0 framework in this release include:
- Reservoir Hydropower Production 3.0 beta
- Minor bug fixes.
- Finfish Aquaculture
- Minor bug fixes and usability enhancements.
- Wave Energy 3.0 beta
- Runtimes for non-global runs decreased by a factor of 7
- Minor bugs in interpolation that exist in the 2.x model is fixed in 3.0 beta.
- Crop Pollination 3.0 beta
- Runtimes decreased by a factor of over 10,000
This release also includes the new models which only exist in the 3.0 framework:
- Marine Water Quality 3.0 alpha with a preliminary user’s guide.
InVEST models in the 3.0 framework from previous releases that now have a standalone executable include:
- Managed Timber Production Model
- Carbon Storage and Sequestration
Additionally there are a handful of other minor fixes and feature enhancements since the previous release:
- Minor bug fix to 2.x sedimentation model that now correctly calculates slope exponentials.
- Minor fixes to several chapters in the user’s guide.
- The 3.0 version of the Carbon model now can value the price of carbon in metric tons of C or CO2.
- Other minor bug fixes and runtime performance tweaks in the 3.0 framework.
2.2.2 (2012-03-03)¶
Changes in InVEST 2.2.2
General:
This is a minor release which fixes the following defects:
- -Fixed an issue with sediment retention model where large watersheds
- allowed loading per cell was incorrectly rounded to integer values.
- -Fixed bug where changing the threshold didn’t affect the retention output
- because function was incorrectly rounded to integer values.
-Added total water yield in meters cubed to to output table by watershed.
- -Fixed bug where smaller than default (2000) resolutions threw an error about
- not being able to find the field in “unitynew”. With non-default resolution, “unitynew” was created without an attribute table, so one was created by force.
- -Removed mention of beta state and ecoinformatics from header of software
- license.
- -Modified overlap analysis toolbox so it reports an error directly in the
- toolbox if the workspace name is too long.
2.2.1 (2012-01-26)¶
Changes in InVEST 2.2.1
General:
This is a minor release which fixes the following defects:
-A variety of miscellaneous bugs were fixed that were causing crashes of the Coastal Protection model in Arc 9.3. -Fixed an issue in the Pollination model that was looking for an InVEST1005 directory. -The InVEST “models only” release had an entry for the InVEST 3.0 Beta tools, but was missing the underlying runtime. This has been added to the models only 2.2.1 release at the cost of a larger installer. -The default InVEST ArcMap document wouldn’t open in ArcGIS 9.3. It can now be opened by Arc 9.3 and above. -Minor updates to the Coastal Protection user’s guide.
2.2.0 (2011-12-22)¶
In this release we include updates to the habitat risk assessment model, updates to Coastal Vulnerability Tier 0 (previously named Coastal Protection), and a new tier 1 Coastal Vulnerability tool. Additionally, we are releasing a beta version of our 3.0 platform that includes the terrestrial timber and carbon models.
See the “Marine Models” and “InVEST 3.0 Beta” sections below for more details.
Marine Models
Marine Python Extension Check
This tool has been updated to include extension requirements for the new Coastal Protection T1 model. It also reflects changes to the Habitat Risk Assessment and Coastal Protection T0 models, as they no longer require the PythonWin extension.
Habitat Risk Assessment (HRA)
This model has been updated and is now part of three-step toolset. The first step is a new Ratings Survey Tool which eliminates the need for Microsoft Excel when users are providing habitat-stressor ratings. This Survey Tool now allows users to up- and down-weight the importance of various criteria. For step 2, a copy of the Grid the Seascape tool has been placed in the HRA toolset. In the last step, users will run the HRA model which includes the following updates:
- New habitat outputs classifying risk as low, medium, and high
- Model run status updates (% complete) in the message window
- Improved habitat risk plots embedded in the output HTML
Coastal Protection
This module is now split into sub-models, each with two parts. The first sub-model is Coastal Vulnerability (Tier 0) and the new addition is Coastal Protection (Tier 1).
Coastal Vulnerability (T0) Step 1) Fetch Calculator - there are no updates to this tool. Step 2) Vulnerability Index
- Wave Exposure: In this version of the model, we define wave exposure for sites facing the open ocean as the maximum of the weighted average of wave’s power coming from the ocean or generated by local winds. We weight wave power coming from each of the 16 equiangular sector by the percent of time that waves occur in that sector, and based on whether or not fetch in that sector exceeds 20km. For sites that are sheltered, wave exposure is the average of wave power generated by the local storm winds weighted by the percent occurrence of those winds in each sector. This new method takes into account the seasonality of wind and wave patterns (storm waves generally come from a preferential direction), and helps identify regions that are not exposed to powerful waves although they are open to the ocean (e.g. the leeside of islands).
- Natural Habitats: The ranking is now computed using the rank of all natural habitats present in front of a segment, and we weight the lowest ranking habitat 50% more than all other habitats. Also, rankings and protective distance information are to be provided by CSV file instead of Excel. With this new method, shoreline segments that have more habitats than others will have a lower risk of inundation and/or erosion during storms.
- Structures: The model has been updated to now incorporate the presence of structures by decreasing the ranking of shoreline segments that adjoin structures.
Coastal Protection (T1) - This is a new model which plots the amount of sandy beach erosion or consolidated bed scour that backshore regions experience in the presence or absence of natural habitats. It is composed of two steps: a Profile Generator and Nearshore Waves and Erosion. It is recommended to run the Profile Generator before the Nearshore Waves and Erosion model.
Step 1) Profile Generator: This tool helps the user generate a 1-dimensional bathymetric and topographic profile perpendicular to the shoreline at the user-defined location. This model provides plenty of guidance for building backshore profiles for beaches, marshes and mangroves. It will help users modify bathymetry profiles that they already have, or can generate profiles for sandy beaches if the user has not bathymetric data. Also, the model estimates and maps the location of natural habitats present in front of the region of interest. Finally, it provides sample wave and wind data that can be later used in the Nearshore Waves and Erosion model, based on computed fetch values and default Wave Watch III data.
Step 2) Nearshore Waves and Erosion: This model estimates profiles of beach erosion or values of rates of consolidated bed scour at a site as a function of the type of habitats present in the area of interest. The model takes into account the protective effects of vegetation, coral and oyster reefs, and sand dunes. It also shows the difference of protection provided when those habitats are present, degraded, or gone.
Aesthetic Quality
This model no longer requires users to provide a projection for Overlap Analysis. Instead, it uses the projection from the user-specified Area of Interest (AOI) polygon. Additionally, the population estimates for this model have been fixed.
InVEST 3.0 Beta
The 2.2.0 release includes a preliminary version of our InVEST 3.0 beta platform. It is included as a toolset named “InVEST 3.0 Beta” in the InVEST220.tbx. It is currently only supported with ArcGIS 10. To launch an InVEST 3.0 beta tool, double click on the desired tool in the InVEST 3.0 toolset then click “Ok” on the Arc toolbox screen that opens. The InVEST 3.0 tool panel has inputs very similar to the InVEST 2.2.0 versions of the tools with the following modifications:
- InVEST 3.0 Carbon:
- Fixes a minor bug in the 2.2 version that ignored floating point values in carbon pool inputs.
- Separation of carbon model into a biophysical and valuation model.
- Calculates carbon storage and sequestration at the minimum resolution of the input maps.
- Runtime efficiency improved by an order of magnitude.
- User interface streamlined including dynamic activation of inputs based on user preference, direct link to documentation, and recall of inputs based on user’s previous run.
- InVEST 3.0 Timber:
- User interface streamlined including dynamic activation of inputs based on user preference, direct link to documentation, and recall of inputs based on user’s previous run.
2.1.1 (2011-10-17)¶
Changes in InVEST 2.1.1
General:
This is a minor release which fixes the following defects:
-A truncation error was fixed on nutrient retention and sedimentation model that involved division by the number of cells in a watershed. Now correctly calculates floating point division. -Minor typos were fixed across the user’s guide.
2.1 Beta (2011-05-11)¶
Updates to InVEST Beta
InVEST 2.1 . Beta
Changes in InVEST 2.1
General:
1. InVEST versioning We have altered our versioning scheme. Integer changes will reflect major changes (e.g. the addition of marine models warranted moving from 1.x to 2.0). An increment in the digit after the primary decimal indicates major new features (e.g the addition of a new model) or major revisions. For example, this release is numbered InVEST 2.1 because two new models are included). We will add another decimal to reflect minor feature revisions or bug fixes. For example, InVEST 2.1.1 will likely be out soon as we are continually working to improve our tool. 2. HTML guide With this release, we have migrated the entire InVEST users. guide to an HTML format. The HTML version will output a pdf version for use off-line, printing, etc.
MARINE MODELS
1.Marine Python Extension Check
-This tool has been updated to allow users to select the marine models they intend to run. Based on this selection, it will provide a summary of which Python and ArcGIS extensions are necessary and if the Python extensions have been successfully installed on the user.s machine.
2.Grid the Seascape (GS)
-This tool has been created to allow marine model users to generate an seascape analysis grid within a specified area of interest (AOI).
-It only requires an AOI and cell size (in meters) as inputs, and produces a polygon grid which can be used as inputs for the Habitat Risk Assessment and Overlap Analysis models.
- Coastal Protection
- This is now a two-part model for assessing Coastal Vulnerability. The first part is a tool for calculating fetch and the second maps the value of a Vulnerability Index, which differentiates areas with relatively high or low exposure to erosion and inundation during storms.
- The model has been updated to now incorporate coastal relief and the protective influence of up to eight natural habitat input layers.
- A global Wave Watch 3 dataset is also provided to allow users to quickly generate rankings for wind and wave exposure worldwide.
- Habitat Risk Assessment (HRA)
This new model allows users to assess the risk posed to coastal and marine habitats by human activities and the potential consequences of exposure for the delivery of ecosystem services and biodiversity. The HRA model is suited to screening the risk of current and future human activities in order to prioritize management strategies that best mitigate risk.
- Overlap Analysis
This new model maps current human uses in and around the seascape and summarizes the relative importance of various regions for particular activities. The model was designed to produce maps that can be used to identify marine and coastal areas that are most important for human use, in particular recreation and fisheries, but also other activities.
FRESHWATER MODELS
All Freshwater models now support ArcMap 10.
Sample data:
- Bug fix for error in Water_Tables.mdb Biophysical table where many field values were shifted over one column relative to the correct field name.
- Bug fix for incorrect units in erosivity layer.
Hydropower:
1.In Water Yield, new output tables have been added containing mean biophysical outputs (precipitation, actual and potential evapotranspiration, water yield) for each watershed and sub-watershed.
Water Purification:
- The Water Purification Threshold table now allows users to specify separate thresholds for nitrogen and phosphorus. Field names thresh_n and thresh_p replace the old ann_load.
- The Nutrient Retention output tables nutrient_watershed.dbf and nutrient_subwatershed.dbf now include a column for nutrient retention per watershed/sub-watershed.
- In Nutrient Retention, some output file names have changed.
- The user’s guide has been updated to explain more accurately the inclusion of thresholds in the biophysical service estimates.
Sedimentation:
- The Soil Loss output tables sediment_watershed.dbf and sediment_subwatershed.dbf now include a column for sediment retention per watershed/sub-watershed.
- In Soil Loss, some output file names have changed.
- The default input value for Slope Threshold is now 75.
- The user’s guide has been updated to explain more accurately the inclusion of thresholds in the biophysical service estimates.
- Valuation: Bug fix where the present value was not being applied correctly.
2.0 Beta (2011-02-14)¶
Changes in InVEST 2.0
InVEST 1.005 is a minor release with the following modification:
Aesthetic Quality
This new model allows users to determine the locations from which new nearshore or offshore features can be seen. It generates viewshed maps that can be used to identify the visual footprint of new offshore development.
Coastal Vulnerability
This new model produces maps of coastal human populations and a coastal exposure to erosion and inundation index map. These outputs can be used to understand the relative contributions of different variables to coastal exposure and to highlight the protective services offered by natural habitats.
Aquaculture
This new model is used to evaluate how human activities (e.g., addition or removal of farms, changes in harvest management practices) and climate change (e.g., change in sea surface temperature) may affect the production and economic value of aquacultured Atlantic salmon.
Wave Energy
This new model provides spatially explicit information, showing potential areas for siting Wave Energy conversion (WEC) facilities with the greatest energy production and value. This site- and device-specific information for the WEC facilities can then be used to identify and quantify potential trade-offs that may arise when siting WEC facilities.
Avoided Reservoir Sedimentation
- The name of this model has been changed to the Sediment Retention model.
- We have added a water quality valuation model for sediment retention. The user now has the option to select avoided dredge cost analysis, avoided water treatment cost analysis or both. The water quality valuation approach is the same as that used in the Water Purification: Nutrient Retention model.
- The threshold information for allowed sediment loads (TMDL, dead volume, etc.) are now input in a stand alone table instead of being included in the valuation table. This adjusts the biophysical service output for any social allowance of pollution. Previously, the adjustment was only done in the valuation model.
- The watersheds and sub-watershed layers are now input as shapefiles instead of rasters.
- Final outputs are now aggregated to the sub-basin scale. The user must input a sub-basin shapefile. We provide the Hydro 1K dataset as a starting option. See users guide for changes to many file output names.
- Users are strongly advised not to interpret pixel-scale outputs for hydrological understanding or decision-making of any kind. Pixel outputs should only be used for calibration/validation or model checking.
Hydropower Production
- The watersheds and sub-watershed layers are now input as shapefiles instead of rasters.
- Final outputs are now aggregated to the sub-basin scale. The user must input a sub-basin shapefile. We provide the Hydro 1K dataset as a starting option. See users guide for changes to many file output names.
- Users are strongly advised not to interpret pixel-scale outputs for hydrological understanding or decision-making of any kind. Pixel outputs should only be used for calibration/validation or model checking.
- The calibration constant for each watershed is now input in a stand-alone table instead of being included in the valuation table. This makes running the water scarcity model simpler.
Water Purification: Nutrient Retention
- The threshold information for allowed pollutant levels (TMDL, etc.) are now input in a stand alone table instead of being included in the valuation table. This adjusts the biophysical service output for any social allowance of pollution. Previously, the adjustment was only done in the valuation model.
- The watersheds and sub-watershed layers are now input as shapefiles instead of rasters.
- Final outputs are now aggregated to the sub-basin scale. The user must input a sub-basin shapefile. We provide the Hydro 1K dataset as a starting option. See users guide for changes to many file output names.
- Users are strongly advised not to interpret pixel-scale outputs for hydrological understanding or decision-making of any kind. Pixel outputs should only be used for calibration/validation or model checking.
Carbon Storage and Sequestration
The model now outputs an aggregate sum of the carbon storage.
Habitat Quality and Rarity
This model had an error while running ReclassByACII if the land cover codes were not sorted alphabetically. This has now been corrected and it sorts the reclass file before running the reclassification
The model now outputs an aggregate sum of the habitat quality.
Pollination
In this version, the pollination model accepts an additional parameter which indicated the proportion of a crops yield that is attributed to wild pollinators.
Tutorial: Batch Processing on Windows¶
Introduction¶
These are the steps that will need to be taken in order to use the batch
scripting framework for InVEST models available in the natcap.invest
python
package.
Note
The natcap.invest
python package is currently only supported in Python
2.7. Other versions of python may be supported at a later date.
Setting up your Python environment¶
Install Python 2.7.11 or later.
Python can be downloaded from here. When installing, be sure to allow
python.exe
to be added to the path in the installation options.Put pip on the PATH.
The
pip
utility for installing python packages is already included with Python 2.7.9 and later. Be sure to addC:\Python27\Scripts
to the Windows PATH environment variable so thatpip
can be called from the command line without needing to use its full path.After this is done (and you’ve opened a new command-line window), you will be able to use
pip
at the command-line to install packages like so:> pip install <packagename>
Install packages needed to run InVEST.
Most (maybe even all) of these packages can be downloaded as precompiled wheels from Christoph Gohlke’s build page. Others should be able to be installed via
pip install <packagename>
.# # Any lines with "# pip-only" at the end will be processed by # scripts/convert-requirements-to-conda-yml.py as though it can only be found # on pip. GDAL>=2.0,<3.0 Pyro4==4.41 # pip-only pandas>=0.22.0 numpy>=1.11.0,!=1.16.0 Rtree>=0.8.2 scipy>=0.16.1 Shapely>=1.6.4 six pygeoprocessing>=1.6.1 taskgraph[niced_processes]>=0.8.2 psutil<5.6.0 chardet>=3.0.4 matplotlib xlrd>=1.2.0
Install the InVEST python package
4a. Download a release of the
natcap.invest
python package.4b. Install the downloaded python package..
win32.whl
files are prebuilt binary distributions and can be installed via pip. See the pip docs for installing a package from a wheelwin32-py2.7.exe
files are also prebuilt binary distributions, but cannot be installed by pip. Instead, double-click the downloaded file to launch an installer..zip
and.tar.gz
files are source archives. See Installing from Source for details.
Creating Sample Python Scripts¶
Launch InVEST Model
Once an InVEST model is selected for scripting, launch that model from the Windows Start menu. This example in this guide follows the NDR model.
Fill in InVEST Model Input Parameters
Once the user interface loads, populate the inputs in the model likely to be used in the Python script. For testing purposes the default InVEST’s data is appropriate. However, if a user wishes to write a batch for several InVEST runs, it would be reasonable to populate the user interface with data for the first run.
Generate a sample Python Script from the User Interface
Open the Development menu at the top of the user interface and select “Save to python script…” and save the file to a known location.
Execute the script in the InVEST Python Environment
Launch a Windows PowerShell from the Start menu (type “powershell” in the search box), then invoke the Python interpreter on the InVEST Python script from that shell. In this example the Python interpreter is installed in
C:\Python27\python.exe
and the script was saved inC:\Users\rpsharp\Desktop\ndr.py
, thus the command to invoke the interpreter is:> C:\Python27\python.exe C:\Users\rpsharp\Desktop\ndr.py
Output Results
As the model executes, status information will be printed to the console. Once complete, model results can be found in the workspace folder selected during the initial configuration.
Modifying a Python Script¶
InVEST Python scripts consist of two sections:
- The argument dictionary that represents the model’s user interface input boxes and parameters.
- The call to the InVEST model itself.
For reference, consider the following script generated by the Nutrient model with default data inputs:
"""
This is a saved model run from natcap.invest.ndr.ndr.
Generated: Mon 16 May 2016 03:52:59 PM
InVEST version: 3.3.0
"""
import natcap.invest.ndr.ndr
args = {
u'k_param': u'2',
u'runoff_proxy_uri': u'C:\InVEST_3.3.0_x86\Base_Data\Freshwater\precip',
u'subsurface_critical_length_n': u'150',
u'subsurface_critical_length_p': u'150',
u'subsurface_eff_n': u'0.8',
u'subsurface_eff_p': u'0.8',
u'threshold_flow_accumulation': u'1000',
u'biophysical_table_uri': u'C:\InVEST_3.3.0_x86\WP_Nutrient_Retention\Input\water_biophysical_table.csv',
u'calc_n': True,
u'calc_p': True,
u'suffix': '',
u'dem_uri': u'C:\InVEST_3.3.0_x86\Base_Data\Freshwater\dem',
u'lulc_uri': u'C:\InVEST_3.3.0_x86\Base_Data\Freshwater\landuse_90',
u'watersheds_uri': u'C:\InVEST_3.3.0_x86\Base_Data\Freshwater\watersheds.shp',
u'workspace_dir': u'C:\InVEST_3.3.0_x86\ndr_workspace',
}
if __name__ == '__main__':
natcap.invest.ndr.ndr.execute(args)
Elements to note:
- Parameter Python Dictionary: Key elements include the
‘args’
dictionary. Note the similarities between the key values such as‘workspace_dir’
and the equivalent “Workspace” input parameter in the user interface. Every key in the‘args’
dictionary has a corresponding reference in the user interface.
In the example below we’ll modify the script to execute the nutrient model for a parameter study of ‘threshold_flow_accumulation’.
- Execution of the InVEST model: The InVEST API invokes models with a consistent syntax where the module name that contains the InVEST model is listed first and is followed by a function called ‘execute’ that takes a single parameter called
‘args’
. This parameter is the dictionary of input parameters discussed above. In this example, the line
natcap.invest.ndr.ndr.execute(args)
executes the nutrient model end-to-end. If the user wishes to make batch calls to InVEST, this line will likely be placed inside a loop.
Example: Threshold Flow Accumulation Parameter Study¶
This example executes the InVEST NDR model on 10 values of threshold accumulation stepping from 500 to 1000 pixels in steps of 50. To modify the script above, replace the execution call with the following loop:
#Loops through the values 500, 550, 600, ... 1000
for threshold_flow_accumulation in range(500, 1001, 50):
#set the accumulation threshold to the current value in the loop
args['threshold_flow_accumulation'] = threshold_flow_accumulation
#set the suffix to be accum### for the current threshold_flow_accumulation
args['suffix'] = 'accum' + str(threshold_flow_accumulation)
natcap.invest.ndr.ndr.execute(args)
This loop executes the InVEST nutrient model 10 times for accumulation values
500, 550, 600, ... 1000
and adds a suffix to the output files so results
can be distinguished.
Example: Invoke NDR Model on a directory of Land Cover Maps¶
In this case we invoke the InVEST nutrient model on a directory of land cover data located at C:UserRichDesktoplandcover_data. As in the previous example, replace the last line in the UI generated Python script with:
import os
landcover_dir = r'C:\User\Rich\Desktop\landcover_data'
#Loop over all the filenames in the landcover dir
for landcover_file in os.listdir(landcover_dir):
#Point the landuse uri parameter at the directory+filename
args['lulc_uri'] = os.path.join(landcover_dir, landcover_file)
#Make a useful suffix so we can differentiate the results
args['suffix'] = 'landmap' + os.path.splitext(landcover_file)[0]
#call the nutrient model
natcap.invest.ndr.ndr.execute(args)
This loop covers all the files located in
C:\User\Rich\Desktop\landcover_data
and updates the relevant lulc_uri
key in the args dictionary to each
of those files during execution as well as making a useful suffix so output
files can be distinguished from each other.
Example: Saving model log messages to a file¶
There are many cases where you may want or need to capture all of the log
messages generated by the model. When we run models through the InVEST user
interface application, the UI captures all of this logging and saves it to a
logfile. We can replicate this behavior through the python logging package,
by adding the following code just after the import
statements in the
example script.
import logging
import pygeoprocessing
# Write all NDR log messages to logfile.txt
MODEL_LOGGER = natcap.invest.ndr.ndr.LOGGER
handler = logging.FileHandler('logfile.txt')
MODEL_LOGGER.addHandler(handler)
# log pygeoprocessing messages to the same logfile
PYGEO_LOGGER = pygeoprocessing.geoprocessing.LOGGER
PYGEO_LOGGER.addHandler(handler)
This will capture all logging generated by the ndr
model and by
pygeoprocessing
, writing all messages to logfile.txt
. While
this is a common use case, the logging
package provides functionality
for many more complex logging features. For more
advanced use of the python logging module, refer to the Python project’s
Logging Cookbook
Summary¶
The InVEST scripting framework was designed to assist InVEST users in automating batch runs or adding custom functionality to the existing InVEST software suite. Support questions can be directed to the NatCap support forums at http://forums.naturalcapitalproject.org.
API Reference¶
InVEST Model Entry Points¶
All InVEST models share a consistent python API:
- The model has a function called
execute
that takes a single python dict ("args"
) as its argument.- This arguments dict contains an entry,
'workspace_dir'
, which points to the folder on disk where all files created by the model should be saved.
Calling a model requires importing the model’s execute function and then calling the model with the correct parameters. For example, if you were to call the Carbon Storage and Sequestration model, your script might include
import natcap.invest.carbon.carbon_combined
args = {
'workspace_dir': 'path/to/workspace'
# Other arguments, as needed for Carbon.
}
natcap.invest.carbon.carbon_combined.execute(args)
For examples of scripts that could be created around a model run, or multiple successive model runs, see Creating Sample Python Scripts.
Available Models and Tools:
- Annual Water Yield: Reservoir Hydropower Production
- Coastal Blue Carbon
- Coastal Blue Carbon Preprocessor
- Crop Production Percentile Model
- Crop Production Regression Model
- Finfish Aquaculture
- Fisheries: Habitat Scenario Tool
- Forest Carbon Edge Effect
- GLOBIO
- Habitat Quality
- InVEST Carbon Model
- InVEST Habitat Risk Assessment (HRA) Model
- InVEST Pollination Model
- Recreation
- RouteDEM: Hydrological routing
- Scenario Generator: Proximity-Based
- Sediment Delivery Ratio
- Wave Energy
- Wind Energy
Annual Water Yield: Reservoir Hydropower Production¶
-
natcap.invest.hydropower.hydropower_water_yield.
execute
(args)¶ Annual Water Yield: Reservoir Hydropower Production.
Executes the hydropower/water_yield model
Parameters: - args['workspace_dir'] (string) – a path to the directory that will write output and other temporary files during calculation. (required)
- args['lulc_path'] (string) – a path to a land use/land cover raster whose LULC indexes correspond to indexes in the biophysical table input. Used for determining soil retention and other biophysical properties of the landscape. (required)
- args['depth_to_root_rest_layer_path'] (string) – a path to an input raster describing the depth of “good” soil before reaching this restrictive layer (required)
- args['precipitation_path'] (string) – a path to an input raster describing the average annual precipitation value for each cell (mm) (required)
- args['pawc_path'] (string) – a path to an input raster describing the plant available water content value for each cell. Plant Available Water Content fraction (PAWC) is the fraction of water that can be stored in the soil profile that is available for plants’ use. PAWC is a fraction from 0 to 1 (required)
- args['eto_path'] (string) – a path to an input raster describing the annual average evapotranspiration value for each cell. Potential evapotranspiration is the potential loss of water from soil by both evaporation from the soil and transpiration by healthy Alfalfa (or grass) if sufficient water is available (mm) (required)
- args['watersheds_path'] (string) – a path to an input shapefile of the watersheds of interest as polygons. (required)
- args['sub_watersheds_path'] (string) – a path to an input shapefile of
the subwatersheds of interest that are contained in the
args['watersheds_path']
shape provided as input. (optional) - args['biophysical_table_path'] (string) – a path to an input CSV table of land use/land cover classes, containing data on biophysical coefficients such as root_depth (mm) and Kc, which are required. A column with header LULC_veg is also required which should have values of 1 or 0, 1 indicating a land cover type of vegetation, a 0 indicating non vegetation or wetland, water. NOTE: these data are attributes of each LULC class rather than attributes of individual cells in the raster map (required)
- args['seasonality_constant'] (float) – floating point value between 1 and 30 corresponding to the seasonal distribution of precipitation (required)
- args['results_suffix'] (string) – a string that will be concatenated onto the end of file names (optional)
- args['demand_table_path'] (string) – (optional) if a non-empty string, a path to an input CSV table of LULC classes, showing consumptive water use for each landuse / land-cover type (cubic meters per year) to calculate water scarcity.
- args['valuation_table_path'] (string) –
(optional) if a non-empty string, a path to an input CSV table of hydropower stations with the following fields to calculate valuation:
(‘ws_id’, ‘time_span’, ‘discount’, ‘efficiency’, ‘fraction’, ‘cost’, ‘height’, ‘kw_price’)Required if
calculate_valuation
is True. - args['n_workers'] (int) – (optional) The number of worker processes to use for processing this model. If omitted, computation will take place in the current process.
Returns: None
Coastal Blue Carbon¶
-
natcap.invest.coastal_blue_carbon.coastal_blue_carbon.
execute
(args)¶ Coastal Blue Carbon.
Parameters: - workspace_dir (str) – location into which all intermediate and output files should be placed.
- results_suffix (str) – a string to append to output filenames.
- lulc_lookup_uri (str) – filepath to a CSV table used to convert the lulc code to a name. Also used to determine if a given lulc type is a coastal blue carbon habitat.
- lulc_transition_matrix_uri (str) – generated by the preprocessor. This file must be edited before it can be used by the main model. The left-most column represents the source lulc class, and the top row represents the destination lulc class.
- carbon_pool_initial_uri (str) – the provided CSV table contains information related to the initial conditions of the carbon stock within each of the three pools of a habitat. Biomass includes carbon stored above and below ground. All non-coastal blue carbon habitat lulc classes are assumed to contain no carbon. The values for ‘biomass’, ‘soil’, and ‘litter’ should be given in terms of Megatonnes CO2 e/ ha.
- carbon_pool_transient_uri (str) – the provided CSV table contains information related to the transition of carbon into and out of coastal blue carbon pools. All non-coastal blue carbon habitat lulc classes are assumed to neither sequester nor emit carbon as a result of change. The ‘yearly_accumulation’ values should be given in terms of Megatonnes of CO2 e/ha-yr. The ‘half-life’ values must be given in terms of years. The ‘disturbance’ values must be given as a decimal (e.g. 0.5 for 50%) of stock distrubed given a transition occurs away from a lulc-class.
- lulc_baseline_map_uri (str) – a GDAL-supported raster representing the baseline landscape/seascape.
- lulc_baseline_year (int) – The year of the baseline snapshot.
- lulc_transition_maps_list (list) – a list of GDAL-supported rasters representing the landscape/seascape at particular points in time. Provided in chronological order.
- lulc_transition_years_list (list) – a list of years that respectively correspond to transition years of the rasters. Provided in chronological order.
- analysis_year (int) – optional. Indicates how many timesteps to run the transient analysis beyond the last transition year. Must come chronologically after the last transition year if provided. Otherwise, the final timestep of the model will be set to the last transition year.
- do_economic_analysis (bool) – boolean value indicating whether model should run economic analysis.
- do_price_table (bool) – boolean value indicating whether a price table is included in the arguments and to be used or a price and interest rate is provided and to be used instead.
- price (float) – the price per Megatonne CO2 e at the base year.
- inflation_rate (float) – the interest rate on the price per Megatonne CO2e, compounded yearly. Provided as a percentage (e.g. 3.0 for 3%).
- price_table_uri (bool) – if args[‘do_price_table’] is set to True the provided CSV table is used in place of the initial price and interest rate inputs. The table contains the price per Megatonne CO2e sequestered for a given year, for all years from the original snapshot to the analysis year, if provided.
- discount_rate (float) – the discount rate on future valuations of sequestered carbon, compounded yearly. Provided as a percentage (e.g. 3.0 for 3%).
Example Args:
args = { 'workspace_dir': 'path/to/workspace/', 'results_suffix': '', 'lulc_lookup_uri': 'path/to/lulc_lookup_uri', 'lulc_transition_matrix_uri': 'path/to/lulc_transition_uri', 'carbon_pool_initial_uri': 'path/to/carbon_pool_initial_uri', 'carbon_pool_transient_uri': 'path/to/carbon_pool_transient_uri', 'lulc_baseline_map_uri': 'path/to/baseline_map.tif', 'lulc_baseline_year': <int>, 'lulc_transition_maps_list': [raster1_uri, raster2_uri, ...], 'lulc_transition_years_list': [2000, 2005, ...], 'analysis_year': 2100, 'do_economic_analysis': '<boolean>', 'do_price_table': '<boolean>', 'price': '<float>', 'inflation_rate': '<float>', 'price_table_uri': 'path/to/price_table', 'discount_rate': '<float>' }
Coastal Blue Carbon Preprocessor¶
-
natcap.invest.coastal_blue_carbon.preprocessor.
execute
(args)¶ Coastal Blue Carbon Preprocessor.
The preprocessor accepts a list of rasters and checks for cell-transitions across the rasters. The preprocessor outputs a CSV file representing a matrix of land cover transitions, each cell prefilled with a string indicating whether carbon accumulates or is disturbed as a result of the transition, if a transition occurs.
Parameters: - workspace_dir (string) – directory path to workspace
- results_suffix (string) – append to outputs directory name if provided
- lulc_lookup_uri (string) – filepath of lulc lookup table
- lulc_snapshot_list (list) – a list of filepaths to lulc rasters
Example Args:
args = { 'workspace_dir': 'path/to/workspace_dir/', 'results_suffix': '', 'lulc_lookup_uri': 'path/to/lookup.csv', 'lulc_snapshot_list': ['path/to/raster1', 'path/to/raster2', ...] }
Crop Production Percentile Model¶
-
natcap.invest.crop_production_percentile.
execute
(args)¶ Crop Production Percentile Model.
This model will take a landcover (crop cover?) map and produce yields, production, and observed crop yields, a nutrient table, and a clipped observed map.
Parameters: - args['workspace_dir'] (string) – output directory for intermediate, temporary, and final files
- args['results_suffix'] (string) – (optional) string to append to any output file names
- args['landcover_raster_path'] (string) – path to landcover raster
- args['landcover_to_crop_table_path'] (string) –
path to a table that converts landcover types to crop names that has two headers: * lucode: integer value corresponding to a landcover code in
args[‘landcover_raster_path’].- crop_name: a string that must match one of the crops in args[‘model_data_path’]/climate_bin_maps/[cropname]_* A ValueError is raised if strings don’t match.
- args['aggregate_polygon_path'] (string) – path to polygon shapefile that will be used to aggregate crop yields and total nutrient value. (optional, if value is None, then skipped)
- args['model_data_path'] (string) –
path to the InVEST Crop Production global data directory. This model expects that the following directories are subdirectories of this path * climate_bin_maps (contains [cropname]_climate_bin.tif files) * climate_percentile_yield (contains
[cropname]_percentile_yield_table.csv files)Please see the InVEST user’s guide chapter on crop production for details about how to download these data.
- args['n_workers'] (int) – (optional) The number of worker processes to use for processing this model. If omitted, computation will take place in the current process.
Returns: None.
Crop Production Regression Model¶
-
natcap.invest.crop_production_regression.
execute
(args)¶ Crop Production Regression Model.
This model will take a landcover (crop cover?), N, P, and K map and produce modeled yields, and a nutrient table.
Parameters: - args['workspace_dir'] (string) – output directory for intermediate, temporary, and final files
- args['results_suffix'] (string) – (optional) string to append to any output file names
- args['landcover_raster_path'] (string) – path to landcover raster
- args['landcover_to_crop_table_path'] (string) –
path to a table that converts landcover types to crop names that has two headers: * lucode: integer value corresponding to a landcover code in
args[‘landcover_raster_path’].- crop_name: a string that must match one of the crops in args[‘model_data_path’]/climate_regression_yield_tables/[cropname]_* A ValueError is raised if strings don’t match.
- args['fertilization_rate_table_path'] (string) – path to CSV table that contains fertilization rates for the crops in the simulation, though it can contain additional crops not used in the simulation. The headers must be ‘crop_name’, ‘nitrogen_rate’, ‘phosphorous_rate’, and ‘potassium_rate’, where ‘crop_name’ is the name string used to identify crops in the ‘landcover_to_crop_table_path’, and rates are in units kg/Ha.
- args['aggregate_polygon_path'] (string) – path to polygon shapefile that will be used to aggregate crop yields and total nutrient value. (optional, if value is None, then skipped)
- args['model_data_path'] (string) –
path to the InVEST Crop Production global data directory. This model expects that the following directories are subdirectories of this path * climate_bin_maps (contains [cropname]_climate_bin.tif files) * climate_percentile_yield (contains
[cropname]_percentile_yield_table.csv files)Please see the InVEST user’s guide chapter on crop production for details about how to download these data.
Returns: None.
Finfish Aquaculture¶
-
natcap.invest.finfish_aquaculture.finfish_aquaculture.
execute
(args)¶ Finfish Aquaculture.
This function will take care of preparing files passed into the finfish aquaculture model. It will handle all files/inputs associated with biophysical and valuation calculations and manipulations. It will create objects to be passed to the aquaculture_core.py module. It may write log, warning, or error messages to stdout.
Parameters: - workspace_dir (string) – The directory in which to place all result files.
- ff_farm_loc (string) – URI that points to a shape file of fishery locations
- farm_ID (string) – column heading used to describe individual farms. Used to link GIS location data to later inputs.
- g_param_a (float) – Growth parameter alpha, used in modeling fish growth, should be an int or float.
- g_param_b (float) – Growth parameter beta, used in modeling fish growth, should be an int or float.
- g_param_tau (float) – Growth parameter tau, used in modeling fish growth, should be an int or float
- use_uncertainty (boolean) –
- g_param_a_sd (float) – (description)
- g_param_b_sd (float) – (description)
- num_monte_carlo_runs (int) –
- water_temp_tbl (string) – URI to a CSV table where daily water temperature values are stored from one year
- farm_op_tbl (string) – URI to CSV table of static variables for calculations
- outplant_buffer (int) – This value will allow the outplanting start day to be flexible plus or minus the number of days specified here.
- do_valuation (boolean) – Boolean that indicates whether or not valuation should be performed on the aquaculture model
- p_per_kg (float) – Market price per kilogram of processed fish
- frac_p (float) – Fraction of market price that accounts for costs rather than profit
- discount (float) – Daily market discount rate
Example Args Dictionary:
{ 'workspace_dir': 'path/to/workspace_dir', 'ff_farm_loc': 'path/to/shapefile', 'farm_ID': 'FarmID' 'g_param_a': 0.038, 'g_param_b': 0.6667, 'g_param_tau': 0.08, 'use_uncertainty': True, 'g_param_a_sd': 0.005, 'g_param_b_sd': 0.05, 'num_monte_carlo_runs': 1000, 'water_temp_tbl': 'path/to/water_temp_tbl', 'farm_op_tbl': 'path/to/farm_op_tbl', 'outplant_buffer': 3, 'do_valuation': True, 'p_per_kg': 2.25, 'frac_p': 0.3, 'discount': 0.000192, }
Fisheries: Habitat Scenario Tool¶
-
natcap.invest.fisheries.fisheries_hst.
execute
(args)¶ Fisheries: Habitat Scenario Tool.
The Fisheries Habitat Scenario Tool generates a new Population Parameters CSV File with modified survival attributes across classes and regions based on habitat area changes and class-level dependencies on those habitats.
- args[‘workspace_dir’] (str): location into which the resultant
- modified Population Parameters CSV file should be placed.
- args[‘sexsp’] (str): specifies whether or not the age and stage
- classes are distinguished by sex. Options: ‘Yes’ or ‘No’
- args[‘population_csv_path’] (str): location of the population
- parameters csv file. This file contains all age and stage specific parameters.
- args[‘habitat_chg_csv_path’] (str): location of the habitat change
- parameters csv file. This file contains habitat area change information.
- args[‘habitat_dep_csv_path’] (str): location of the habitat dependency
- parameters csv file. This file contains habitat-class dependency information.
- args[‘gamma’] (float): describes the relationship between a change
- in habitat area and a change in survival of life stages dependent on that habitat
Returns: None Example Args:
args = { 'workspace_dir': 'path/to/workspace_dir/', 'sexsp': 'Yes', 'population_csv_path': 'path/to/csv', 'habitat_chg_csv_path': 'path/to/csv', 'habitat_dep_csv_path': 'path/to/csv', 'gamma': 0.5, }
Note
- Modified Population Parameters CSV File saved to ‘workspace_dir/output/’
Forest Carbon Edge Effect¶
-
natcap.invest.forest_carbon_edge_effect.
execute
(args)¶ Forest Carbon Edge Effect.
InVEST Carbon Edge Model calculates the carbon due to edge effects in tropical forest pixels.
Parameters: - args['workspace_dir'] (string) – a path to the directory that will write output and other temporary files during calculation. (required)
- args['results_suffix'] (string) – a string to append to any output file name (optional)
- args['n_nearest_model_points'] (int) – number of nearest neighbor model points to search for
- args['aoi_vector_path'] (string) – (optional) if present, a path to a shapefile that will be used to aggregate carbon stock results at the end of the run.
- args['biophysical_table_path'] (string) –
a path to a CSV table that has at least the fields ‘lucode’ and ‘c_above’. If
args['compute_forest_edge_effects'] == True
, table must also contain an ‘is_tropical_forest’ field. Ifargs['pools_to_calculate'] == 'all'
, this table must contain the fields ‘c_below’, ‘c_dead’, and ‘c_soil’.lucode
: an integer that corresponds to landcover codes in the rasterargs['lulc_raster_path']
is_tropical_forest
: either 0 or 1 indicating whether the landcover type is forest (1) or not (0). If 1, the value inc_above
is ignored and instead calculated from the edge regression model.c_above
: floating point number indicating tons of above ground carbon per hectare for that landcover type{'c_below', 'c_dead', 'c_soil'}
: three other optional carbon pools that will statically map landcover types to the carbon densities in the table.
Example:
lucode,is_tropical_forest,c_above,c_soil,c_dead,c_below 0,0,32.8,5,5.2,2.1 1,1,n/a,2.5,0.0,0.0 2,1,n/a,1.8,1.0,0.0 16,0,28.1,4.3,0.0,2.0
Note the “n/a” in
c_above
are optional since that field is ignored whenis_tropical_forest==1
. - args['lulc_raster_path'] (string) – path to a integer landcover code raster
- args['pools_to_calculate'] (string) – if “all” then all carbon pools will be calculted. If any other value only above ground carbon pools will be calculated and expect only a ‘c_above’ header in the biophysical table. If “all” model expects ‘c_above’, ‘c_below’, ‘c_dead’, ‘c_soil’ in header of biophysical_table and will make a translated carbon map for each based off the landcover map.
- args['compute_forest_edge_effects'] (boolean) – if True, requires biophysical table to have ‘is_tropical_forest’ forest field, and any landcover codes that have a 1 in this column calculate carbon stocks using the Chaplin-Kramer et. al method and ignore ‘c_above’.
- args['tropical_forest_edge_carbon_model_vector_path'] (string) –
path to a shapefile that defines the regions for the local carbon edge models. Has at least the fields ‘method’, ‘theta1’, ‘theta2’, ‘theta3’. Where ‘method’ is an int between 1..3 describing the biomass regression model, and the thetas are floating point numbers that have different meanings depending on the ‘method’ parameter. Specifically,
- method 1 (asymptotic model):
biomass = theta1 - theta2 * exp(-theta3 * edge_dist_km)
- method 2 (logarithmic model):
# NOTE: theta3 is ignored for this method biomass = theta1 + theta2 * numpy.log(edge_dist_km)
- method 3 (linear regression):
biomass = theta1 + theta2 * edge_dist_km
- method 1 (asymptotic model):
- args['biomass_to_carbon_conversion_factor'] (string/float) – Number by which to multiply forest biomass to convert to carbon in the edge effect calculation.
- args['n_workers'] (int) – (optional) The number of worker processes to use for processing this model. If omitted, computation will take place in the current process.
Returns: None
GLOBIO¶
-
natcap.invest.globio.
execute
(args)¶ GLOBIO.
The model operates in two modes. Mode (a) generates a landcover map based on a base landcover map and information about crop yields, infrastructure, and more. Mode (b) assumes the globio landcover map is generated. These modes are used below to describe input parameters.
Parameters: - args['workspace_dir'] (string) – output directory for intermediate, temporary, and final files
- args['predefined_globio'] (boolean) – if True then “mode (b)” else “mode (a)”
- args['results_suffix'] (string) – (optional) string to append to any output files
- args['lulc_path'] (string) – used in “mode (a)” path to a base landcover map with integer codes
- args['lulc_to_globio_table_path'] (string) –
used in “mode (a)” path to table that translates the land-cover args[‘lulc_path’] to intermediate GLOBIO classes, from which they will be further differentiated using the additional data in the model. Contains at least the following fields:
- ’lucode’: Land use and land cover class code of the dataset used. LULC codes match the ‘values’ column in the LULC raster of mode (b) and must be numeric and unique.
- ’globio_lucode’: The LULC code corresponding to the GLOBIO class to which it should be converted, using intermediate codes described in the example below.
- args['infrastructure_dir'] (string) – used in “mode (a) and (b)” a path to a folder containing maps of either gdal compatible rasters or OGR compatible shapefiles. These data will be used in the infrastructure to calculation of MSA.
- args['pasture_path'] (string) – used in “mode (a)” path to pasture raster
- args['potential_vegetation_path'] (string) – used in “mode (a)” path to potential vegetation raster
- args['pasture_threshold'] (float) – used in “mode (a)”
- args['intensification_fraction'] (float) – used in “mode (a)”; a value between 0 and 1 denoting proportion of total agriculture that should be classified as ‘high input’
- args['primary_threshold'] (float) – used in “mode (a)”
- args['msa_parameters_path'] (string) – path to MSA classification parameters
- args['aoi_path'] (string) – (optional) if it exists then final MSA raster is summarized by AOI
- args['globio_lulc_path'] (string) – used in “mode (b)” path to predefined globio raster.
- args['n_workers'] (int) – (optional) The number of worker processes to use for processing this model. If omitted, computation will take place in the current process.
Returns: None
Habitat Quality¶
-
natcap.invest.habitat_quality.
execute
(args)¶ Habitat Quality.
Open files necessary for the portion of the habitat_quality model.
Parameters: - workspace_dir (string) – a path to the directory that will write output and other temporary files (required)
- lulc_cur_path (string) – a path to an input land use/land cover raster (required)
- lulc_fut_path (string) – a path to an input land use/land cover raster (optional)
- lulc_bas_path (string) – a path to an input land use/land cover raster (optional, but required for rarity calculations)
- threat_folder (string) – a path to the directory that will contain all threat rasters (required)
- threats_table_path (string) – a path to an input CSV containing data of all the considered threats. Each row is a degradation source and each column a different attribute of the source with the following names: ‘THREAT’,’MAX_DIST’,’WEIGHT’ (required).
- access_vector_path (string) – a path to an input polygon shapefile containing data on the relative protection against threats (optional)
- sensitivity_table_path (string) – a path to an input CSV file of LULC types, whether they are considered habitat, and their sensitivity to each threat (required)
- half_saturation_constant (float) – a python float that determines the spread and central tendency of habitat quality scores (required)
- suffix (string) – a python string that will be inserted into all raster path paths just before the file extension.
Example Args Dictionary:
{ 'workspace_dir': 'path/to/workspace_dir', 'lulc_cur_path': 'path/to/lulc_cur_raster', 'lulc_fut_path': 'path/to/lulc_fut_raster', 'lulc_bas_path': 'path/to/lulc_bas_raster', 'threat_raster_folder': 'path/to/threat_rasters/', 'threats_table_path': 'path/to/threats_csv', 'access_vector_path': 'path/to/access_shapefile', 'sensitivity_table_path': 'path/to/sensitivity_csv', 'half_saturation_constant': 0.5, 'suffix': '_results', }
Returns: None
InVEST Carbon Model¶
-
natcap.invest.carbon.
execute
(args)¶ InVEST Carbon Model.
Calculate the amount of carbon stocks given a landscape, or the difference due to a future change, and/or the tradeoffs between that and a REDD scenario, and calculate economic valuation on those scenarios.
The model can operate on a single scenario, a combined present and future scenario, as well as an additional REDD scenario.
Parameters: - args['workspace_dir'] (string) – a path to the directory that will write output and other temporary files during calculation.
- args['results_suffix'] (string) – appended to any output file name.
- args['lulc_cur_path'] (string) – a path to a raster representing the current carbon stocks.
- args['calc_sequestration'] (bool) – if true, sequestration should be calculated and ‘lulc_fut_path’ and ‘do_redd’ should be defined.
- args['lulc_fut_path'] (string) – a path to a raster representing future landcover scenario. Optional, but if present and well defined will trigger a sequestration calculation.
- args['do_redd'] (bool) – if true, REDD analysis should be calculated and ‘lulc_redd_path’ should be defined
- args['lulc_redd_path'] (string) – a path to a raster representing the alternative REDD scenario which is only possible if the args[‘lulc_fut_path’] is present and well defined.
- args['carbon_pools_path'] (string) – path to CSV or that indexes carbon storage density to lulc codes. (required if ‘do_uncertainty’ is false)
- args['lulc_cur_year'] (int/string) – an integer representing the year of args[‘lulc_cur_path’] used if args[‘calc_sequestration’] is True.
- args['lulc_fut_year'] (int/string) – an integer representing the year of args[‘lulc_fut_path’] used in valuation if it exists. Required if args[‘do_valuation’] is True and args[‘lulc_fut_path’] is present and well defined.
- args['do_valuation'] (bool) – if true then run the valuation model on available outputs. At a minimum will run on carbon stocks, if sequestration with a future scenario is done and/or a REDD scenario calculate NPV for either and report in final HTML document.
- args['price_per_metric_ton_of_c'] (float) – Is the present value of carbon per metric ton. Used if args[‘do_valuation’] is present and True.
- args['discount_rate'] (float) – Discount rate used if NPV calculations are required. Used if args[‘do_valuation’] is present and True.
- args['rate_change'] (float) – Annual rate of change in price of carbon as a percentage. Used if args[‘do_valuation’] is present and True.
- args['n_workers'] (int) – (optional) The number of worker processes to use for processing this model. If omitted, computation will take place in the current process.
Returns: None.
InVEST Habitat Risk Assessment (HRA) Model¶
-
natcap.invest.hra.
execute
(args)¶ InVEST Habitat Risk Assessment (HRA) Model.
Parameters: - args['workspace_dir'] (str) – a path to the output workspace folder. It will overwrite any files that exist if the path already exists.
- args['results_suffix'] (str) – a string appended to each output file path. (optional)
- args['info_table_path'] (str) – a path to the CSV or Excel file that
contains the name of the habitat (H) or stressor (s) on the
NAME
column that matches the names in criteria_table_path. Each H/S has its corresponding vector or raster path on thePATH
column. TheSTRESSOR BUFFER (meters)
column should have a buffer value if theTYPE
column is a stressor. - args['criteria_table_path'] (str) – a path to the CSV or Excel file that contains the set of criteria ranking of each stressor on each habitat.
- args['resolution'] (int) – a number representing the desired pixel dimensions of output rasters in meters.
- args['max_rating'] (str, int or float) – a number representing the highest potential value that should be represented in rating in the criteria scores table.
- args['risk_eq'] (str) – a string identifying the equation that should be used in calculating risk scores for each H-S overlap cell. This will be either ‘Euclidean’ or ‘Multiplicative’.
- args['decay_eq'] (str) – a string identifying the equation that should be used in calculating the decay of stressor buffer influence. This can be ‘None’, ‘Linear’, or ‘Exponential’.
- args['aoi_vector_path'] (str) – a path to the shapefile containing one or more planning regions used to get the average risk value for each habitat-stressor combination over each area. Optionally, if each of the shapefile features contain a ‘name’ field, it will be used as a way of identifying each individual shape.
- args['n_workers'] (int) – the number of worker processes to use for processing this model. If omitted, computation will take place in the current process. (optional)
- args['visualize_outputs'] (bool) – if True, create output GeoJSONs and save them in a visualization_outputs folder, so users can visualize results on the web app. Default to True if not specified. (optional)
Returns: None.
InVEST Pollination Model¶
-
natcap.invest.pollination.
execute
(args)¶ InVEST Pollination Model.
Parameters: - args['workspace_dir'] (string) – a path to the output workspace folder. Will overwrite any files that exist if the path already exists.
- args['results_suffix'] (string) – string appended to each output file path.
- args['landcover_raster_path'] (string) – file path to a landcover raster.
- args['guild_table_path'] (string) –
file path to a table indicating the bee species to analyze in this model run. Table headers must include:
- ’species’: a bee species whose column string names will
- be referred to in other tables and the model will output analyses per species.
- one or more columns matching _NESTING_SUITABILITY_PATTERN
- with values in the range [0.0, 1.0] indicating the suitability of the given species to nest in a particular substrate.
- one or more columns matching _FORAGING_ACTIVITY_RE_PATTERN
- with values in the range [0.0, 1.0] indicating the relative level of foraging activity for that species during a particular season.
- _ALPHA_HEADER the sigma average flight distance of that bee
- species in meters.
- ’relative_abundance’: a weight indicating the relative
- abundance of the particular species with respect to the sum of all relative abundance weights in the table.
- args['landcover_biophysical_table_path'] (string) –
path to a table mapping landcover codes in args[‘landcover_path’] to indexes of nesting availability for each nesting substrate referenced in guilds table as well as indexes of abundance of floral resources on that landcover type per season in the bee activity columns of the guild table.
All indexes are in the range [0.0, 1.0].
- Columns in the table must be at least
- ’lucode’: representing all the unique landcover codes in
- the raster ast args[‘landcover_path’]
- For every nesting matching _NESTING_SUITABILITY_PATTERN in the guild stable, a column matching the pattern in _LANDCOVER_NESTING_INDEX_HEADER.
- For every season matching _FORAGING_ACTIVITY_RE_PATTERN in the guilds table, a column matching the pattern in _LANDCOVER_FLORAL_RESOURCES_INDEX_HEADER.
- args['farm_vector_path'] (string) –
(optional) path to a single layer polygon shapefile representing farms. If present will trigger the farm yield component of the model.
The layer must have at least the following fields:
- season (string): season in which the farm needs pollination
- crop_type (string): a text field to identify the crop type for
- summary statistics.
- half_sat (float): a real in the range [0.0, 1.0] representing
- the proportion of wild pollinators to achieve a 50% yield of that crop.
- p_dep (float): a number in the range [0.0, 1.0]
- representing the proportion of yield dependent on pollinators.
- p_managed (float): proportion of pollinators that come from
- non-native/managed hives.
- fr_[season] (float): one or more fields that match this pattern
- such that season also matches the season headers in the biophysical and guild table. Any areas that overlap the landcover map will replace seasonal floral resources with this value. Ranges from 0..1.
- n_[substrate] (float): One or more fields that match this
- pattern such that substrate also matches the nesting substrate headers in the biophysical and guild table. Any areas that overlap the landcover map will replace nesting substrate suitability with this value. Ranges from 0..1.
- args['n_workers'] (int) – (optional) The number of worker processes to use for processing this model. If omitted, computation will take place in the current process.
Returns: None
Recreation¶
-
natcap.invest.recreation.recmodel_client.
execute
(args)¶ Recreation.
Execute recreation client model on remote server.
Parameters: - args['workspace_dir'] (string) – path to workspace directory
- args['aoi_path'] (string) – path to AOI vector
- args['hostname'] (string) – FQDN to recreation server
- args['port'] (string or int) – port on hostname for recreation server
- args['start_year'] (string) – start year in form YYYY. This year is the inclusive lower bound to consider points in the PUD and regression.
- args['end_year'] (string) – end year in form YYYY. This year is the inclusive upper bound to consider points in the PUD and regression.
- args['grid_aoi'] (boolean) – if true the polygon vector in args[‘aoi_path’] should be gridded into a new vector and the recreation model should be executed on that
- args['grid_type'] (string) – optional, but must exist if args[‘grid_aoi’] is True. Is one of ‘hexagon’ or ‘square’ and indicates the style of gridding.
- args['cell_size'] (string/float) – optional, but must exist if args[‘grid_aoi’] is True. Indicates the cell size of square pixels and the width of the horizontal axis for the hexagonal cells.
- args['compute_regression'] (boolean) – if True, then process the predictor table and scenario table (if present).
- args['predictor_table_path'] (string) –
required if args[‘compute_regression’] is True. Path to a table that describes the regression predictors, their IDs and types. Must contain the fields ‘id’, ‘path’, and ‘type’ where:
- ’id’: is a <=10 character length ID that is used to uniquely describe the predictor. It will be added to the output result shapefile attribute table which is an ESRI Shapefile, thus limited to 10 characters.
- ’path’: an absolute or relative (to this table) path to the predictor dataset, either a vector or raster type.
- ’type’: one of the following,
- ’raster_mean’: mean of values in the raster under the response polygon
- ’raster_sum’: sum of values in the raster under the response polygon
- ’point_count’: count of the points contained in the response polygon
- ’point_nearest_distance’: distance to the nearest point from the response polygon
- ’line_intersect_length’: length of lines that intersect with the response polygon in projected units of AOI
- ’polygon_area’: area of the polygon contained within response polygon in projected units of AOI
- args['scenario_predictor_table_path'] (string) – (optional) if present runs the scenario mode of the recreation model with the datasets described in the table on this path. Field headers are identical to args[‘predictor_table_path’] and ids in the table are required to be identical to the predictor list.
- args['results_suffix'] (string) – optional, if exists is appended to any output file paths.
Returns: None
RouteDEM: Hydrological routing¶
-
natcap.invest.routedem.
execute
(args)¶ RouteDEM: Hydrological routing.
This model exposes the pygeoprocessing D8 and Multiple Flow Direction routing functionality as an InVEST model.
This tool will always fill pits on the input DEM.
Parameters: - args['workspace_dir'] (string) – output directory for intermediate, temporary, and final files
- args['results_suffix'] (string) – (optional) string to append to any output file names
- args['dem_path'] (string) – path to a digital elevation raster
- args['dem_band_index'] (int) – Optional. The band index to operate on. If not provided, band index 1 is assumed.
- args['algorithm'] (string) – The routing algorithm to use. Must be one of ‘D8’ or ‘MFD’ (case-insensitive). Required when calculating flow direction, flow accumulation, stream threshold, and downstream distance.
- args['calculate_flow_direction'] (bool) – If True, model will calculate flow direction for the filled DEM.
- args['calculate_flow_accumulation'] (bool) – If True, model will calculate a flow accumulation raster. Only applies when args[‘calculate_flow_direction’] is True.
- args['calculate_stream_threshold'] (bool) – if True, model will
calculate a stream classification layer by thresholding flow
accumulation to the provided value in
args['threshold_flow_accumulation']
. Only applies when args[‘calculate_flow_accumulation’] and args[‘calculate_flow_direction’] are True. - args['threshold_flow_accumulation'] (int) – The number of upstream cells that must flow into a cell before it’s classified as a stream.
- args['calculate_downstream_distance'] (bool) – If True, and a stream threshold is calculated, model will calculate a downstream distance raster in units of pixels. Only applies when args[‘calculate_flow_accumulation’], args[‘calculate_flow_direction’], and args[‘calculate_stream_threshold’] are all True.
- args['calculate_slope'] (bool) – If True, model will calculate a slope raster from the DEM.
- args['n_workers'] (int) – The
n_workers
parameter to pass to the task graph. The default is-1
if not provided.
Returns: None
Scenario Generator: Proximity-Based¶
-
natcap.invest.scenario_gen_proximity.
execute
(args)¶ Scenario Generator: Proximity-Based.
Main entry point for proximity based scenario generator model.
Parameters: - args['workspace_dir'] (string) – output directory for intermediate, temporary, and final files
- args['results_suffix'] (string) – (optional) string to append to any output files
- args['base_lulc_path'] (string) – path to the base landcover map
- args['replacment_lucode'] (string or int) – code to replace when converting pixels
- args['area_to_convert'] (string or float) – max area (Ha) to convert
- args['focal_landcover_codes'] (string) – a space separated string of landcover codes that are used to determine the proximity when refering to “towards” or “away” from the base landcover codes
- args['convertible_landcover_codes'] (string) – a space separated string of landcover codes that can be converted in the generation phase found in args[‘base_lulc_path’].
- args['n_fragmentation_steps'] (string) – an int as a string indicating the number of steps to take for the fragmentation conversion
- args['aoi_path'] (string) – (optional) path to a shapefile that indicates area of interest. If present, the expansion scenario operates only under that AOI and the output raster is clipped to that shape.
- args['convert_farthest_from_edge'] (boolean) – if True will run the conversion simulation starting from the furthest pixel from the edge and work inwards. Workspace will contain output files named ‘toward_base{suffix}.{tif,csv}.
- args['convert_nearest_to_edge'] (boolean) – if True will run the conversion simulation starting from the nearest pixel on the edge and work inwards. Workspace will contain output files named ‘toward_base{suffix}.{tif,csv}.
- args['n_workers'] (int) – (optional) The number of worker processes to use for processing this model. If omitted, computation will take place in the current process.
Returns: None.
Sediment Delivery Ratio¶
-
natcap.invest.sdr.
execute
(args)¶ Sediment Delivery Ratio.
This function calculates the sediment export and retention of a landscape using the sediment delivery ratio model described in the InVEST user’s guide.
Parameters: - args['workspace_dir'] (string) – output directory for intermediate, temporary, and final files
- args['results_suffix'] (string) – (optional) string to append to any output file names
- args['dem_path'] (string) – path to a digital elevation raster
- args['erosivity_path'] (string) – path to rainfall erosivity index raster
- args['erodibility_path'] (string) – a path to soil erodibility raster
- args['lulc_path'] (string) – path to land use/land cover raster
- args['watersheds_path'] (string) – path to vector of the watersheds
- args['biophysical_table_path'] (string) – path to CSV file with biophysical information of each land use classes. contain the fields ‘usle_c’ and ‘usle_p’
- args['threshold_flow_accumulation'] (number) – number of upstream pixels on the dem to threshold to a stream.
- args['k_param'] (number) – k calibration parameter
- args['sdr_max'] (number) – max value the SDR
- args['ic_0_param'] (number) – ic_0 calibration parameter
- args['drainage_path'] (string) – (optional) path to drainage raster that is used to add additional drainage areas to the internally calculated stream layer
- args['n_workers'] (int) – if present, indicates how many worker processes should be used in parallel processing. -1 indicates single process mode, 0 is single process but non-blocking mode, and >= 1 is number of processes.
Returns: None.
Wave Energy¶
-
natcap.invest.wave_energy.
execute
(args)¶ Wave Energy.
Executes both the biophysical and valuation parts of the wave energy model (WEM). Files will be written on disk to the intermediate and output directories. The outputs computed for biophysical and valuation include: wave energy capacity raster, wave power raster, net present value raster, percentile rasters for the previous three, and a point shapefile of the wave points with attributes.
Parameters: - workspace_dir (str) – Where the intermediate and output folder/files will be saved. (required)
- wave_base_data_path (str) – Directory location of wave base data including WAVEWATCH III (WW3) data and analysis area shapefile. (required)
- analysis_area_path (str) – A string identifying the analysis area of interest. Used to determine wave data shapefile, wave data text file, and analysis area boundary shape. (required)
- aoi_path (str) – A polygon OGR vector outlining a more detailed area within the analysis area. This vector should be projected with linear units being in meters. (required to run Valuation model)
- machine_perf_path (str) – The path of a CSV file that holds the machine performance table. (required)
- machine_param_path (str) – The path of a CSV file that holds the machine parameter table. (required)
- dem_path (str) – The path of the Global Digital Elevation Model (DEM). (required)
- suffix (str) – A python string of characters to append to each output filename (optional)
- valuation_container (boolean) – Indicates whether the model includes valuation
- land_gridPts_path (str) – A CSV file path containing the Landing and Power Grid Connection Points table. (required for Valuation)
- machine_econ_path (str) – A CSV file path for the machine economic parameters table. (required for Valuation)
- number_of_machines (int) – An integer specifying the number of machines for a wave farm site. (required for Valuation)
- n_workers (int) – The number of worker processes to use for processing this model. If omitted, computation will take place in the current process. (optional)
Example Args Dictionary:
{ 'workspace_dir': 'path/to/workspace_dir', 'wave_base_data_path': 'path/to/base_data_dir', 'analysis_area_path': 'West Coast of North America and Hawaii', 'aoi_path': 'path/to/vector', 'machine_perf_path': 'path/to/csv', 'machine_param_path': 'path/to/csv', 'dem_path': 'path/to/raster', 'suffix': '_results', 'valuation_container': True, 'land_gridPts_path': 'path/to/csv', 'machine_econ_path': 'path/to/csv', 'number_of_machines': 28, }
Wind Energy¶
-
natcap.invest.wind_energy.
execute
(args)¶ Wind Energy.
This module handles the execution of the wind energy model given the following dictionary:
Parameters: - workspace_dir (str) – a path to the output workspace folder (required)
- wind_data_path (str) – path to a CSV file with the following header: [‘LONG’,’LATI’,’LAM’, ‘K’, ‘REF’]. Each following row is a location with at least the Longitude, Latitude, Scale (‘LAM’), Shape (‘K’), and reference height (‘REF’) at which the data was collected (required)
- aoi_vector_path (str) – a path to an OGR polygon vector that is projected in linear units of meters. The polygon specifies the area of interest for the wind data points. If limiting the wind farm bins by distance, then the aoi should also cover a portion of the land polygon that is of interest (optional for biophysical and no distance masking, required for biophysical and distance masking, required for valuation)
- bathymetry_path (str) – a path to a GDAL raster that has the depth values of the area of interest (required)
- land_polygon_vector_path (str) – a path to an OGR polygon vector that provides a coastline for determining distances from wind farm bins. Enabled by AOI and required if wanting to mask by distances or run valuation
- global_wind_parameters_path (str) – a float for the average distance in kilometers from a grid connection point to a land connection point (required for valuation if grid connection points are not provided)
- suffix (str) – a str to append to the end of the output files (optional)
- turbine_parameters_path (str) – a path to a CSV file that holds the turbines biophysical parameters as well as valuation parameters (required)
- number_of_turbines (int) – an integer value for the number of machines for the wind farm (required for valuation)
- min_depth (float) – a float value for the minimum depth for offshore wind farm installation (meters) (required)
- max_depth (float) – a float value for the maximum depth for offshore wind farm installation (meters) (required)
- min_distance (float) – a float value for the minimum distance from shore for offshore wind farm installation (meters) The land polygon must be selected for this input to be active (optional, required for valuation)
- max_distance (float) – a float value for the maximum distance from shore for offshore wind farm installation (meters) The land polygon must be selected for this input to be active (optional, required for valuation)
- valuation_container (boolean) – Indicates whether model includes valuation
- foundation_cost (float) – a float representing how much the foundation will cost for the specific type of turbine (required for valuation)
- discount_rate (float) – a float value for the discount rate (required for valuation)
- grid_points_path (str) – a path to a CSV file that specifies the landing and grid point locations (optional)
- avg_grid_distance (float) – a float for the average distance in kilometers from a grid connection point to a land connection point (required for valuation if grid connection points are not provided)
- price_table (boolean) – a bool indicating whether to use the wind energy price table or not (required)
- wind_schedule (str) – a path to a CSV file for the yearly prices of wind energy for the lifespan of the farm (required if ‘price_table’ is true)
- wind_price (float) – a float for the wind energy price at year 0 (required if price_table is false)
- rate_change (float) – a float as a percent for the annual rate of change in the price of wind energy. (required if price_table is false)
- n_workers (int) – The number of worker processes to use for processing this model. If omitted, computation will take place in the current process. (optional)
Example Args Dictionary:
{ 'workspace_dir': 'path/to/workspace_dir', 'wind_data_path': 'path/to/file', 'aoi_vector_path': 'path/to/shapefile', 'bathymetry_path': 'path/to/raster', 'land_polygon_vector_path': 'path/to/shapefile', 'global_wind_parameters_path': 'path/to/csv', 'suffix': '_results', 'turbine_parameters_path': 'path/to/csv', 'number_of_turbines': 10, 'min_depth': 3, 'max_depth': 60, 'min_distance': 0, 'max_distance': 200000, 'valuation_container': True, 'foundation_cost': 3.4, 'discount_rate': 7.0, 'grid_points_path': 'path/to/csv', 'avg_grid_distance': 4, 'price_table': True, 'wind_schedule': 'path/to/csv', 'wind_price': 0.4, 'rate_change': 0.0, }
Returns: None
API Reference¶
Note
For the function documentation of available models, see InVEST Model Entry Points.
natcap¶
natcap package¶
Subpackages¶
Coastal Blue Carbon Model.
-
natcap.invest.coastal_blue_carbon.coastal_blue_carbon.
execute
(args)¶ Coastal Blue Carbon.
Parameters: - workspace_dir (str) – location into which all intermediate and output files should be placed.
- results_suffix (str) – a string to append to output filenames.
- lulc_lookup_uri (str) – filepath to a CSV table used to convert the lulc code to a name. Also used to determine if a given lulc type is a coastal blue carbon habitat.
- lulc_transition_matrix_uri (str) – generated by the preprocessor. This file must be edited before it can be used by the main model. The left-most column represents the source lulc class, and the top row represents the destination lulc class.
- carbon_pool_initial_uri (str) – the provided CSV table contains information related to the initial conditions of the carbon stock within each of the three pools of a habitat. Biomass includes carbon stored above and below ground. All non-coastal blue carbon habitat lulc classes are assumed to contain no carbon. The values for ‘biomass’, ‘soil’, and ‘litter’ should be given in terms of Megatonnes CO2 e/ ha.
- carbon_pool_transient_uri (str) – the provided CSV table contains information related to the transition of carbon into and out of coastal blue carbon pools. All non-coastal blue carbon habitat lulc classes are assumed to neither sequester nor emit carbon as a result of change. The ‘yearly_accumulation’ values should be given in terms of Megatonnes of CO2 e/ha-yr. The ‘half-life’ values must be given in terms of years. The ‘disturbance’ values must be given as a decimal (e.g. 0.5 for 50%) of stock distrubed given a transition occurs away from a lulc-class.
- lulc_baseline_map_uri (str) – a GDAL-supported raster representing the baseline landscape/seascape.
- lulc_baseline_year (int) – The year of the baseline snapshot.
- lulc_transition_maps_list (list) – a list of GDAL-supported rasters representing the landscape/seascape at particular points in time. Provided in chronological order.
- lulc_transition_years_list (list) – a list of years that respectively correspond to transition years of the rasters. Provided in chronological order.
- analysis_year (int) – optional. Indicates how many timesteps to run the transient analysis beyond the last transition year. Must come chronologically after the last transition year if provided. Otherwise, the final timestep of the model will be set to the last transition year.
- do_economic_analysis (bool) – boolean value indicating whether model should run economic analysis.
- do_price_table (bool) – boolean value indicating whether a price table is included in the arguments and to be used or a price and interest rate is provided and to be used instead.
- price (float) – the price per Megatonne CO2 e at the base year.
- inflation_rate (float) – the interest rate on the price per Megatonne CO2e, compounded yearly. Provided as a percentage (e.g. 3.0 for 3%).
- price_table_uri (bool) – if args[‘do_price_table’] is set to True the provided CSV table is used in place of the initial price and interest rate inputs. The table contains the price per Megatonne CO2e sequestered for a given year, for all years from the original snapshot to the analysis year, if provided.
- discount_rate (float) – the discount rate on future valuations of sequestered carbon, compounded yearly. Provided as a percentage (e.g. 3.0 for 3%).
Example Args:
args = { 'workspace_dir': 'path/to/workspace/', 'results_suffix': '', 'lulc_lookup_uri': 'path/to/lulc_lookup_uri', 'lulc_transition_matrix_uri': 'path/to/lulc_transition_uri', 'carbon_pool_initial_uri': 'path/to/carbon_pool_initial_uri', 'carbon_pool_transient_uri': 'path/to/carbon_pool_transient_uri', 'lulc_baseline_map_uri': 'path/to/baseline_map.tif', 'lulc_baseline_year': <int>, 'lulc_transition_maps_list': [raster1_uri, raster2_uri, ...], 'lulc_transition_years_list': [2000, 2005, ...], 'analysis_year': 2100, 'do_economic_analysis': '<boolean>', 'do_price_table': '<boolean>', 'price': '<float>', 'inflation_rate': '<float>', 'price_table_uri': 'path/to/price_table', 'discount_rate': '<float>' }
-
natcap.invest.coastal_blue_carbon.coastal_blue_carbon.
get_inputs
(args)¶ Get Inputs.
Parameters: - workspace_dir (str) – workspace directory
- results_suffix (str) – optional suffix appended to results
- lulc_lookup_uri (str) – lulc lookup table filepath
- lulc_transition_matrix_uri (str) – lulc transition table filepath
- carbon_pool_initial_uri (str) – initial conditions table filepath
- carbon_pool_transient_uri (str) – transient conditions table filepath
- lulc_baseline_map_uri (str) – baseline map filepath
- lulc_transition_maps_list (list) – ordered list of transition map filepaths
- lulc_transition_years_list (list) – ordered list of transition years
- analysis_year (int) – optional final year to extend the analysis beyond the last transition year
- do_economic_analysis (bool) – whether to run economic component of the analysis
- do_price_table (bool) – whether to use the price table for the economic component of the analysis
- price (float) – the price of net sequestered carbon
- inflation_rate (float) – the interest rate on the price of carbon
- price_table_uri (str) – price table filepath
- discount_rate (float) – the discount rate on future valuations of carbon
Returns: data dictionary.
Return type: d (dict)
- Example Returns:
- d = {
- ‘do_economic_analysis’: <bool>, ‘lulc_to_Sb’: <dict>, ‘lulc_to_Ss’: <dict> ‘lulc_to_L’: <dict>, ‘lulc_to_Yb’: <dict>, ‘lulc_to_Ys’: <dict>, ‘lulc_to_Hb’: <dict>, ‘lulc_to_Hs’: <dict>, ‘lulc_trans_to_Db’: <dict>, ‘lulc_trans_to_Ds’: <dict>, ‘C_r_rasters’: <list>, ‘transition_years’: <list>, ‘snapshot_years’: <list>, ‘timesteps’: <int>, ‘transitions’: <list>, ‘price_t’: <list>, ‘File_Registry’: <dict>
}
-
natcap.invest.coastal_blue_carbon.coastal_blue_carbon.
get_num_blocks
(raster_uri)¶ Get the number of blocks in a raster file.
Parameters: raster_uri (str) – filepath to raster Returns: number of blocks in raster Return type: num_blocks (int)
-
natcap.invest.coastal_blue_carbon.coastal_blue_carbon.
is_transition_year
(snapshot_years, transitions, timestep)¶ Check whether given timestep is a transition year.
Parameters: - snapshot_years (list) – list of snapshot years.
- transitions (int) – number of transitions.
- timestep (int) – current timestep.
Returns: - whether the year corresponding to the
timestep is a transition year.
Return type: is_transition_year (bool)
-
natcap.invest.coastal_blue_carbon.coastal_blue_carbon.
read_from_raster
(input_raster, offset_block)¶ Read numpy array from raster block.
Parameters: - input_raster (str) – filepath to input raster
- offset_block (dict) – dictionary of offset information
Returns: a blocked array of the input raster
Return type: array (numpy.array)
-
natcap.invest.coastal_blue_carbon.coastal_blue_carbon.
reclass
(array, d, out_dtype=None, nodata_mask=None)¶ Reclassify values in array.
If a nodata value is not provided, the function will return an array with NaN values in its place to mark cells that could not be reclassed.
Parameters: - array (numpy.array) – input data
- d (dict) – reclassification map
- out_dtype (numpy.dtype) – a numpy datatype for the reclass_array
- nodata_mask (number) – for floats, a nodata value that is set to numpy.nan if provided to make reclass_array nodata values consistent
Returns: reclassified array
Return type: reclass_array (numpy.array)
-
natcap.invest.coastal_blue_carbon.coastal_blue_carbon.
reclass_transition
(a_prev, a_next, trans_dict, out_dtype=None, nodata_mask=None)¶ Reclass arrays based on element-wise combinations between two arrays.
Parameters: - a_prev (numpy.array) – previous lulc array
- a_next (numpy.array) – next lulc array
- trans_dict (dict) – reclassification map
- out_dtype (numpy.dtype) – a numpy datatype for the reclass_array
- nodata_mask (number) – for floats, a nodata value that is set to numpy.nan if provided to make reclass_array nodata values consistent
Returns: reclassified array
Return type: reclass_array (numpy.array)
-
natcap.invest.coastal_blue_carbon.coastal_blue_carbon.
s_to_timestep
(snapshot_years, snapshot_idx)¶ Convert snapshot index position to timestep.
Parameters: - snapshot_years (list) – list of snapshot years.
- snapshot_idx (int) – index of snapshot
Returns: timestep of the snapshot
Return type: snapshot_timestep (int)
-
natcap.invest.coastal_blue_carbon.coastal_blue_carbon.
timestep_to_transition_idx
(snapshot_years, transitions, timestep)¶ Convert timestep to transition index.
Parameters: - snapshot_years (list) – a list of years corresponding to the provided rasters
- transitions (int) – the number of transitions in the scenario
- timestep (int) – the current timestep
Returns: the current transition
Return type: transition_idx (int)
-
natcap.invest.coastal_blue_carbon.coastal_blue_carbon.
write_to_raster
(output_raster, array, xoff, yoff)¶ Write numpy array to raster block.
Parameters: - output_raster (str) – filepath to output raster
- array (numpy.array) – block to save to raster
- xoff (int) – offset index for x-dimension
- yoff (int) – offset index for y-dimension
Coastal Blue Carbon Preprocessor.
-
natcap.invest.coastal_blue_carbon.preprocessor.
execute
(args)¶ Coastal Blue Carbon Preprocessor.
The preprocessor accepts a list of rasters and checks for cell-transitions across the rasters. The preprocessor outputs a CSV file representing a matrix of land cover transitions, each cell prefilled with a string indicating whether carbon accumulates or is disturbed as a result of the transition, if a transition occurs.
Parameters: - workspace_dir (string) – directory path to workspace
- results_suffix (string) – append to outputs directory name if provided
- lulc_lookup_uri (string) – filepath of lulc lookup table
- lulc_snapshot_list (list) – a list of filepaths to lulc rasters
Example Args:
args = { 'workspace_dir': 'path/to/workspace_dir/', 'results_suffix': '', 'lulc_lookup_uri': 'path/to/lookup.csv', 'lulc_snapshot_list': ['path/to/raster1', 'path/to/raster2', ...] }
-
natcap.invest.coastal_blue_carbon.preprocessor.
read_from_raster
(input_raster, offset_block)¶ Read block from raster.
Parameters: - input_raster (str) – filepath to raster.
- offset_block (dict) – where the block is indexed.
Returns: the raster block.
Return type: a (np.array)
Coastal Blue Carbon package.
inVEST finfish aquaculture filehandler for biophysical and valuation data
-
natcap.invest.finfish_aquaculture.finfish_aquaculture.
execute
(args)¶ Finfish Aquaculture.
This function will take care of preparing files passed into the finfish aquaculture model. It will handle all files/inputs associated with biophysical and valuation calculations and manipulations. It will create objects to be passed to the aquaculture_core.py module. It may write log, warning, or error messages to stdout.
Parameters: - workspace_dir (string) – The directory in which to place all result files.
- ff_farm_loc (string) – URI that points to a shape file of fishery locations
- farm_ID (string) – column heading used to describe individual farms. Used to link GIS location data to later inputs.
- g_param_a (float) – Growth parameter alpha, used in modeling fish growth, should be an int or float.
- g_param_b (float) – Growth parameter beta, used in modeling fish growth, should be an int or float.
- g_param_tau (float) – Growth parameter tau, used in modeling fish growth, should be an int or float
- use_uncertainty (boolean) –
- g_param_a_sd (float) – (description)
- g_param_b_sd (float) – (description)
- num_monte_carlo_runs (int) –
- water_temp_tbl (string) – URI to a CSV table where daily water temperature values are stored from one year
- farm_op_tbl (string) – URI to CSV table of static variables for calculations
- outplant_buffer (int) – This value will allow the outplanting start day to be flexible plus or minus the number of days specified here.
- do_valuation (boolean) – Boolean that indicates whether or not valuation should be performed on the aquaculture model
- p_per_kg (float) – Market price per kilogram of processed fish
- frac_p (float) – Fraction of market price that accounts for costs rather than profit
- discount (float) – Daily market discount rate
Example Args Dictionary:
{ 'workspace_dir': 'path/to/workspace_dir', 'ff_farm_loc': 'path/to/shapefile', 'farm_ID': 'FarmID' 'g_param_a': 0.038, 'g_param_b': 0.6667, 'g_param_tau': 0.08, 'use_uncertainty': True, 'g_param_a_sd': 0.005, 'g_param_b_sd': 0.05, 'num_monte_carlo_runs': 1000, 'water_temp_tbl': 'path/to/water_temp_tbl', 'farm_op_tbl': 'path/to/farm_op_tbl', 'outplant_buffer': 3, 'do_valuation': True, 'p_per_kg': 2.25, 'frac_p': 0.3, 'discount': 0.000192, }
-
natcap.invest.finfish_aquaculture.finfish_aquaculture.
format_ops_table
(op_path, farm_ID, ff_aqua_args)¶ Takes in the path to the operating parameters table as well as the keyword to look for to identify the farm number to go with the parameters, and outputs a 2D dictionary that contains all parameters by farm and description. The outer key is farm number, and the inner key is a string description of the parameter.
- Input:
op_path: URI to CSV table of static variables for calculations farm_ID: The string to look for in order to identify the column in
which the farm numbers are stored. That column data will become the keys for the dictionary output.- ff_aqua_args: Dictionary of arguments being created in order to be
- passed to the aquaculture core function.
- Output:
- ff_aqua_args[‘farm_op_dict’]: A dictionary that is built up to store
- the static parameters for the aquaculture model run. This is a 2D dictionary, where the outer key is the farm ID number, and the inner keys are strings of parameter names.
Returns nothing.
-
natcap.invest.finfish_aquaculture.finfish_aquaculture.
format_temp_table
(temp_path, ff_aqua_args)¶ This function is doing much the same thing as format_ops_table- it takes in information from a temperature table, and is formatting it into a 2D dictionary as an output.
- Input:
- temp_path: URI to a CSV file containing temperature data for 365 days
- for the farms on which we will look at growth cycles.
- ff_aqua_args: Dictionary of arguments that we are building up in order
- to pass it to the aquaculture core module.
- Output:
- ff_aqua_args[‘water_temp_dict’]: A 2D dictionary containing temperature
- data for 365 days. The outer keys are days of the year from 0 to 364 (we need to be able to check the day modulo 365) which we manually shift down by 1, and the inner keys are farm ID numbers.
Returns nothing.
Implementation of the aquaculture calculations, and subsequent outputs. This will pull from data passed in by finfish_aquaculture.
-
natcap.invest.finfish_aquaculture.finfish_aquaculture_core.
calc_farm_cycles
(outplant_buffer, a, b, tau, water_temp_dict, farm_op_dict, dur)¶ - Input:
- outplant_buffer: The number of days surrounding the outplant day during
- which the fish growth cycle can still be started.
- a: Growth parameter alpha. Float used as a scaler in the fish growth
- equation.
- b: Growth paramater beta. Float used as an exponential multiplier in
- the fish growth equation.
- water_temp_dict: 2D dictionary which contains temperature values for
- farms. The outer keys are calendar days as strings, and the inner are farm numbers as strings.
- farm_op_dict: 2D dictionary which contains individual operating
- parameters for each farm. The outer key is farm number as a string, and the inner is string descriptors of each parameter.
- dur: Float which describes the length for the growth simulation to run
- in years.
Returns cycle_history where:
- cycle_history: Dictionary which contains mappings from farms to a
history of growth for each cycle completed on that farm. These entries are formatted as follows…
- Farm->List of Type (day of outplanting,day of harvest, fish weight
- (grams))
-
natcap.invest.finfish_aquaculture.finfish_aquaculture_core.
calc_hrv_weight
(farm_op_dict, frac, mort, cycle_history)¶ - Input:
- farm_op_dict: 2D dictionary which contains individual operating
- parameters for each farm. The outer key is farm number as a string, and the inner is string descriptors of each parameter.
- frac: A float representing the fraction of the fish that remains after
- processing.
- mort: A float referring to the daily mortality rate of fishes on an
- aquaculture farm.
- cycle_history: Farm->List of Type (day of outplanting,
- day of harvest, fish weight (grams))
- Returns a tuple (curr_cycle_totals,indiv_tpw_totals) where:
- curr_cycle_totals_: dictionary which will hold a mapping from every
- farm (as identified by farm_ID) to the total processed weight of each farm
- indiv_tpw_totals: dictionary which will hold a farm->list mapping,
- where the list holds the individual tpw for all cycles that the farm completed
-
natcap.invest.finfish_aquaculture.finfish_aquaculture_core.
compute_uncertainty_data
(args, output_dir)¶ Does uncertainty analysis via a Monte Carlo simulation.
Returns a tuple with two 2D dicts. -a dict containing relative file paths to produced histograms -a dict containining statistical results (mean and std deviation) Each dict has farm IDs as outer keys, and result types (e.g. ‘value’, ‘weight’, and ‘cycles’) as inner keys.
-
natcap.invest.finfish_aquaculture.finfish_aquaculture_core.
create_HTML_table
(output_dir, args, cycle_history, sum_hrv_weight, hrv_weight, farms_npv, value_history, histogram_paths, uncertainty_stats)¶ - Inputs:
- output_dir: The directory in which we will be creating our .html file
- output.
- cycle_history: dictionary mapping farm ID->list of tuples, each of
- which contains 3 things- (day of outplanting, day of harvest,
- harvest weight of a single fish in grams)
- sum_hrv_weight: dictionary which holds a mapping from farm ID->total
- processed weight of each farm
- hrv_weight: dictionary which holds a farm->list mapping, where the list
- holds the individual tpw for all cycles that the farm completed
- do_valuation: boolean variable that says whether or not valuation is
- desired
- farms_npv: dictionary with a farm-> float mapping, where each float is
- the net processed value of the fish processed on that farm, in $1000s of dollars.
- value_history: dictionary which holds a farm->list mapping, where the
- list holds tuples containing (Net Revenue, Net Present Value) for each cycle completed by that farm
- Output:
- HTML file: contains 3 tables that summarize inputs and outputs for the
duration of the model. - Input Table: Farm Operations provided data, including Farm ID #,
Cycle Number, weight of fish at start, weight of fish at harvest, number of fish in farm, start day for growing, and length of fallowing period- Output Table 1: Farm Harvesting data, including a summary table
- for each harvest cycle of each farm. Will show Farm ID, cycle number, days since outplanting date, harvested weight, net revenue, outplant day, and year.
- Output Table 2: Model outputs for each farm, including Farm ID,
- net present value, number of completed harvest cycles, and total volume harvested.
Returns nothing.
-
natcap.invest.finfish_aquaculture.finfish_aquaculture_core.
do_monte_carlo_simulation
(args)¶ Performs a Monte Carlo simulation and returns the results.
-
natcap.invest.finfish_aquaculture.finfish_aquaculture_core.
execute
(args)¶ ‘ Runs the biophysical and valuation parts of the finfish aquaculture model. This will output: 1. a shape file showing farm locations w/ addition of # of harvest cycles,
total processed weight at that farm, and if valuation is true, total discounted net revenue at each farm location.- Three HTML tables summarizing all model I/O- summary of user-provided
- data, summary of each harvest cycle, and summary of the outputs/farm
- A .txt file that is named according to the date and time the model is
- run, which lists the values used during that run
Data in args should include the following: –Biophysical Arguments– args: a python dictionary containing the following data: args[‘workspace_dir’]- The directory in which to place all result files. args[‘ff_farm_file’]- An open shape file containing the locations of
individual fisheries- args[‘farm_ID’]- column heading used to describe individual farms. Used to
- link GIS location data to later inputs.
- args[‘g_param_a’]- Growth parameter alpha, used in modeling fish growth,
- should be int or a float.
- args[‘g_param_b’]- Growth parameter beta, used in modeling fish growth,
- should be int or a float.
- args[‘water_temp_dict’]- A dictionary which links a specific date to the
farm numbers, and their temperature values on that day. (Note: in this case, the outer keys 1 and 2 are calendar days out of 365, starting with January 1 (day 0), and the inner 1, 2, and 3 are farm numbers.)
- Format: {‘0’: ‘{‘1’: ‘8.447, ‘2’: ‘8.447’, ‘3’:‘8.947’, …}’ ,
- ‘1’: ‘{‘1’: ‘8.406, ‘2’: ‘8.406’, ‘3’:‘8.906’, …}’ ,
. . . . . . . . . }
- args[‘farm_op_dict’]- Dictionary which links a specific farm ID # to
another dictionary containing operating parameters mapped to their value for that particular farm (Note: in this case, the 1 and 2 are farm ID’s, not dates out of 365.)
- Format: {‘1’: ‘{‘Wt of Fish’: ‘0.06’, ‘Tar Weight’: ‘5.4’, …}’,
- ‘2’: ‘{‘Wt of Fish’: ‘0.06’, ‘Tar Weight’: ‘5.4’, …}’, . . . . . . . . . }
- args[‘frac_post_process’]- the fraction of edible fish left after
- processing is done to remove undesirable parts
- args[‘mort_rate_daily’]- mortality rate among fish in a year, divided by
- 365
args[‘duration’]- duration of the simulation, in years args[‘outplant_buffer’] - This value will allow the outplant start day to
be flexible plus or minus the number of days specified here.–Valuation arguments– args[‘do_valuation’]- boolean indicating whether or not to run the
valuation processargs[‘p_per_kg’]: Market price per kilogram of processed fish args[‘frac_p’]: Fraction of market price that accounts for costs rather
than profitargs[‘discount’]: Daily market discount rate
returns nothing
-
natcap.invest.finfish_aquaculture.finfish_aquaculture_core.
make_histograms
(farm, results, output_dir, total_num_runs)¶ Makes a histogram for the given farm and data.
Returns a dict mapping type (e.g. ‘value’, ‘weight’) to the relative file path for the respective histogram.
-
natcap.invest.finfish_aquaculture.finfish_aquaculture_core.
valuation
(price_per_kg, frac_mrkt_price, discount, hrv_weight, cycle_history)¶ This performs the valuation calculations, and returns tuple containing a dictionary with a farm-> float mapping, where each float is the net processed value of the fish processed on that farm, in $1000s of dollars, and a dictionary containing a farm-> list mapping, where each entry in the list is a tuple of (Net Revenue, Net Present Value) for every cycle on that farm.
- Inputs:
- price_per_kg: Float representing the price per kilogram of finfish for
- valuation purposes.
- frac_mrkt_price: Float that represents the fraction of market price
- that is attributable to costs.
discount: Float that is the daily market discount rate. cycle_hisory: Farm->List of Type (day of outplanting,
day of harvest, fish weight (grams))hrv_weight: Farm->List of TPW for each cycle (kilograms)
- Returns a tuple (val_history, valuations):
- val_history: dictionary which will hold a farm->list mapping, where the
- list holds tuples containing (Net Revenue, Net Present Value) for each cycle completed by that farm
- valuations: dictionary with a farm-> float mapping, where each float is
- the net processed value of the fish processed on that farm
The Fisheries module contains the high-level code for executing the fisheries model
-
natcap.invest.fisheries.fisheries.
execute
(args, create_outputs=True)¶ Fisheries.
- args[‘workspace_dir’] (str): location into which all intermediate
- and output files should be placed.
args[‘results_suffix’] (str): a string to append to output filenames
- args[‘aoi_vector_path’] (str): location of shapefile which will be
- used as subregions for calculation. Each region must contain a ‘Name’ attribute (case-sensitive) matching the given name in the population parameters csv file.
- args[‘timesteps’] (int): represents the number of time steps that
- the user desires the model to run.
- args[‘population_type’] (str): specifies whether the model
- is age-specific or stage-specific. Options will be either “Age Specific” or “Stage Specific” and will change which equation is used in modeling growth.
- args[‘sexsp’] (str): specifies whether or not the age and stage
- classes are distinguished by sex.
- args[‘harvest_units’] (str): specifies how the user wants to get
- the harvest data. Options are either “Individuals” or “Weight”, and will change the harvest equation used in core. (Required if args[‘val_cont’] is True)
- args[‘do_batch’] (bool): specifies whether program will perform a
- single model run or a batch (set) of model runs.
- args[‘population_csv_path’] (str): location of the population
- parameters csv. This will contain all age and stage specific parameters. (Required if args[‘do_batch’] is False)
- args[‘population_csv_dir’] (str): location of the directory that
- contains the Population Parameters CSV files for batch processing (Required if args[‘do_batch’] is True)
- args[‘spawn_units’] (str): specifies whether the spawner abundance used in
- the recruitment function should be calculated in terms of number of individuals (‘Individuals’) or in terms of biomass (‘Weight’). If ‘Weight’ is selected, the user must provide a ‘Weight’ vector alongside the survival matrix in the Population Parameters CSV File. The ‘alpha’ and ‘beta’ parameters provided by the user should correspond to the selected choice.
- args[‘total_init_recruits’] (float): represents the initial number of
- recruits that will be used in calculation of population on a per area basis.
- args[‘recruitment_type’] (str): name corresponding to one of the built-in
- recruitment functions {‘Beverton-Holt’, ‘Ricker’, ‘Fecundity’, Fixed}, or ‘Other’, meaning that the user is passing in their own recruitment function as an anonymous python function via the optional dictionary argument ‘recruitment_func’.
- args[‘recruitment_func’] (function): Required if args[‘recruitment_type’]
- is set to ‘Other’. See below for instructions on how to create a user- defined recruitment function.
- args[‘alpha’] (float): must exist within args for BH or Ricker Recruitment.
- Parameter that will be used in calculation of recruitment.
- args[‘beta’] (float): must exist within args for BH or Ricker Recruitment.
- Parameter that will be used in calculation of recruitment.
- args[‘total_recur_recruits’] (float): must exist within args for Fixed
- Recruitment. Parameter that will be used in calculation of recruitment.
args[‘migr_cont’] (bool): if True, model uses migration.
- args[‘migration_dir’] (str): if this parameter exists, it means migration
- is desired. This is the location of the parameters folder containing files for migration. There should be one file for every age class which migrates. (Required if args[‘migr_cont’] is True)
args[‘val_cont’] (bool): if True, model computes valuation.
- args[‘frac_post_process’] (float): represents the fraction of the species
- remaining after processing of the whole carcass is complete. This will exist only if valuation is desired for the particular species. (Required if args[‘val_cont’] is True)
- args[‘unit_price’] (float): represents the price for a single unit of
- harvest. Exists only if valuation is desired. (Required if args[‘val_cont’] is True)
Example Args:
args = { 'workspace_dir': 'path/to/workspace_dir/', 'results_suffix': 'scenario_name', 'aoi_vector_path': 'path/to/aoi_vector_path', 'total_timesteps': 100, 'population_type': 'Stage-Based', 'sexsp': 'Yes', 'harvest_units': 'Individuals', 'do_batch': False, 'population_csv_path': 'path/to/csv_path', 'population_csv_dir': '', 'spawn_units': 'Weight', 'total_init_recruits': 100000.0, 'recruitment_type': 'Ricker', 'alpha': 32.4, 'beta': 54.2, 'total_recur_recruits': 92.1, 'migr_cont': True, 'migration_dir': 'path/to/mig_dir/', 'val_cont': True, 'frac_post_process': 0.5, 'unit_price': 5.0, }
Creating a User-Defined Recruitment Function
An optional argument has been created in the Fisheries Model to allow users proficient in Python to pass their own recruitment function into the program via the args dictionary.
Using the Beverton-Holt recruitment function as an example, here’s how a user might create and pass in their own recruitment function:
import natcap.invest import numpy as np # define input data Matu = np.array([...]) # the Maturity vector in the Population Parameters File Weight = np.array([...]) # the Weight vector in the Population Parameters File LarvDisp = np.array([...]) # the LarvalDispersal vector in the Population Parameters File alpha = 2.0 # scalar value beta = 10.0 # scalar value sexsp = 2 # 1 = not sex-specific, 2 = sex-specific # create recruitment function def spawners(N_prev): return (N_prev * Matu * Weight).sum() def rec_func_BH(N_prev): N_0 = (LarvDisp * ((alpha * spawners( N_prev) / (beta + spawners(N_prev)))) / sexsp) return (N_0, spawners(N_prev)) # fill out args dictionary args = {} # ... define other arguments ... args['recruitment_type'] = 'Other' # lets program know to use user- defined function args['recruitment_func'] = rec_func_BH # pass recruitment function as 'anonymous' Python function # run model natcap.invest.fisheries.fisheries.execute(args)
Conditions that a new recruitment function must meet to run properly:
- The function must accept as an argument: a single numpy three-
- dimensional array (N_prev) representing the state of the population at the previous time step. N_prev has three dimensions: the indices of the first dimension correspond to the region (must be in same order as provided in the Population Parameters File), the indices of the second dimension represent the sex if it is specific (i.e. two indices representing female, then male if the model is ‘sex-specific’, else just a single zero index representing the female and male populations aggregated together), and the indicies of the third dimension represent age/stage in ascending order.
- The function must return: a tuple of two values. The first value
- (N_0) being a single numpy one-dimensional array representing the youngest age of the population for the next time step. The indices of the array correspond to the regions of the population (outputted in same order as provided). If the model is sex-specific, it is currently assumed that males and females are produced in equal number and that the returned array has been already been divided by 2 in the recruitment function. The second value (spawners) is the number or weight of the spawners created by the population from the previous time step, provided as a scalar float value (non-negative).
Example of How Recruitment Function Operates within Fisheries Model:
# input data N_prev_xsa = [[[region0-female-age0, region0-female-age1], [region0-male-age0, region1-male-age1]], [[region1-female-age0, region1-female-age1], [region1-male-age0], [region1-male-age1]]] # execute function N_0_x, spawners = rec_func(N_prev_xsa) # output data - where N_0 contains information about the youngest # age/stage of the population for the next time step: N_0_x = [region0-age0, region1-age0] # if sex-specific, rec_func should divide by two before returning type(spawners) is float
The Fisheries Habitat Scenario Tool module contains the high-level code for generating a new Population Parameters CSV File based on habitat area change and the dependencies that particular classes of the given species have on particular habitats.
-
natcap.invest.fisheries.fisheries_hst.
convert_survival_matrix
(vars_dict)¶ Creates a new survival matrix based on the information provided by the user related to habitat area changes and class-level dependencies on those habitats.
Parameters: vars_dict (dictionary) – see fisheries_preprocessor_io.fetch_args for example Returns: - modified vars_dict with new Survival matrix
- accessible using the key ‘Surv_nat_xsa_mod’ with element values that exist between [0,1]
Return type: vars_dict (dictionary) Example Returns:
ret = { # Other Variables... 'Surv_nat_xsa_mod': np.ndarray([...]) }
-
natcap.invest.fisheries.fisheries_hst.
execute
(args)¶ Fisheries: Habitat Scenario Tool.
The Fisheries Habitat Scenario Tool generates a new Population Parameters CSV File with modified survival attributes across classes and regions based on habitat area changes and class-level dependencies on those habitats.
- args[‘workspace_dir’] (str): location into which the resultant
- modified Population Parameters CSV file should be placed.
- args[‘sexsp’] (str): specifies whether or not the age and stage
- classes are distinguished by sex. Options: ‘Yes’ or ‘No’
- args[‘population_csv_path’] (str): location of the population
- parameters csv file. This file contains all age and stage specific parameters.
- args[‘habitat_chg_csv_path’] (str): location of the habitat change
- parameters csv file. This file contains habitat area change information.
- args[‘habitat_dep_csv_path’] (str): location of the habitat dependency
- parameters csv file. This file contains habitat-class dependency information.
- args[‘gamma’] (float): describes the relationship between a change
- in habitat area and a change in survival of life stages dependent on that habitat
Returns: None Example Args:
args = { 'workspace_dir': 'path/to/workspace_dir/', 'sexsp': 'Yes', 'population_csv_path': 'path/to/csv', 'habitat_chg_csv_path': 'path/to/csv', 'habitat_dep_csv_path': 'path/to/csv', 'gamma': 0.5, }
Note
- Modified Population Parameters CSV File saved to ‘workspace_dir/output/’
The Fisheries Habitat Scenarios Tool IO module contains functions for handling inputs and outputs
-
natcap.invest.fisheries.fisheries_hst_io.
fetch_args
(args)¶ Fetches input arguments from the user, verifies for correctness and completeness, and returns a list of variables dictionaries
Parameters: args (dictionary) – arguments from the user (same as Fisheries Preprocessor entry point) Returns: dictionary containing necessary variables Return type: vars_dict (dictionary) Raises: ValueError
– parameter mismatch between Population and Habitat CSV filesExample Returns:
vars_dict = { 'workspace_dir': 'path/to/workspace_dir/', 'output_dir': 'path/to/output_dir/', 'sexsp': 2, 'gamma': 0.5, # Pop Vars 'population_csv_path': 'path/to/csv_path', 'Surv_nat_xsa': np.array( [[[...], [...]], [[...], [...]], ...]), 'Classes': np.array([...]), 'Class_vectors': { 'Vulnfishing': np.array([...], [...]), 'Maturity': np.array([...], [...]), 'Duration': np.array([...], [...]), 'Weight': np.array([...], [...]), 'Fecundity': np.array([...], [...]), }, 'Regions': np.array([...]), 'Region_vectors': { 'Exploitationfraction': np.array([...]), 'Larvaldispersal': np.array([...]), }, # Habitat Vars 'habitat_chg_csv_path': 'path/to/csv', 'habitat_dep_csv_path': 'path/to/csv', 'Habitats': ['habitat1', 'habitat2', ...], 'Hab_classes': ['class1', 'class2', ...], 'Hab_regions': ['region1', 'region2', ...], 'Hab_chg_hx': np.array( [[[...], [...]], [[...], [...]], ...]), 'Hab_dep_ha': np.array( [[[...], [...]], [[...], [...]], ...]), 'Hab_class_mvmt_a': np.array([...]), 'Hab_dep_num_a': np.array([...]), }
-
natcap.invest.fisheries.fisheries_hst_io.
read_habitat_chg_csv
(args)¶ Parses and verifies a Habitat Change Parameters CSV file and returns a dictionary of information related to the interaction between a species and the given habitats.
Parses the Habitat Change Parameters CSV file for the following vectors:
- Names of Habitats and Regions
- Habitat Area Change
Parameters: args (dictionary) – arguments from the user (same as Fisheries HST entry point)
Returns: - dictionary containing necessary
variables
Return type: habitat_chg_dict (dictionary)
Raises: MissingParameter
– required parameter not includedValueError
– values are out of bounds or of wrong typeIndexError
– likely a file formatting issue
Example Returns:
habitat_chg_dict = { 'Habitats': ['habitat1', 'habitat2', ...], 'Hab_regions': ['region1', 'region2', ...], 'Hab_chg_hx': np.array( [[[...], [...]], [[...], [...]], ...]), }
-
natcap.invest.fisheries.fisheries_hst_io.
read_habitat_dep_csv
(args)¶ Parses and verifies a Habitat Dependency Parameters CSV file and returns a dictionary of information related to the interaction between a species and the given habitats.
Parses the Habitat Parameters CSV file for the following vectors:
- Names of Habitats and Classes
- Habitat-Class Dependency
The following vectors are derived from the information given in the file:
- Classes where movement between habitats occurs
- Number of habitats that a particular class depends upon
Parameters: args (dictionary) – arguments from the user (same as Fisheries HST entry point)
Returns: - dictionary containing necessary
variables
Return type: habitat_dep_dict (dictionary)
Raises: - MissingParameter - required parameter not included
- ValueError - values are out of bounds or of wrong type
- IndexError - likely a file formatting issue
Example Returns:
habitat_dep_dict = { 'Habitats': ['habitat1', 'habitat2', ...], 'Hab_classes': ['class1', 'class2', ...], 'Hab_dep_ha': np.array( [[[...], [...]], [[...], [...]], ...]), 'Hab_class_mvmt_a': np.array([...]), 'Hab_dep_num_a': np.array([...]), }
-
natcap.invest.fisheries.fisheries_hst_io.
read_population_csv
(args)¶ Parses and verifies a single Population Parameters CSV file
Parses and verifies inputs from the Population Parameters CSV file. If not all necessary vectors are included, the function will raise a MissingParameter exception. Survival matrix will be arranged by class-elements, 2nd dim: sex, and 3rd dim: region. Class vectors will be arranged by class-elements, 2nd dim: sex (depending on whether model is sex-specific) Region vectors will be arraged by region-elements, sex-agnostic.
Parameters: args (dictionary) – arguments provided by user
Returns: - dictionary containing verified population
arguments
Return type: pop_dict (dictionary)
Raises: MissingParameter
– required parameter not includedValueError
– values are out of bounds or of wrong type
Example Returns:
pop_dict = { 'population_csv_path': 'path/to/csv', 'Surv_nat_xsa': np.array( [[...], [...]], [[...], [...]], ...), # Class Vectors 'Classes': np.array([...]), 'Class_vector_names': [...], 'Class_vectors': { 'Vulnfishing': np.array([...], [...]), 'Maturity': np.array([...], [...]), 'Duration': np.array([...], [...]), 'Weight': np.array([...], [...]), 'Fecundity': np.array([...], [...]), }, # Region Vectors 'Regions': np.array([...]), 'Region_vector_names': [...], 'Region_vectors': { 'Exploitationfraction': np.array([...]), 'Larvaldispersal': np.array([...]), }, }
-
natcap.invest.fisheries.fisheries_hst_io.
save_population_csv
(vars_dict)¶ Creates a new Population Parameters CSV file based the provided inputs.
Parameters: vars_dict (dictionary) – variables generated by preprocessor arguments and run. Example Args:
args = { 'workspace_dir': 'path/to/workspace_dir/', 'output_dir': 'path/to/output_dir/', 'sexsp': 2, 'population_csv_path': 'path/to/csv', # original csv file 'Surv_nat_xsa': np.ndarray([...]), 'Surv_nat_xsa_mod': np.ndarray([...]), # Class Vectors 'Classes': np.array([...]), 'Class_vector_names': [...], 'Class_vectors': { 'Vulnfishing': np.array([...], [...]), 'Maturity': np.array([...], [...]), 'Duration': np.array([...], [...]), 'Weight': np.array([...], [...]), 'Fecundity': np.array([...], [...]), }, # Region Vectors 'Regions': np.array([...]), 'Region_vector_names': [...], 'Region_vectors': { 'Exploitationfraction': np.array([...]), 'Larvaldispersal': np.array([...]), }, # other arguments are ignored ... }
Note
- Creates a modified Population Parameters CSV file located in the ‘workspace/output/’ folder
- Currently appends ‘_modified’ to original filename for new filename
The Fisheries IO module contains functions for handling inputs and outputs
-
exception
natcap.invest.fisheries.fisheries_io.
MissingParameter
¶ Bases:
exceptions.ValueError
An exception class that may be raised when a necessary parameter is not provided by the user.
-
natcap.invest.fisheries.fisheries_io.
create_outputs
(vars_dict)¶ Creates outputs from variables generated in the run_population_model() function in the fisheries_model module
Creates the following:
- Results CSV File
- Results HTML Page
- Results Shapefile (if provided)
- Intermediate CSV File
Parameters: vars_dict (dictionary) – contains variables generated by model run
-
natcap.invest.fisheries.fisheries_io.
fetch_args
(args, create_outputs=True)¶ Fetches input arguments from the user, verifies for correctness and completeness, and returns a list of variables dictionaries
Parameters: args (dictionary) – arguments from the user Returns: - set of variable dictionaries for each
- model
Return type: model_list (list) Example Returns:
model_list = [ { 'workspace_dir': 'path/to/workspace_dir', 'results_suffix': 'scenario_name', 'output_dir': 'path/to/output_dir', 'aoi_vector_path': 'path/to/aoi_vector_path', 'total_timesteps': 100, 'population_type': 'Stage-Based', 'sexsp': 2, 'harvest_units': 'Individuals', 'do_batch': False, 'spawn_units': 'Weight', 'total_init_recruits': 100.0, 'recruitment_type': 'Ricker', 'alpha': 32.4, 'beta': 54.2, 'total_recur_recruits': 92.1, 'migr_cont': True, 'val_cont': True, 'frac_post_process': 0.5, 'unit_price': 5.0, # Pop Params 'population_csv_path': 'path/to/csv_path', 'Survnaturalfrac': numpy.array( [[[...], [...]], [[...], [...]], ...]), 'Classes': numpy.array([...]), 'Vulnfishing': numpy.array([...], [...]), 'Maturity': numpy.array([...], [...]), 'Duration': numpy.array([...], [...]), 'Weight': numpy.array([...], [...]), 'Fecundity': numpy.array([...], [...]), 'Regions': numpy.array([...]), 'Exploitationfraction': numpy.array([...]), 'Larvaldispersal': numpy.array([...]), # Mig Params 'migration_dir': 'path/to/mig_dir', 'Migration': [numpy.matrix, numpy.matrix, ...] }, { ... # additional dictionary doesn't exist when 'do_batch' # is false } ]
Note
This function receives an unmodified ‘args’ dictionary from the user
-
natcap.invest.fisheries.fisheries_io.
read_migration_tables
(args, class_list, region_list)¶ Parses, verifies and orders list of migration matrices necessary for program.
Parameters: - args (dictionary) – same args as model entry point
- class_list (list) – list of class names
- region_list (list) – list of region names
Returns: see example below
Return type: mig_dict (dictionary)
Example Returns:
mig_dict = { 'Migration': [numpy.matrix, numpy.matrix, ...] }
Note
If migration matrices are not provided for all classes, the function will generate identity matrices for missing classes
-
natcap.invest.fisheries.fisheries_io.
read_population_csv
(args, path)¶ Parses and verifies a single Population Parameters CSV file
Parses and verifies inputs from the Population Parameters CSV file. If not all necessary vectors are included, the function will raise a MissingParameter exception. Survival matrix will be arranged by class-elements, 2nd dim: sex, and 3rd dim: region. Class vectors will be arranged by class-elements, 2nd dim: sex (depending on whether model is sex-specific) Region vectors will be arraged by region-elements, sex-agnostic.
Parameters: - args (dictionary) – arguments provided by user
- path (string) – the particular Population Parameters CSV file to parse and verifiy
Returns: - dictionary containing verified population
arguments
Return type: pop_dict (dictionary)
Example Returns:
pop_dict = { 'population_csv_path': 'path/to/csv', 'Survnaturalfrac': numpy.array( [[...], [...]], [[...], [...]], ...), # Class Vectors 'Classes': numpy.array([...]), 'Vulnfishing': numpy.array([...], [...]), 'Maturity': numpy.array([...], [...]), 'Duration': numpy.array([...], [...]), 'Weight': numpy.array([...], [...]), 'Fecundity': numpy.array([...], [...]), # Region Vectors 'Regions': numpy.array([...]), 'Exploitationfraction': numpy.array([...]), 'Larvaldispersal': numpy.array([...]), }
-
natcap.invest.fisheries.fisheries_io.
read_population_csvs
(args)¶ Parses and verifies the Population Parameters CSV files
Parameters: args (dictionary) – arguments provided by user Returns: - list of dictionaries containing verified population
- arguments
Return type: pop_list (list) Example Returns:
pop_list = [ { 'Survnaturalfrac': numpy.array( [[...], [...]], [[...], [...]], ...), # Class Vectors 'Classes': numpy.array([...]), 'Vulnfishing': numpy.array([...], [...]), 'Maturity': numpy.array([...], [...]), 'Duration': numpy.array([...], [...]), 'Weight': numpy.array([...], [...]), 'Fecundity': numpy.array([...], [...]), # Region Vectors 'Regions': numpy.array([...]), 'Exploitationfraction': numpy.array([...]), 'Larvaldispersal': numpy.array([...]), }, { ... } ]
The Fisheries Model module contains functions for running the model
Variable Suffix Notation: t: time x: area/region a: age/class s: sex
-
natcap.invest.fisheries.fisheries_model.
initialize_vars
(vars_dict)¶ Initializes variables for model run
Parameters: vars_dict (dictionary) – verified arguments and variables Returns: modified vars_dict with additional variables Return type: vars_dict (dictionary) Example Returns:
vars_dict = { # (original vars) 'Survtotalfrac': np.array([...]), # a,s,x 'G_survtotalfrac': np.array([...]), # (same) 'P_survtotalfrac': np.array([...]), # (same) 'N_tasx': np.array([...]), # Index Order: t,a,s,x 'H_tx': np.array([...]), # t,x 'V_tx': np.array([...]), # t,x 'Spawners_t': np.array([...]), }
-
natcap.invest.fisheries.fisheries_model.
run_population_model
(vars_dict, init_cond_func, cycle_func, harvest_func)¶ Runs the model
Parameters: - vars_dict (dictionary) –
- init_cond_func (lambda function) – sets initial conditions
- cycle_func (lambda function) – computes numbers for the next time step
- harvest_func (lambda function) – computes harvest and valuation
Returns: vars_dict (dictionary)
Example Returned Dictionary:
{ # (other items) ... 'N_tasx': np.array([...]), # Index Order: time, class, sex, region 'H_tx': np.array([...]), # Index Order: time, region 'V_tx': np.array([...]), # Index Order: time, region 'Spawners_t': np,array([...]), 'equilibrate_timestep': <int>, }
-
natcap.invest.fisheries.fisheries_model.
set_cycle_func
(vars_dict, rec_func)¶ Creates a function to run a single cycle in the model
Parameters: - vars_dict (dictionary) –
- rec_func (lambda function) – recruitment function
Example Output of Returned Cycle Function:
N_asx = np.array([...]) spawners = <int> N_next, spawners = cycle_func(N_prev)
-
natcap.invest.fisheries.fisheries_model.
set_harvest_func
(vars_dict)¶ Creates harvest function that calculates the given harvest and valuation of the fisheries population over each time step for a given region. Returns None if harvest isn’t selected by user.
Example Outputs of Returned Harvest Function:
H_x, V_x = harv_func(N_tasx) H_x = np.array([3.0, 4.5, 2.5, ...]) V_x = np.array([6.0, 9.0, 5.0, ...])
-
natcap.invest.fisheries.fisheries_model.
set_init_cond_func
(vars_dict)¶ Creates a function to set the initial conditions of the model
Parameters: vars_dict (dictionary) – variables Returns: initial conditions function Return type: init_cond_func (lambda function) Example Return Array:
N_asx = np.ndarray([...])
-
natcap.invest.fisheries.fisheries_model.
set_recru_func
(vars_dict)¶ Creates recruitment function that calculates the number of recruits for class 0 at time t for each region (currently sex agnostic). Also returns number of spawners
Parameters: vars_dict (dictionary) – Returns: recruitment function Return type: rec_func (function) Example Output of Returned Recruitment Function:
N_next[0], spawners = rec_func(N_prev)
InVEST Hydropower Water Yield model.
-
natcap.invest.hydropower.hydropower_water_yield.
aet_op
(fractp, precip, precip_nodata, output_nodata)¶ Compute actual evapotranspiration values.
Parameters: - fractp (numpy.ndarray float) – fractp raster values.
- precip (numpy.ndarray) – precipitation raster values (mm).
- precip_nodata (float) – nodata value from the precip raster.
- output_nodata (float) – nodata value assigned to output of raster_calculator.
Returns: numpy.ndarray of actual evapotranspiration values (mm).
-
natcap.invest.hydropower.hydropower_water_yield.
compute_rsupply_volume
(watershed_results_vector_path)¶ Calculate the total realized water supply volume.
And the mean realized water supply volume per hectare for the given sheds. Output units in cubic meters and cubic meters per hectare respectively.
Parameters: watershed_results_vector_path (string) – a path to a vector that contains fields ‘wyield_vol’ and ‘wyield_mn’. Returns: None.
-
natcap.invest.hydropower.hydropower_water_yield.
compute_water_yield_volume
(watershed_results_vector_path)¶ Calculate the water yield volume per sub-watershed or watershed.
Results are added to a ‘wyield_vol’ field in watershed_results_vector_path. Units are cubic meters.
Parameters: watershed_results_vector_path (str) – Path to a sub-watershed or watershed vector. This vector’s features should have a ‘wyield_mn’ attribute. Returns: None.
-
natcap.invest.hydropower.hydropower_water_yield.
compute_watershed_valuation
(watershed_results_vector_path, val_dict)¶ Compute net present value and energy for the watersheds.
Parameters: - watershed_results_vector_path (string) – Path to an OGR shapefile for the watershed results. Where the results will be added.
- val_dict (dict) – a python dictionary that has all the valuation parameters for each watershed.
Returns: None.
-
natcap.invest.hydropower.hydropower_water_yield.
convert_vector_to_csv
(base_vector_path, target_csv_path)¶ Create a CSV with all the fields present in vector attribute table.
Parameters: - base_vector_path (string) – Path to the watershed shapefile in the output workspace.
- target_csv_path (string) – Path to a CSV to create in the output workspace.
Returns: None
-
natcap.invest.hydropower.hydropower_water_yield.
create_vector_output
(base_vector_path, target_vector_path, ws_id_name, stats_path_list, valuation_params)¶ Create the main vector outputs of this model.
Join results of zonal stats to copies of the watershed shapefiles. Also do optional scarcity and valuation calculations.
Parameters: - base_vector_path (string) – Path to a watershed shapefile provided in the args dictionary.
- target_vector_path (string) – Path where base_vector_path will be copied to in the output workspace.
- ws_id_name (string) – Either ‘ws_id’ or ‘subws_id’, which are required names of a unique ID field in the watershed and subwatershed shapefiles, respectively. Used to determine if the polygons represent watersheds or subwatersheds.
- stats_path_list (list) – List of file paths to pickles storing the zonal stats results.
- valuation_params (dict) – The dictionary built from args[‘valuation_table_path’]. Or None if valuation table was not provided.
Returns: None
-
natcap.invest.hydropower.hydropower_water_yield.
execute
(args)¶ Annual Water Yield: Reservoir Hydropower Production.
Executes the hydropower/water_yield model
Parameters: - args['workspace_dir'] (string) – a path to the directory that will write output and other temporary files during calculation. (required)
- args['lulc_path'] (string) – a path to a land use/land cover raster whose LULC indexes correspond to indexes in the biophysical table input. Used for determining soil retention and other biophysical properties of the landscape. (required)
- args['depth_to_root_rest_layer_path'] (string) – a path to an input raster describing the depth of “good” soil before reaching this restrictive layer (required)
- args['precipitation_path'] (string) – a path to an input raster describing the average annual precipitation value for each cell (mm) (required)
- args['pawc_path'] (string) – a path to an input raster describing the plant available water content value for each cell. Plant Available Water Content fraction (PAWC) is the fraction of water that can be stored in the soil profile that is available for plants’ use. PAWC is a fraction from 0 to 1 (required)
- args['eto_path'] (string) – a path to an input raster describing the annual average evapotranspiration value for each cell. Potential evapotranspiration is the potential loss of water from soil by both evaporation from the soil and transpiration by healthy Alfalfa (or grass) if sufficient water is available (mm) (required)
- args['watersheds_path'] (string) – a path to an input shapefile of the watersheds of interest as polygons. (required)
- args['sub_watersheds_path'] (string) – a path to an input shapefile of
the subwatersheds of interest that are contained in the
args['watersheds_path']
shape provided as input. (optional) - args['biophysical_table_path'] (string) – a path to an input CSV table of land use/land cover classes, containing data on biophysical coefficients such as root_depth (mm) and Kc, which are required. A column with header LULC_veg is also required which should have values of 1 or 0, 1 indicating a land cover type of vegetation, a 0 indicating non vegetation or wetland, water. NOTE: these data are attributes of each LULC class rather than attributes of individual cells in the raster map (required)
- args['seasonality_constant'] (float) – floating point value between 1 and 30 corresponding to the seasonal distribution of precipitation (required)
- args['results_suffix'] (string) – a string that will be concatenated onto the end of file names (optional)
- args['demand_table_path'] (string) – (optional) if a non-empty string, a path to an input CSV table of LULC classes, showing consumptive water use for each landuse / land-cover type (cubic meters per year) to calculate water scarcity.
- args['valuation_table_path'] (string) –
(optional) if a non-empty string, a path to an input CSV table of hydropower stations with the following fields to calculate valuation:
(‘ws_id’, ‘time_span’, ‘discount’, ‘efficiency’, ‘fraction’, ‘cost’, ‘height’, ‘kw_price’)Required if
calculate_valuation
is True. - args['n_workers'] (int) – (optional) The number of worker processes to use for processing this model. If omitted, computation will take place in the current process.
Returns: None
-
natcap.invest.hydropower.hydropower_water_yield.
fractp_op
(Kc, eto, precip, root, soil, pawc, veg, nodata_dict, seasonality_constant)¶ Calculate actual evapotranspiration fraction of precipitation.
Parameters: - Kc (numpy.ndarray) – Kc (plant evapotranspiration coefficient) raster values
- eto (numpy.ndarray) – potential evapotranspiration raster values (mm)
- precip (numpy.ndarray) – precipitation raster values (mm)
- root (numpy.ndarray) – root depth (maximum root depth for vegetated land use classes) raster values (mm)
- soil (numpy.ndarray) – depth to root restricted layer raster values (mm)
- pawc (numpy.ndarray) – plant available water content raster values
- veg (numpy.ndarray) – 1 or 0 where 1 depicts the land type as vegetation and 0 depicts the land type as non vegetation (wetlands, urban, water, etc…). If 1 use regular AET equation if 0 use: AET = Kc * ETo
- nodata_dict (dict) – stores nodata values keyed by raster names
- seasonality_constant (float) – floating point value between 1 and 30 corresponding to the seasonal distribution of precipitation.
Returns: - numpy.ndarray (float) of actual evapotranspiration as fraction
of precipitation.
-
natcap.invest.hydropower.hydropower_water_yield.
pet_op
(eto_pix, Kc_pix, eto_nodata, output_nodata)¶ Calculate the plant potential evapotranspiration.
Parameters: - eto_pix (numpy.ndarray) – a numpy array of ETo
- Kc_pix (numpy.ndarray) – a numpy array of Kc coefficient
- precip_nodata (float) – nodata value from the precip raster
- output_nodata (float) – nodata value assigned to output of raster_calculator
Returns: numpy.ndarray of potential evapotranspiration (mm)
-
natcap.invest.hydropower.hydropower_water_yield.
wyield_op
(fractp, precip, precip_nodata, output_nodata)¶ Calculate water yield.
Parameters: - fractp (numpy.ndarray float) – fractp raster values.
- precip (numpy.ndarray) – precipitation raster values (mm).
- precip_nodata (float) – nodata value from the precip raster.
- output_nodata (float) – nodata value assigned to output of raster_calculator.
Returns: numpy.ndarray of water yield value (mm).
-
natcap.invest.hydropower.hydropower_water_yield.
zonal_stats_tofile
(base_vector_path, raster_path, target_stats_pickle)¶ Calculate zonal statistics for watersheds and write results to a file.
Parameters: - base_vector_path (string) – Path to the watershed shapefile in the output workspace.
- raster_path (string) – Path to raster to aggregate.
- target_stats_pickle (string) – Path to pickle file to store dictionary returned by zonal stats.
Returns: None
DBF accessing helpers.
FIXME: more documentation needed
Examples
Create new table, setup structure, add records:
dbf = Dbf(filename, new=True) dbf.addField(
(“NAME”, “C”, 15), (“SURNAME”, “C”, 25), (“INITIALS”, “C”, 10), (“BIRTHDATE”, “D”),) for (n, s, i, b) in (
(“John”, “Miller”, “YC”, (1980, 10, 11)), (“Andy”, “Larkin”, “”, (1980, 4, 11)),
- ):
- rec = dbf.newRecord() rec[“NAME”] = n rec[“SURNAME”] = s rec[“INITIALS”] = i rec[“BIRTHDATE”] = b rec.store()
dbf.close()
Open existed dbf, read some data:
dbf = Dbf(filename, True) for rec in dbf:
- for fldName in dbf.fieldNames:
- print ‘%s: %s (%s)’ % (fldName, rec[fldName],
- type(rec[fldName]))
dbf.close()
-
class
natcap.invest.pygeoprocessing_0_3_3.dbfpy.dbf.
Dbf
(f, readOnly=False, new=False, ignoreErrors=False)¶ Bases:
object
DBF accessor.
- FIXME:
- docs and examples needed (dont’ forget to tell about problems adding new fields on the fly)
- Implementation notes:
_new
field is used to indicate whether this is a new data table. addField could be used only for the new tables! If at least one record was appended to the table it’s structure couldn’t be changed.
-
HeaderClass
¶ alias of
natcap.invest.pygeoprocessing_0_3_3.dbfpy.header.DbfHeader
-
INVALID_VALUE
= <INVALID>¶
-
RecordClass
¶ alias of
natcap.invest.pygeoprocessing_0_3_3.dbfpy.record.DbfRecord
-
addField
(*defs)¶ Add field definitions.
For more information see header.DbfHeader.addField.
-
append
(record)¶ Append
record
to the database.
-
changed
¶
-
close
()¶
-
closed
¶
-
fieldDefs
¶
-
fieldNames
¶
-
flush
()¶ Flush data to the associated stream.
-
header
¶
-
ignoreErrors
¶ Error processing mode for DBF field value conversion
if set, failing field value conversion will return
INVALID_VALUE
instead of raising conversion error.
-
indexOfFieldName
(name)¶ Index of field named
name
.
-
name
¶
-
newRecord
()¶ Return new record, which belong to this table.
-
recordCount
¶
-
stream
¶
.DBF creation helpers.
- Note: this is a legacy interface. New code should use Dbf class
- for table creation (see examples in dbf.py)
-
class
natcap.invest.pygeoprocessing_0_3_3.dbfpy.dbfnew.
dbf_new
¶ Bases:
object
New .DBF creation helper.
Example Usage:
dbfn = dbf_new() dbfn.add_field(“name”,’C’,80) dbfn.add_field(“price”,’N’,10,2) dbfn.add_field(“date”,’D’,8) dbfn.write(“tst.dbf”)Note
This module cannot handle Memo-fields, they are special.
-
FieldDefinitionClass
¶ alias of
_FieldDefinition
-
add_field
(name, typ, len, dec=0)¶ Add field definition.
Parameters: - name – field name (str object). field name must not contain ASCII NULs and it’s length shouldn’t exceed 10 characters.
- typ – type of the field. this must be a single character from the “CNLMDT” set meaning character, numeric, logical, memo, date and date/time respectively.
- len – length of the field. this argument is used only for the character and numeric fields. all other fields have fixed length. FIXME: use None as a default for this argument?
- dec – decimal precision. used only for the numric fields.
-
fields
¶
-
write
(filename)¶ Create empty .DBF file using current structure.
-
DBF fields definitions.
-
natcap.invest.pygeoprocessing_0_3_3.dbfpy.fields.
lookupFor
(typeCode)¶ Return field definition class for the given type code.
typeCode
must be a single character. That type should be previously registered.Use registerField to register new field class.
Returns: Return value is a subclass of the DbfFieldDef.
-
class
natcap.invest.pygeoprocessing_0_3_3.dbfpy.fields.
DbfCharacterFieldDef
(name, length=None, decimalCount=None, start=None, stop=None, ignoreErrors=False)¶ Bases:
natcap.invest.pygeoprocessing_0_3_3.dbfpy.fields.DbfFieldDef
Definition of the character field.
-
decodeValue
(value)¶ Return string object.
Return value is a
value
argument with stripped right spaces.
-
defaultValue
= ''¶
-
encodeValue
(value)¶ Return raw data string encoded from a
value
.
-
typeCode
= 'C'¶
-
-
class
natcap.invest.pygeoprocessing_0_3_3.dbfpy.fields.
DbfFloatFieldDef
(name, length=None, decimalCount=None, start=None, stop=None, ignoreErrors=False)¶ Bases:
natcap.invest.pygeoprocessing_0_3_3.dbfpy.fields.DbfNumericFieldDef
Definition of the float field - same as numeric.
-
typeCode
= 'F'¶
-
-
class
natcap.invest.pygeoprocessing_0_3_3.dbfpy.fields.
DbfLogicalFieldDef
(name, length=None, decimalCount=None, start=None, stop=None, ignoreErrors=False)¶ Bases:
natcap.invest.pygeoprocessing_0_3_3.dbfpy.fields.DbfFieldDef
Definition of the logical field.
-
decodeValue
(value)¶ Return True, False or -1 decoded from
value
.
-
defaultValue
= -1¶
-
encodeValue
(value)¶ Return a character from the “TF?” set.
Returns: Return value is “T” if value
is True “?” if value is -1 or False otherwise.
-
length
= 1¶
-
typeCode
= 'L'¶
-
-
class
natcap.invest.pygeoprocessing_0_3_3.dbfpy.fields.
DbfDateFieldDef
(name, length=None, decimalCount=None, start=None, stop=None, ignoreErrors=False)¶ Bases:
natcap.invest.pygeoprocessing_0_3_3.dbfpy.fields.DbfFieldDef
Definition of the date field.
-
decodeValue
(value)¶ Return a
datetime.date
instance decoded fromvalue
.
-
defaultValue
= datetime.date(2019, 5, 9)¶
-
encodeValue
(value)¶ Return a string-encoded value.
value
argument should be a value suitable for the utils.getDate call.Returns: Return value is a string in format “yyyymmdd”.
-
length
= 8¶
-
typeCode
= 'D'¶
-
-
class
natcap.invest.pygeoprocessing_0_3_3.dbfpy.fields.
DbfMemoFieldDef
(name, length=None, decimalCount=None, start=None, stop=None, ignoreErrors=False)¶ Bases:
natcap.invest.pygeoprocessing_0_3_3.dbfpy.fields.DbfFieldDef
Definition of the memo field.
Note: memos aren’t currenly completely supported.
-
decodeValue
(value)¶ Return int .dbt block number decoded from the string object.
-
defaultValue
= ' '¶
-
encodeValue
(value)¶ Return raw data string encoded from a
value
.Note: this is an internal method.
-
length
= 10¶
-
typeCode
= 'M'¶
-
-
class
natcap.invest.pygeoprocessing_0_3_3.dbfpy.fields.
DbfNumericFieldDef
(name, length=None, decimalCount=None, start=None, stop=None, ignoreErrors=False)¶ Bases:
natcap.invest.pygeoprocessing_0_3_3.dbfpy.fields.DbfFieldDef
Definition of the numeric field.
-
decodeValue
(value)¶ Return a number decoded from
value
.If decimals is zero, value will be decoded as an integer; or as a float otherwise.
Returns: Return value is a int (long) or float instance.
-
defaultValue
= 0¶
-
encodeValue
(value)¶ Return string containing encoded
value
.
-
typeCode
= 'N'¶
-
-
class
natcap.invest.pygeoprocessing_0_3_3.dbfpy.fields.
DbfCurrencyFieldDef
(name, length=None, decimalCount=None, start=None, stop=None, ignoreErrors=False)¶ Bases:
natcap.invest.pygeoprocessing_0_3_3.dbfpy.fields.DbfFieldDef
Definition of the currency field.
-
decodeValue
(value)¶ Return float number decoded from
value
.
-
defaultValue
= 0.0¶
-
encodeValue
(value)¶ Return string containing encoded
value
.
-
length
= 8¶
-
typeCode
= 'Y'¶
-
-
class
natcap.invest.pygeoprocessing_0_3_3.dbfpy.fields.
DbfIntegerFieldDef
(name, length=None, decimalCount=None, start=None, stop=None, ignoreErrors=False)¶ Bases:
natcap.invest.pygeoprocessing_0_3_3.dbfpy.fields.DbfFieldDef
Definition of the integer field.
-
decodeValue
(value)¶ Return an integer number decoded from
value
.
-
defaultValue
= 0¶
-
encodeValue
(value)¶ Return string containing encoded
value
.
-
length
= 4¶
-
typeCode
= 'I'¶
-
-
class
natcap.invest.pygeoprocessing_0_3_3.dbfpy.fields.
DbfDateTimeFieldDef
(name, length=None, decimalCount=None, start=None, stop=None, ignoreErrors=False)¶ Bases:
natcap.invest.pygeoprocessing_0_3_3.dbfpy.fields.DbfFieldDef
Definition of the timestamp field.
-
JDN_GDN_DIFF
= 1721425¶
-
decodeValue
(value)¶ Return a datetime.datetime instance.
-
defaultValue
= datetime.datetime(2019, 5, 9, 23, 35, 1, 211541)¶
-
encodeValue
(value)¶ Return a string-encoded
value
.
-
length
= 8¶
-
typeCode
= 'T'¶
-
DBF header definition.
-
class
natcap.invest.pygeoprocessing_0_3_3.dbfpy.header.
DbfHeader
(fields=None, headerLength=0, recordLength=0, recordCount=0, signature=3, lastUpdate=None, ignoreErrors=False)¶ Bases:
object
Dbf header definition.
For more information about dbf header format visit http://www.clicketyclick.dk/databases/xbase/format/dbf.html#DBF_STRUCT
Examples
- Create an empty dbf header and add some field definitions:
- dbfh = DbfHeader() dbfh.addField((“name”, “C”, 10)) dbfh.addField((“date”, “D”)) dbfh.addField(DbfNumericFieldDef(“price”, 5, 2))
- Create a dbf header with field definitions:
- dbfh = DbfHeader([
- (“name”, “C”, 10), (“date”, “D”), DbfNumericFieldDef(“price”, 5, 2),
])
-
addField
(*defs)¶ Add field definition to the header.
Examples
- dbfh.addField(
- (“name”, “C”, 20), dbf.DbfCharacterFieldDef(“surname”, 20), dbf.DbfDateFieldDef(“birthdate”), (“member”, “L”),
) dbfh.addField((“price”, “N”, 5, 2)) dbfh.addField(dbf.DbfNumericFieldDef(“origprice”, 5, 2))
-
changed
¶
-
day
¶
-
fields
¶
-
classmethod
fromStream
(stream)¶ Return header object from the stream.
-
classmethod
fromString
(string)¶ Return header instance from the string object.
-
headerLength
¶
-
ignoreErrors
¶ Error processing mode for DBF field value conversion
if set, failing field value conversion will return
INVALID_VALUE
instead of raising conversion error.
-
lastUpdate
¶
-
month
¶
-
recordCount
¶
-
recordLength
¶
-
setCurrentDate
()¶ Update
self.lastUpdate
field with current date value.
-
signature
¶
-
toString
()¶ Returned 32 chars length string with encoded header.
-
write
(stream)¶ Encode and write header to the stream.
-
year
¶
DBF record definition.
-
class
natcap.invest.pygeoprocessing_0_3_3.dbfpy.record.
DbfRecord
(dbf, index=None, deleted=False, data=None)¶ Bases:
object
DBF record.
Instances of this class shouldn’t be created manualy, use dbf.Dbf.newRecord instead.
Class implements mapping/sequence interface, so fields could be accessed via their names or indexes (names is a preffered way to access fields).
Hint
Use store method to save modified record.
Examples
- Add new record to the database:
- db = Dbf(filename) rec = db.newRecord() rec[“FIELD1”] = value1 rec[“FIELD2”] = value2 rec.store()
Or the same, but modify existed (second in this case) record:
db = Dbf(filename) rec = db[2] rec[“FIELD1”] = value1 rec[“FIELD2”] = value2 rec.store()-
asDict
()¶ Return a dictionary of fields.
Note
Change of the dicts’s values won’t change real values stored in this object.
-
asList
()¶ Return a flat list of fields.
Note
Change of the list’s values won’t change real values stored in this object.
-
dbf
¶
-
delete
()¶ Mark method as deleted.
-
deleted
¶
-
fieldData
¶
-
classmethod
fromStream
(dbf, index)¶ Return a record read from the stream.
Parameters: - dbf – A Dbf.Dbf instance new record should belong to.
- index – Index of the record in the records’ container. This argument can’t be None in this call.
Return value is an instance of the current class.
-
classmethod
fromString
(dbf, string, index=None)¶ Return record read from the string object.
Parameters: - dbf – A Dbf.Dbf instance new record should belong to.
- string – A string new record should be created from.
- index – Index of the record in the container. If this argument is None, record will be appended.
Return value is an instance of the current class.
-
index
¶
-
position
¶
-
classmethod
rawFromStream
(dbf, index)¶ Return raw record contents read from the stream.
Parameters: - dbf – A Dbf.Dbf instance containing the record.
- index – Index of the record in the records’ container. This argument can’t be None in this call.
Return value is a string containing record data in DBF format.
-
store
()¶ Store current record in the DBF.
If
self.index
is None, this record will be appended to the records of the DBF this records belongs to; or replaced otherwise.
-
toString
()¶ Return string packed record values.
String utilities.
-
class
natcap.invest.pygeoprocessing_0_3_3.dbfpy.utils.
classproperty
¶ Bases:
property
Works in the same way as a
property
, but for the classes.
-
natcap.invest.pygeoprocessing_0_3_3.dbfpy.utils.
getDate
(date=None)¶ Return datetime.date instance.
- Type of the
date
argument could be one of the following: - None:
- use current date value;
- datetime.date:
- this value will be returned;
- datetime.datetime:
- the result of the date.date() will be returned;
- string:
- assuming “%Y%m%d” or “%y%m%dd” format;
- number:
- assuming it’s a timestamp (returned for example by the time.time() call;
- sequence:
- assuming (year, month, day, …) sequence;
Additionaly, if
date
has callableticks
attribute, it will be used and result of the called would be treated as a timestamp value.- Type of the
-
natcap.invest.pygeoprocessing_0_3_3.dbfpy.utils.
getDateTime
(value=None)¶ Return datetime.datetime instance.
- Type of the
value
argument could be one of the following: - None:
- use current date value;
- datetime.date:
- result will be converted to the datetime.datetime instance using midnight;
- datetime.datetime:
value
will be returned as is;- string:
- * CURRENTLY NOT SUPPORTED *;
- number:
- assuming it’s a timestamp (returned for example by the time.time() call;
- sequence:
- assuming (year, month, day, …) sequence;
Additionaly, if
value
has callableticks
attribute, it will be used and result of the called would be treated as a timestamp value.- Type of the
-
natcap.invest.pygeoprocessing_0_3_3.dbfpy.utils.
unzfill
(str)¶ Return a string without ASCII NULs.
This function searchers for the first NUL (ASCII 0) occurance and truncates string till that position.
Assertions for geospatial testing.
-
natcap.invest.pygeoprocessing_0_3_3.testing.assertions.
assert_checksums_equal
(checksum_file, base_folder=None)¶ Assert all files in the checksum_file have the same checksum.
Checksum files could be created by pygeoprocessing.testing.utils.checksum_folder(), but this function may also assert the output of GNU md5sum or BSD md5. Either format (GNU or BSD) may be provided as input to this assertion function. Any files not included in checksum_file are ignored.
Parameters: - checksum_file (string) – the path to the snapshot file to use.
- base_folder=None (string or None) – the folder to refer to as the base path for files in the checksum_file. If None, the current working directory will be used.
Raises: AssertionError
– when a nonmatching md5sum is found.
-
natcap.invest.pygeoprocessing_0_3_3.testing.assertions.
assert_close
(value_a, value_b, rel_tol=1e-05, abs_tol=1e-08, msg=None)¶ Assert equality to an absolute tolerance.
Parameters: - value_a (int or float) – The first value to test.
- value_b (int or float) – The second value to test.
- rel_tol (int or float) – The relative numerical tolerance. If the relative difference of these values are less than this value, assertion will pass.
- abs_tol (int or float) – The absolute numerical tolerance. If the difference of the values being tested are less than this value, the assertion will pass.
- msg=None (string or None) – The assertion message to use if value_a and value_b are not found to be equal out to the target tolerance.
Returns: None.
Raises: AssertionError
– Raised when the values are not equal out to the- desired tolerance.
-
natcap.invest.pygeoprocessing_0_3_3.testing.assertions.
assert_csv_equal
(a_uri, b_uri, rel_tol=1e-05, abs_tol=1e-08)¶ Assert the equality of CSV files at a_uri and b_uri.
Tests if csv files a and b are ‘almost equal’ to each other on a per cell basis. Numeric cells are asserted to be equal within the given tolerance. Other cell types are asserted to be equal.
Parameters: - a_uri (string) – a URI to a csv file
- b_uri (string) – a URI to a csv file
- rel_tol (int or float) – The relative numerical tolerance allowed, if relative difference of values are less than this, assertion passes.
- abs_tol (int or float) – The absolute numerical tolerance allowed, if difference of values are less than this, assertion passes.
Raises: AssertionError
– Raised when the two CSV files are found to bedifferent.
Returns: None
-
natcap.invest.pygeoprocessing_0_3_3.testing.assertions.
assert_json_equal
(json_1_uri, json_2_uri)¶ Assert two JSON files against each other.
The two JSON files provided will be opened, read, and their contents will be asserted to be equal. If the two are found to be different, the diff of the two files will be printed.
Parameters: - json_1_uri (string) – a uri to a JSON file.
- json_2_uri (string) – a uri to a JSON file.
Raises: AssertionError
– Raised when the two JSON objects differ.Returns: None
-
natcap.invest.pygeoprocessing_0_3_3.testing.assertions.
assert_md5_equal
(uri, regression_hash)¶ Assert the MD5sum of a file against a regression MD5sum.
This method is a convenience method that uses
natcap.invest.testing.digest_file()
to determine the MD5sum of the file located at uri. It is functionally equivalent to calling:if digest_file(uri) != '<some md5sum>': raise AssertionError
Regression MD5sums can be calculated for you by using
natcap.invest.testing.digest_file()
or a system-level md5sum program.Parameters: - uri (string) – a string URI to the file to be tested.
- regression_hash (string) – a string md5sum to be tested against.
Raises: AssertionError
– Raised when the MD5sum of the file at uri- differs from the provided regression md5sum hash.
Returns: None
-
natcap.invest.pygeoprocessing_0_3_3.testing.assertions.
assert_rasters_equal
(a_uri, b_uri, rel_tol=1e-05, abs_tol=1e-08)¶ Assert te equality of rasters a and b out to the given tolerance.
This assertion method asserts the equality of these raster characteristics:
- Raster height and width
- The number of layers in the raster
- Each pixel value, such that the absolute difference between the
- pixels is less than tolerance.
- Projection
Parameters: - a_uri (string) – a URI to a GDAL dataset
- b_uri (string) – a URI to a GDAL dataset
- rel_tol (int or float) – the relative tolerance to which values should be asserted. This is a numerical tolerance, not the number of places to which a value should be rounded. If the relative value is below this value then the assert passes.
- abs_tol (int or float) – absolute tolerance to which values should be asserted. If absolute difference in values is below this value then the assert passes.
Returns: None
Raises: IOError
– Raised when one of the input files is not found on disk.AssertionError
– Raised when the two rasters are found to be not- equal to each other.
-
natcap.invest.pygeoprocessing_0_3_3.testing.assertions.
assert_text_equal
(text_1_uri, text_2_uri)¶ Assert that two text files are equal.
This comparison is done line-by-line.
Parameters: - text_1_uri (string) – a python string uri to a text file. Considered the file to be tested.
- text_2_uri (string) – a python string uri to a text file. Considered the regression file.
Raises: AssertionError
– Raised when a line differs in the two files.Returns: None
-
natcap.invest.pygeoprocessing_0_3_3.testing.assertions.
assert_vectors_equal
(a_uri, b_uri)¶ Assert that the vectors at a_uri and b_uri are equal to each other.
This assertion method asserts the equality of these vector characteristics:
- Number of layers in the vector
- Number of features in each layer
- Geometry type of the layer
- Feature geometry type
- Number of fields in each feature
- Name of each field
- Field values for each feature
- Projection
Parameters: - a_uri (string) – a URI to an OGR vector
- b_uri (string) – a URI to an OGR vector
Raises: IOError
– Raised if one of the input files is not found on disk.AssertionError
– Raised if the vectors are not found to be equal to- one another.
- Returns
- None
-
natcap.invest.pygeoprocessing_0_3_3.testing.assertions.
isclose
(val_a, val_b, rel_tol=1e-05, abs_tol=1e-08)¶ Assert that values are equal to the given tolerance.
Adapted from the python 3.5 standard library based on the specification found in PEP485.
Parameters: - val_a (int or float) – The first value to test
- val_b (int or float) – The second value to test
- rel_tol (int or float) – is the relative tolerance - it is the maximum allowed difference between a and b, relative to the larger absolute value of a or b. For example, to set a tolerance of 5%, pass rel_tol=0.05. The default tolerance is 1e-09, which assures that the two values are the same within about 9 decimal digits. rel_tol must be greater than zero.
- abs_tol (float) – is the minimum absolute tolerance - useful for comparisons near zero. abs_tol must be at least zero.
Returns: A boolean.
The sampledata module provides functions for creating raster and vector data, constants for assisting with the creation of data, and some sample spatial reference data.
-
natcap.invest.pygeoprocessing_0_3_3.testing.sampledata.
SRS_COLOMBIA
¶ An instance of the ReferenceData namedtuple for the Colombia projection.
-
natcap.invest.pygeoprocessing_0_3_3.testing.sampledata.
SRS_WILLAMETTE
¶ An instance of the ReferenceData namedtuple for the Willamette projection (UTM zone 10N)
-
natcap.invest.pygeoprocessing_0_3_3.testing.sampledata.
VECTOR_FIELD_TYPES
¶ A dictionary mapping string field names to OGR field types.
-
class
natcap.invest.pygeoprocessing_0_3_3.testing.sampledata.
ReferenceData
¶ A namedtuple with two attributes and a function:
- projection: The well-known-text projection string.
- origin: A two-tuple of ints or floats representing the origin
- of the projection in the cartesian plane.
- pixel_size: A function accepting one parameter (an int or float).
- Returns a two-tuple of the height and width of pixels in this projection.
-
class
natcap.invest.pygeoprocessing_0_3_3.testing.sampledata.
ReferenceData
(projection, origin, pixel_size) Bases:
tuple
-
origin
¶ Alias for field number 1
-
pixel_size
¶ Alias for field number 2
-
projection
¶ Alias for field number 0
-
-
natcap.invest.pygeoprocessing_0_3_3.testing.sampledata.
cleanup
(uri)¶ Remove the uri. If it’s a folder, recursively remove its contents.
-
natcap.invest.pygeoprocessing_0_3_3.testing.sampledata.
create_raster_on_disk
(band_matrices, origin, projection_wkt, nodata, pixel_size, datatype='auto', format='GTiff', dataset_opts=None, filename=None)¶ Create a GDAL raster on disk.
Parameters: - band_matrices (list of numpy.ndarray) – a list of 2D numpy matrices representing pixel values, one array per band to be created in the output raster. The index of the matrix will be the index of the corresponding band that is created.
- origin (tuple of numbers) – A 2-element tuple representing the origin of the pixel values in the raster. This must be a tuple of numbers.
- projection_wkt (string) – A string WKT represntation of the projection to use in the output raster.
- nodata (int or float) – The nodata value for the raster. If nodata is None, a nodata value will not be set.
- pixel_size (tuple of numbers) – A 2-element tuple representing the size of all pixels in the raster arranged in the order (x, y). Either or both of these values could be negative.
- datatype (int or 'auto') – A GDAL datatype. If ‘auto’, a reasonable datatype will be chosen based on the datatype of the numpy matrix.
- format='GTiff' (string) – The string driver name to use. Determines the output format of the raster. Defaults to GeoTiff. See http://www.gdal.org/formats_list.html for a list of available formats.
- dataset_opts=None (list of strings) – A list of strings to pass to the underlying GDAL driver for creating this raster. Possible options are usually format dependent. If None, no options will be passed to the driver. For a GTiff, the most common set of options will usually be [‘TILED=YES’]. GTiff block sizes default to 256 along an edge if not otherwise specified.
- filename=None (string) – If provided, the new raster should be created at this filepath. If None, a new temporary file will be created within your tempfile directory (within tempfile.gettempdir()) and this path will be returned from the function.
Notes
- Writes a raster created with the given options.
- File management is up to the user.
Returns: The string path to the new raster created on disk.
-
natcap.invest.pygeoprocessing_0_3_3.testing.sampledata.
create_vector_on_disk
(geometries, projection, fields=None, attributes=None, vector_format='GeoJSON', filename=None)¶ Create an OGR-compatible vector on disk in the target format.
Parameters: - geometries (list) – a list of Shapely objects.
- projection (string) – a WKT representation of the vector’s projection.
- fields (dict or None) – a python dictionary mapping string fieldname
to a string datatype representation of the target ogr fieldtype.
Example: {‘ws_id’: ‘int’}. See
pygeoprocessing.testing.sampledata.VECTOR_FIELD_TYPES.keys()
for the complete list of all allowed types. If None, the datatype will be determined automatically based on the types of the attribute values. - attributes (list of dicts) – a list of python dictionary mapping fieldname to field value. The field value’s type must match the type defined in the fields input. It is an error if it doesn’t.
- vector_format (string) – a python string indicating the OGR format to write. GeoJSON is pretty good for most things, but doesn’t handle multipolygons very well. ‘ESRI Shapefile’ is usually a good bet.
- filename=None (None or string) – None or a python string where the file should be saved. If None, the vector will be saved to a temporary folder.
Notes
- Writes a vector created with the given options.
- File management is up to the user.
Returns: A string filepath to the location of the vector on disk.
-
natcap.invest.pygeoprocessing_0_3_3.testing.sampledata.
dtype_index
(dtype)¶ Return the relative data complexity index of the datatype provided.
Parameters: dtype (numpy.dtype or int GDAL datatype) – The dtype to check. Returns: The data complexity index relative to the other numpy/gdal type pairs.
-
natcap.invest.pygeoprocessing_0_3_3.testing.sampledata.
make_geotransform
(x_len, y_len, origin)¶ Build an array of affine transformation coefficients.
Parameters: - x_len (int or float) – The length of a pixel along the x axis. Generally, this will be a positive value.
- y_len (int of float) – The length of a pixel along the y axis. Generally (in North-up rasters), this will be a negative value.
- origin (tuple of ints or floats) – The origin of the raster, a 2-element tuple.
Returns: [ Origin along x-axis, Length of a pixel along the x axis, 0.0, (this is the standard in north-up projections) Origin along y-axis, Length of a pixel along the y axis, 0.0 (this is the standard in north-up projections) ]
Return type: A 6-element list with this structure
-
natcap.invest.pygeoprocessing_0_3_3.testing.sampledata.
open_files_in_gis_browser
(file_list)¶ Open the specified geospatial files with the provided application or visualization method (qgis is the only supported method at the moment).
NOTE: This functionality does not appear to work on Darwin systems with QGIS installed via homebrew. In this case, QGIS will open, but will not show the files requested.
Parameters: file_list (list) – a list of string filepaths to geospatial files. Returns: Nothing.
-
natcap.invest.pygeoprocessing_0_3_3.testing.sampledata.
projection_wkt
(epsg_id)¶ Import a projection from an EPSG code.
Parameters: proj_id (int) – If an int, it’s an EPSG code Returns: A WKT projection string.
-
natcap.invest.pygeoprocessing_0_3_3.testing.scm.
checkout_svn
(local_path, remote_path, rev=None)¶ Check out (or update) an SVN repository to the target revision.
Parameters: - local_path (string) – The path to an SVN repository on disk.
- remote_path (string) – The path to the SVN repository to check out.
- rev=None (string or None) – The revision to check out. If None, the latest revision will be checked out.
-
natcap.invest.pygeoprocessing_0_3_3.testing.scm.
load_config
(config_file)¶ Load the SCM configuration from a JSON configuration file.
Expects that the config file has three values: {
“local”: <local path to repo>, “remote”: <path to remote repo>, “rev”: <target revision>}
-
natcap.invest.pygeoprocessing_0_3_3.testing.scm.
skip_if_data_missing
(repo_path)¶ Decorator for unittest.TestCase test functions. Raises SkipTest if the pygeoprocessing data repository has not been cloned.
Parameters: repo_path (string) – The path to an SVN data repo on disk. Returns: A reference to a decorated unittest.TestCase test method that will raise SkipTest when executed.
-
natcap.invest.pygeoprocessing_0_3_3.testing.utils.
checksum_folder
(workspace_uri, logfile_uri, style='GNU', ignore_exts=None)¶ Recurse through the workspace_uri and for every file in the workspace, record the filepath and md5sum to the logfile. Additional environment metadata will also be recorded to help debug down the road.
This output logfile will have two sections separated by a blank line. The first section will have relevant system information, with keys and values separated by ‘=’ and some whitespace.
This second section will identify the files we’re snapshotting and the md5sums of these files, separated by ‘::’ and some whitspace on each line. MD5sums are determined by calling natcap.testing.utils.digest_file().
Parameters: - workspace_uri (string) – A URI to the workspace to analyze
- logfile_uri (string) – A URI to the logfile to which md5sums and paths will be recorded.
- style='GNU' (string) – Either ‘GNU’ or ‘BSD’. Corresponds to the style of the output file.
- ignore_exts=None (list/set of strings or None) – Extensions of files to ignore when checksumming. If None, all files will be included. If a list or set of extension strings, any file with that extension will be skipped.
Returns: Nothing.
-
natcap.invest.pygeoprocessing_0_3_3.testing.utils.
digest_file
(filepath)¶ Get the MD5sum for a single file on disk. Files are read in a memory-efficient fashion.
Parameters: filepath (string) – a string path to the file or folder to be tested or a list of files to be analyzed. Returns: An md5sum of the input file
-
natcap.invest.pygeoprocessing_0_3_3.testing.utils.
digest_file_list
(filepath_list, ifdir='skip')¶ Create a single MD5sum from all the files in filepath_list.
This is done by creating an MD5sum from the string MD5sums calculated from each individual file in the list. The filepath_list will be sorted before generating an MD5sum. If a given file is in this list multiple times, it will be double-counted.
Note
When passed a list with a single file in it, this function will produce a different MD5sum than if you were to simply take the md5sum of that single file. This is because this function produces an MD5sum of MD5sums.
Parameters: - filepath_list (list of strings) – A list of files to analyze.
- ifdir (string) – Either ‘skip’ or ‘raise’. Indicates what to do if a directory is encountered in this list. If ‘skip’, the directory skipped will be logged. If ‘raise’, IOError will be raised with the directory name.
Returns: A string MD5sum generated for all of the files in the list.
Raises: IOError
– When a file in filepath_list is a directory and ifdir == skip or a file could not be found.
-
natcap.invest.pygeoprocessing_0_3_3.testing.utils.
digest_folder
(folder)¶ Create a single MD5sum from all of the files in a folder. This recurses through folder and will take the MD5sum of all files found within.
Parameters: folder (string) – A string path to a folder on disk. Note
When there is a single file within this folder, the return value of this function will be different than if you were to take the MD5sum of just that one file. This is because we are taking an MD5sum of MD5sums.
Returns: - A string MD5sum generated from all of the files contained in
- this folder.
The testing subpackage provides reasonable testing functionality for building programmatic tests with and of geospatial data. It provides functions to generate input data of both raster and vector formats, and offers assertions to verify the correctness of geospatial outputs.
Most useful features of this package have been exposed at the pygeoprocessing.testing level.
Select locations have been chosen for their spatial references. These
references are available in the sampledata
module and al have the prefix
SRS_
:
from pygeoprocessing.testing import sampledata
sampledata.SRS_WILLAMETTE
sampledata.SRS_COLOMBIA
For writing tests, write them as you normally do! Assertions are generally
file-based and raise AssertionError
when a failure is encountered.
This example is relatively simplistic, since there will often be many more assertions you may need to make to be able to test your model effectively:
import unittest
import pygeoprocessing.testing
import natcap.invest.example_model
class ExampleTest(unittest.TestCase):
def test_some_model(self):
example_args = {
'workspace_dir': './workspace',
'arg_1': 'foo',
'arg_2': 'bar',
}
natcap.invest.example_model.execute(example_args)
# example assertion
pygeoprocessing.testing.assert_rasters_equal('workspace/raster_1.tif',
'regression_data/raster_1.tif')
-
natcap.invest.pygeoprocessing_0_3_3.testing.
create_raster_on_disk
(band_matrices, origin, projection_wkt, nodata, pixel_size, datatype='auto', format='GTiff', dataset_opts=None, filename=None)¶ Create a GDAL raster on disk.
Parameters: - band_matrices (list of numpy.ndarray) – a list of 2D numpy matrices representing pixel values, one array per band to be created in the output raster. The index of the matrix will be the index of the corresponding band that is created.
- origin (tuple of numbers) – A 2-element tuple representing the origin of the pixel values in the raster. This must be a tuple of numbers.
- projection_wkt (string) – A string WKT represntation of the projection to use in the output raster.
- nodata (int or float) – The nodata value for the raster. If nodata is None, a nodata value will not be set.
- pixel_size (tuple of numbers) – A 2-element tuple representing the size of all pixels in the raster arranged in the order (x, y). Either or both of these values could be negative.
- datatype (int or 'auto') – A GDAL datatype. If ‘auto’, a reasonable datatype will be chosen based on the datatype of the numpy matrix.
- format='GTiff' (string) – The string driver name to use. Determines the output format of the raster. Defaults to GeoTiff. See http://www.gdal.org/formats_list.html for a list of available formats.
- dataset_opts=None (list of strings) – A list of strings to pass to the underlying GDAL driver for creating this raster. Possible options are usually format dependent. If None, no options will be passed to the driver. For a GTiff, the most common set of options will usually be [‘TILED=YES’]. GTiff block sizes default to 256 along an edge if not otherwise specified.
- filename=None (string) – If provided, the new raster should be created at this filepath. If None, a new temporary file will be created within your tempfile directory (within tempfile.gettempdir()) and this path will be returned from the function.
Notes
- Writes a raster created with the given options.
- File management is up to the user.
Returns: The string path to the new raster created on disk.
-
natcap.invest.pygeoprocessing_0_3_3.testing.
create_vector_on_disk
(geometries, projection, fields=None, attributes=None, vector_format='GeoJSON', filename=None)¶ Create an OGR-compatible vector on disk in the target format.
Parameters: - geometries (list) – a list of Shapely objects.
- projection (string) – a WKT representation of the vector’s projection.
- fields (dict or None) – a python dictionary mapping string fieldname
to a string datatype representation of the target ogr fieldtype.
Example: {‘ws_id’: ‘int’}. See
pygeoprocessing.testing.sampledata.VECTOR_FIELD_TYPES.keys()
for the complete list of all allowed types. If None, the datatype will be determined automatically based on the types of the attribute values. - attributes (list of dicts) – a list of python dictionary mapping fieldname to field value. The field value’s type must match the type defined in the fields input. It is an error if it doesn’t.
- vector_format (string) – a python string indicating the OGR format to write. GeoJSON is pretty good for most things, but doesn’t handle multipolygons very well. ‘ESRI Shapefile’ is usually a good bet.
- filename=None (None or string) – None or a python string where the file should be saved. If None, the vector will be saved to a temporary folder.
Notes
- Writes a vector created with the given options.
- File management is up to the user.
Returns: A string filepath to the location of the vector on disk.
-
natcap.invest.pygeoprocessing_0_3_3.testing.
digest_file
(filepath)¶ Get the MD5sum for a single file on disk. Files are read in a memory-efficient fashion.
Parameters: filepath (string) – a string path to the file or folder to be tested or a list of files to be analyzed. Returns: An md5sum of the input file
-
natcap.invest.pygeoprocessing_0_3_3.testing.
digest_file_list
(filepath_list, ifdir='skip')¶ Create a single MD5sum from all the files in filepath_list.
This is done by creating an MD5sum from the string MD5sums calculated from each individual file in the list. The filepath_list will be sorted before generating an MD5sum. If a given file is in this list multiple times, it will be double-counted.
Note
When passed a list with a single file in it, this function will produce a different MD5sum than if you were to simply take the md5sum of that single file. This is because this function produces an MD5sum of MD5sums.
Parameters: - filepath_list (list of strings) – A list of files to analyze.
- ifdir (string) – Either ‘skip’ or ‘raise’. Indicates what to do if a directory is encountered in this list. If ‘skip’, the directory skipped will be logged. If ‘raise’, IOError will be raised with the directory name.
Returns: A string MD5sum generated for all of the files in the list.
Raises: IOError
– When a file in filepath_list is a directory and ifdir == skip or a file could not be found.
-
natcap.invest.pygeoprocessing_0_3_3.testing.
digest_folder
(folder)¶ Create a single MD5sum from all of the files in a folder. This recurses through folder and will take the MD5sum of all files found within.
Parameters: folder (string) – A string path to a folder on disk. Note
When there is a single file within this folder, the return value of this function will be different than if you were to take the MD5sum of just that one file. This is because we are taking an MD5sum of MD5sums.
Returns: - A string MD5sum generated from all of the files contained in
- this folder.
-
natcap.invest.pygeoprocessing_0_3_3.testing.
checksum_folder
(workspace_uri, logfile_uri, style='GNU', ignore_exts=None)¶ Recurse through the workspace_uri and for every file in the workspace, record the filepath and md5sum to the logfile. Additional environment metadata will also be recorded to help debug down the road.
This output logfile will have two sections separated by a blank line. The first section will have relevant system information, with keys and values separated by ‘=’ and some whitespace.
This second section will identify the files we’re snapshotting and the md5sums of these files, separated by ‘::’ and some whitspace on each line. MD5sums are determined by calling natcap.testing.utils.digest_file().
Parameters: - workspace_uri (string) – A URI to the workspace to analyze
- logfile_uri (string) – A URI to the logfile to which md5sums and paths will be recorded.
- style='GNU' (string) – Either ‘GNU’ or ‘BSD’. Corresponds to the style of the output file.
- ignore_exts=None (list/set of strings or None) – Extensions of files to ignore when checksumming. If None, all files will be included. If a list or set of extension strings, any file with that extension will be skipped.
Returns: Nothing.
-
natcap.invest.pygeoprocessing_0_3_3.testing.
assert_close
(value_a, value_b, rel_tol=1e-05, abs_tol=1e-08, msg=None)¶ Assert equality to an absolute tolerance.
Parameters: - value_a (int or float) – The first value to test.
- value_b (int or float) – The second value to test.
- rel_tol (int or float) – The relative numerical tolerance. If the relative difference of these values are less than this value, assertion will pass.
- abs_tol (int or float) – The absolute numerical tolerance. If the difference of the values being tested are less than this value, the assertion will pass.
- msg=None (string or None) – The assertion message to use if value_a and value_b are not found to be equal out to the target tolerance.
Returns: None.
Raises: AssertionError
– Raised when the values are not equal out to the- desired tolerance.
-
natcap.invest.pygeoprocessing_0_3_3.testing.
isclose
(val_a, val_b, rel_tol=1e-05, abs_tol=1e-08)¶ Assert that values are equal to the given tolerance.
Adapted from the python 3.5 standard library based on the specification found in PEP485.
Parameters: - val_a (int or float) – The first value to test
- val_b (int or float) – The second value to test
- rel_tol (int or float) – is the relative tolerance - it is the maximum allowed difference between a and b, relative to the larger absolute value of a or b. For example, to set a tolerance of 5%, pass rel_tol=0.05. The default tolerance is 1e-09, which assures that the two values are the same within about 9 decimal digits. rel_tol must be greater than zero.
- abs_tol (float) – is the minimum absolute tolerance - useful for comparisons near zero. abs_tol must be at least zero.
Returns: A boolean.
-
natcap.invest.pygeoprocessing_0_3_3.testing.
assert_rasters_equal
(a_uri, b_uri, rel_tol=1e-05, abs_tol=1e-08)¶ Assert te equality of rasters a and b out to the given tolerance.
This assertion method asserts the equality of these raster characteristics:
- Raster height and width
- The number of layers in the raster
- Each pixel value, such that the absolute difference between the
- pixels is less than tolerance.
- Projection
Parameters: - a_uri (string) – a URI to a GDAL dataset
- b_uri (string) – a URI to a GDAL dataset
- rel_tol (int or float) – the relative tolerance to which values should be asserted. This is a numerical tolerance, not the number of places to which a value should be rounded. If the relative value is below this value then the assert passes.
- abs_tol (int or float) – absolute tolerance to which values should be asserted. If absolute difference in values is below this value then the assert passes.
Returns: None
Raises: IOError
– Raised when one of the input files is not found on disk.AssertionError
– Raised when the two rasters are found to be not- equal to each other.
-
natcap.invest.pygeoprocessing_0_3_3.testing.
assert_vectors_equal
(a_uri, b_uri)¶ Assert that the vectors at a_uri and b_uri are equal to each other.
This assertion method asserts the equality of these vector characteristics:
- Number of layers in the vector
- Number of features in each layer
- Geometry type of the layer
- Feature geometry type
- Number of fields in each feature
- Name of each field
- Field values for each feature
- Projection
Parameters: - a_uri (string) – a URI to an OGR vector
- b_uri (string) – a URI to an OGR vector
Raises: IOError
– Raised if one of the input files is not found on disk.AssertionError
– Raised if the vectors are not found to be equal to- one another.
- Returns
- None
-
natcap.invest.pygeoprocessing_0_3_3.testing.
assert_csv_equal
(a_uri, b_uri, rel_tol=1e-05, abs_tol=1e-08)¶ Assert the equality of CSV files at a_uri and b_uri.
Tests if csv files a and b are ‘almost equal’ to each other on a per cell basis. Numeric cells are asserted to be equal within the given tolerance. Other cell types are asserted to be equal.
Parameters: - a_uri (string) – a URI to a csv file
- b_uri (string) – a URI to a csv file
- rel_tol (int or float) – The relative numerical tolerance allowed, if relative difference of values are less than this, assertion passes.
- abs_tol (int or float) – The absolute numerical tolerance allowed, if difference of values are less than this, assertion passes.
Raises: AssertionError
– Raised when the two CSV files are found to bedifferent.
Returns: None
-
natcap.invest.pygeoprocessing_0_3_3.testing.
assert_md5_equal
(uri, regression_hash)¶ Assert the MD5sum of a file against a regression MD5sum.
This method is a convenience method that uses
natcap.invest.testing.digest_file()
to determine the MD5sum of the file located at uri. It is functionally equivalent to calling:if digest_file(uri) != '<some md5sum>': raise AssertionError
Regression MD5sums can be calculated for you by using
natcap.invest.testing.digest_file()
or a system-level md5sum program.Parameters: - uri (string) – a string URI to the file to be tested.
- regression_hash (string) – a string md5sum to be tested against.
Raises: AssertionError
– Raised when the MD5sum of the file at uri- differs from the provided regression md5sum hash.
Returns: None
-
natcap.invest.pygeoprocessing_0_3_3.testing.
assert_json_equal
(json_1_uri, json_2_uri)¶ Assert two JSON files against each other.
The two JSON files provided will be opened, read, and their contents will be asserted to be equal. If the two are found to be different, the diff of the two files will be printed.
Parameters: - json_1_uri (string) – a uri to a JSON file.
- json_2_uri (string) – a uri to a JSON file.
Raises: AssertionError
– Raised when the two JSON objects differ.Returns: None
-
natcap.invest.pygeoprocessing_0_3_3.testing.
assert_text_equal
(text_1_uri, text_2_uri)¶ Assert that two text files are equal.
This comparison is done line-by-line.
Parameters: - text_1_uri (string) – a python string uri to a text file. Considered the file to be tested.
- text_2_uri (string) – a python string uri to a text file. Considered the regression file.
Raises: AssertionError
– Raised when a line differs in the two files.Returns: None
-
natcap.invest.pygeoprocessing_0_3_3.testing.
assert_checksums_equal
(checksum_file, base_folder=None)¶ Assert all files in the checksum_file have the same checksum.
Checksum files could be created by pygeoprocessing.testing.utils.checksum_folder(), but this function may also assert the output of GNU md5sum or BSD md5. Either format (GNU or BSD) may be provided as input to this assertion function. Any files not included in checksum_file are ignored.
Parameters: - checksum_file (string) – the path to the snapshot file to use.
- base_folder=None (string or None) – the folder to refer to as the base path for files in the checksum_file. If None, the current working directory will be used.
Raises: AssertionError
– when a nonmatching md5sum is found.
-
class
natcap.invest.pygeoprocessing_0_3_3.fileio.
CSVDriver
(uri, fieldnames=None)¶ Bases:
natcap.invest.pygeoprocessing_0_3_3.fileio.TableDriverTemplate
The CSVDriver class is a subclass of TableDriverTemplate.
-
get_fieldnames
()¶ Return a list of strings containing the fieldnames.
-
get_file_object
(uri=None)¶ Return the library-specific file object by using the input uri. If uri is None, return use self.uri.
-
read_table
()¶ Return the table object with data built from the table using the file-specific package as necessary. Should return a list of dictionaries.
-
write_table
(table_list, uri=None, fieldnames=None)¶ Take the table_list input and write its contents to the appropriate URI. If uri == None, write the file to self.uri. Otherwise, write the table to uri (which may be a new file). If fieldnames == None, assume that the default fieldnames order will be used.
-
-
exception
natcap.invest.pygeoprocessing_0_3_3.fileio.
ColumnMissingFromTable
¶ Bases:
exceptions.KeyError
A custom exception for when a key is missing from a table.
More descriptive than just throwing a KeyError. This class inherits the KeyError exception, so any existing exception handling should still work properly.
-
class
natcap.invest.pygeoprocessing_0_3_3.fileio.
DBFDriver
(uri, fieldnames=None)¶ Bases:
natcap.invest.pygeoprocessing_0_3_3.fileio.TableDriverTemplate
The DBFDriver class is a subclass of TableDriverTemplate.
-
get_fieldnames
()¶ Return a list of strings containing the fieldnames.
-
get_file_object
(uri=None, read_only=True)¶ Return the library-specific file object by using the input uri. If uri is None, return use self.uri.
-
read_table
()¶ Return the table object with data built from the table using the file-specific package as necessary. Should return a list of dictionaries.
-
write_table
(table_list, uri=None, fieldnames=None)¶ Take the table_list input and write its contents to the appropriate URI. If uri == None, write the file to self.uri. Otherwise, write the table to uri (which may be a new file). If fieldnames == None, assume that the default fieldnames order will be used.
-
-
class
natcap.invest.pygeoprocessing_0_3_3.fileio.
TableDriverTemplate
(uri, fieldnames=None)¶ Bases:
object
The TableDriverTemplate classes provide a uniform, simple way to interact with specific tabular libraries. This allows us to interact with multiple filetypes in exactly the same way and in a uniform syntax. By extension, this also allows us to read and write to and from any desired table format as long as the appropriate TableDriver class has been implemented.
These driver classes exist for convenience, and though they can be accessed directly by the user, these classes provide only the most basic functionality. Other classes, such as the TableHandler class, use these drivers to provide a convenient layer of functionality to the end-user.
This class is merely a template to be subclassed for use with appropriate table filetype drivers. Instantiating this object will yield a functional object, but it won’t actually get you any relevant results.
-
get_fieldnames
()¶ Return a list of strings containing the fieldnames.
-
get_file_object
(uri=None)¶ Return the library-specific file object by using the input uri. If uri is None, return use self.uri.
-
read_table
()¶ Return the table object with data built from the table using the file-specific package as necessary. Should return a list of dictionaries.
-
write_table
(table_list, uri=None, fieldnames=None)¶ Take the table_list input and write its contents to the appropriate URI. If uri == None, write the file to self.uri. Otherwise, write the table to uri (which may be a new file). If fieldnames == None, assume that the default fieldnames order will be used.
-
-
class
natcap.invest.pygeoprocessing_0_3_3.fileio.
TableHandler
(uri, fieldnames=None)¶ Bases:
object
-
create_column
(column_name, position=None, default_value=0)¶ Create a new column in the internal table object with the name column_name. If position == None, it will be appended to the end of the fieldnames. Otherwise, the column will be inserted at index position. This function will also loop through the entire table object and create an entry with the default value of default_value.
Note that it’s up to the driver to actually add the field to the file on disk.
Returns nothing
-
find_driver
(uri, fieldnames=None)¶ Locate the driver needed for uri. Returns a driver object as documented by self.driver_types.
-
get_fieldnames
(case='lower')¶ Returns a python list of the original fieldnames, true to their original case.
- case=’lower’ - a python string representing the desired status of the
- fieldnames. ‘lower’ for lower case, ‘orig’ for original case.
returns a python list of strings.
-
get_map
(key_field, value_field)¶ Returns a python dictionary mapping values contained in key_field to values contained in value_field. If duplicate keys are found, they are overwritten in the output dictionary.
This is implemented as a dictionary comprehension on top of self.get_table_list(), so there shouldn’t be a need to reimplement this for each subclass of AbstractTableHandler.
If the table list has not been retrieved, it is retrieved before generating the map.
key_field - a python string. value_field - a python string.
returns a python dictionary mapping key_fields to value_fields.
-
get_table
()¶ Return the table list object.
-
get_table_dictionary
(key_field, include_key=True)¶ Returns a python dictionary mapping a key value to all values in that particular row dictionary (including the key field). If duplicate keys are found, the are overwritten in the output dictionary.
- key_field - a python string of the desired field value to be used as
- the key for the returned dictionary.
- include_key=True - a python boolean indicating whether the
- key_field provided should be included in each row_dictionary.
returns a python dictionary of dictionaries.
-
get_table_row
(key_field, key_value)¶ Return the first full row where the value of key_field is equivalent to key_value. Raises a KeyError if key_field does not exist.
key_field - a python string. key_value - a value of appropriate type for this field.
returns a python dictionary of the row, or None if the row does not exist.
-
set_field_mask
(regexp=None, trim=0, trim_place='front')¶ Set a mask for the table’s self.fieldnames. Any fieldnames that match regexp will have trim number of characters stripped off the front.
- regexp=None - a python string or None. If a python string, this
- will be a regular expression. If None, this represents no regular expression.
trim - a python int. trim_place - a string, either ‘front’ or ‘back’. Indicates where
the trim should take place.Returns nothing.
-
write_table
(table=None, uri=None)¶ Invoke the driver to save the table to disk. If table == None, self.table will be written, otherwise, the list of dictionaries passed in to table will be written. If uri is None, the table will be written to the table’s original uri, otherwise, the table object will be written to uri.
-
-
natcap.invest.pygeoprocessing_0_3_3.fileio.
get_free_space
(folder='/', unit='auto')¶ Get the free space on the drive/folder marked by folder. Returns a float of unit unit.
- folder - (optional) a string uri to a folder or drive on disk. Defaults
- to ‘/’ (‘C:’ on Windows’)
- unit - (optional) a string, one of [‘B’, ‘MB’, ‘GB’, ‘TB’, ‘auto’]. If
- ‘auto’, the unit returned will be automatically calculated based on available space. Defaults to ‘auto’.
returns a string marking the space free and the selected unit. Number is rounded to two decimal places.’
A collection of GDAL dataset and raster utilities.
-
class
natcap.invest.pygeoprocessing_0_3_3.geoprocessing.
AggregatedValues
(total, pixel_mean, hectare_mean, n_pixels, pixel_min, pixel_max)¶ Bases:
tuple
-
hectare_mean
¶ Alias for field number 2
-
n_pixels
¶ Alias for field number 3
-
pixel_max
¶ Alias for field number 5
-
pixel_mean
¶ Alias for field number 1
-
pixel_min
¶ Alias for field number 4
-
total
¶ Alias for field number 0
-
-
exception
natcap.invest.pygeoprocessing_0_3_3.geoprocessing.
DatasetUnprojected
¶ Bases:
exceptions.Exception
An exception in case a dataset is unprojected
-
exception
natcap.invest.pygeoprocessing_0_3_3.geoprocessing.
DifferentProjections
¶ Bases:
exceptions.Exception
An exception in case a set of datasets are not in the same projection
-
exception
natcap.invest.pygeoprocessing_0_3_3.geoprocessing.
SpatialExtentOverlapException
¶ Bases:
exceptions.Exception
- An exeception class for cases when datasets or datasources don’t overlap
- in space.
- Used to raise an exception if rasters, shapefiles, or both don’t overlap
- in regions that should.
-
natcap.invest.pygeoprocessing_0_3_3.geoprocessing.
aggregate_raster_values_uri
(raster_uri, shapefile_uri, shapefile_field=None, ignore_nodata=True, all_touched=False, polygons_might_overlap=True)¶ Collect stats on pixel values which lie within shapefile polygons.
Parameters: - raster_uri (string) – a uri to a raster. In order for hectare mean values to be accurate, this raster must be projected in meter units.
- shapefile_uri (string) – a uri to a OGR datasource that should overlap raster; raises an exception if not.
Keyword Arguments: - shapefile_field (string) – a string indicating which key in shapefile to associate the output dictionary values with whose values are associated with ints; if None dictionary returns a value over the entire shapefile region that intersects the raster.
- ignore_nodata – if operation == ‘mean’ then it does not account for nodata pixels when determining the pixel_mean, otherwise all pixels in the AOI are used for calculation of the mean. This does not affect hectare_mean which is calculated from the geometrical area of the feature.
- all_touched (boolean) – if true will account for any pixel whose geometry passes through the pixel, not just the center point
- polygons_might_overlap (boolean) – if True the function calculates aggregation coverage close to optimally by rasterizing sets of polygons that don’t overlap. However, this step can be computationally expensive for cases where there are many polygons. Setting this flag to False directs the function rasterize in one step.
Returns: - named tuple of the form
- (‘aggregate_values’, ‘total pixel_mean hectare_mean n_pixels
pixel_min pixel_max’)
Each of [sum pixel_mean hectare_mean] contains a dictionary that maps shapefile_field value to the total, pixel mean, hecatare mean, pixel max, and pixel min of the values under that feature. ‘n_pixels’ contains the total number of valid pixels used in that calculation. hectare_mean is None if raster_uri is unprojected.
Return type: result_tuple (tuple)
Raises: AttributeError
TypeError
OSError
-
natcap.invest.pygeoprocessing_0_3_3.geoprocessing.
align_dataset_list
(dataset_uri_list, dataset_out_uri_list, resample_method_list, out_pixel_size, mode, dataset_to_align_index, dataset_to_bound_index=None, aoi_uri=None, assert_datasets_projected=True, all_touched=False)¶ - Create a new list of datasets that are aligned based on a list of
- inputted datasets.
Take a list of dataset uris and generates a new set that is completely aligned with identical projections and pixel sizes.
Parameters: - dataset_uri_list (list) – a list of input dataset uris
- dataset_out_uri_list (list) – a parallel dataset uri list whose positions correspond to entries in dataset_uri_list
- resample_method_list (list) – a list of resampling methods for each output uri in dataset_out_uri list. Each element must be one of “nearest|bilinear|cubic|cubic_spline|lanczos”
- out_pixel_size – the output pixel size
- mode (string) – one of “union”, “intersection”, or “dataset” which defines how the output output extents are defined as either the union or intersection of the input datasets or to have the same bounds as an existing raster. If mode is “dataset” then dataset_to_bound_index must be defined
- dataset_to_align_index (int) – an int that corresponds to the position in one of the dataset_uri_lists that, if positive aligns the output rasters to fix on the upper left hand corner of the output datasets. If negative, the bounding box aligns the intersection/ union without adjustment.
- all_touched (boolean) – if True and an AOI is passed, the ALL_TOUCHED=TRUE option is passed to the RasterizeLayer function when determining the mask of the AOI.
Keyword Arguments: - dataset_to_bound_index – if mode is “dataset” then this index is used to indicate which dataset to define the output bounds of the dataset_out_uri_list
- aoi_uri (string) – a URI to an OGR datasource to be used for the aoi. Irrespective of the mode input, the aoi will be used to intersect the final bounding box.
Returns: None
-
natcap.invest.pygeoprocessing_0_3_3.geoprocessing.
assert_datasets_in_same_projection
(dataset_uri_list)¶ Assert that provided datasets are all in the same projection.
Tests if datasets represented by their uris are projected and in the same projection and raises an exception if not.
Parameters: dataset_uri_list (list) – (description)
Returns: True (otherwise exception raised)
Return type: is_true (boolean)
Raises: DatasetUnprojected
– if one of the datasets is unprojected.DifferentProjections
– if at least one of the datasets is in a different projection
-
natcap.invest.pygeoprocessing_0_3_3.geoprocessing.
assert_file_existance
(dataset_uri_list)¶ Assert that provided uris exist in filesystem.
Verify that the uris passed in the argument exist on the filesystem if not, raise an exeception indicating which files do not exist
Parameters: dataset_uri_list (list) – a list of relative or absolute file paths to validate Returns: None Raises: IOError
– if any files are not found
-
natcap.invest.pygeoprocessing_0_3_3.geoprocessing.
calculate_disjoint_polygon_set
(shapefile_uri)¶ Create a list of sets of polygons that don’t overlap.
Determining the minimal number of those sets is an np-complete problem so this is an approximation that builds up sets of maximal subsets.
Parameters: shapefile_uri (string) – a uri to an OGR shapefile to process Returns: list of sets of FIDs from shapefile_uri Return type: subset_list (list)
-
natcap.invest.pygeoprocessing_0_3_3.geoprocessing.
calculate_intersection_rectangle
(dataset_list, aoi=None)¶ Return bounding box of the intersection of all rasters in the list.
Parameters: dataset_list (list) – a list of GDAL datasets in the same projection and coordinate system Keyword Arguments: aoi – an OGR polygon datasource which may optionally also restrict the extents of the intersection rectangle based on its own extents. Returns: - a 4 element list that bounds the intersection of
- all the rasters in dataset_list. [left, top, right, bottom]
Return type: bounding_box (list) Raises: SpatialExtentOverlapException
– in cases where the dataset list and aoi don’t overlap.
-
natcap.invest.pygeoprocessing_0_3_3.geoprocessing.
calculate_raster_stats_uri
(dataset_uri)¶ Calculate min, max, stdev, and mean for all bands in dataset.
Parameters: dataset_uri (string) – a uri to a GDAL raster dataset that will be modified by having its band statistics set Returns: None
-
natcap.invest.pygeoprocessing_0_3_3.geoprocessing.
calculate_slope
(dem_dataset_uri, slope_uri, aoi_uri=None, process_pool=None)¶ Create slope raster from DEM raster.
Follows the algorithm described here: http://webhelp.esri.com/arcgiSDEsktop/9.3/index.cfm?TopicName=How%20Slope%20works
Parameters: - dem_dataset_uri (string) – a URI to a single band raster of z values.
- slope_uri (string) – a path to the output slope uri in percent.
Keyword Arguments: - aoi_uri (string) – a uri to an AOI input
- process_pool – a process pool for multiprocessing
Returns: None
-
natcap.invest.pygeoprocessing_0_3_3.geoprocessing.
clip_dataset_uri
(source_dataset_uri, aoi_datasource_uri, out_dataset_uri, assert_projections=True, process_pool=None, all_touched=False)¶ Clip raster dataset to bounding box of provided vector datasource aoi.
This function will clip source_dataset to the bounding box of the polygons in aoi_datasource and mask out the values in source_dataset outside of the AOI with the nodata values in source_dataset.
Parameters: - source_dataset_uri (string) – uri to single band GDAL dataset to clip
- aoi_datasource_uri (string) – uri to ogr datasource
- out_dataset_uri (string) – path to disk for the clipped datset
Keyword Arguments: - assert_projections (boolean) – a boolean value for whether the dataset needs to be projected
- process_pool – a process pool for multiprocessing
- all_touched (boolean) – if true the clip uses the option ALL_TOUCHED=TRUE when calling RasterizeLayer for AOI masking.
Returns: None
-
natcap.invest.pygeoprocessing_0_3_3.geoprocessing.
convolve_2d_uri
(signal_path, kernel_path, output_path)¶ Convolve 2D kernel over 2D signal.
Convolves the raster in kernel_path over signal_path. Nodata values are treated as 0.0 during the convolution and masked to nodata for the output result where signal_path has nodata.
Parameters: - signal_path (string) – a filepath to a gdal dataset that’s the source input.
- kernel_path (string) – a filepath to a gdal dataset that’s the source input.
- output_path (string) – a filepath to the gdal dataset that’s the convolution output of signal and kernel that is the same size and projection of signal_path. Any nodata pixels that align with signal_path will be set to nodata.
Returns: None
-
natcap.invest.pygeoprocessing_0_3_3.geoprocessing.
copy_datasource_uri
(shape_uri, copy_uri)¶ Create a copy of an ogr shapefile.
Parameters: - shape_uri (string) – a uri path to the ogr shapefile that is to be copied
- copy_uri (string) – a uri path for the destination of the copied shapefile
Returns: None
-
natcap.invest.pygeoprocessing_0_3_3.geoprocessing.
create_directories
(directory_list)¶ Make directories provided in list of path strings.
This function will create any of the directories in the directory list if possible and raise exceptions if something exception other than the directory previously existing occurs.
Parameters: directory_list (list) – a list of string uri paths Returns: None
-
natcap.invest.pygeoprocessing_0_3_3.geoprocessing.
create_raster_from_vector_extents
(xRes, yRes, format, nodata, rasterFile, shp)¶ Create a blank raster based on a vector file extent.
This code is adapted from http://trac.osgeo.org/gdal/wiki/FAQRaster#HowcanIcreateablankrasterbasedonavectorfilesextentsforusewithgdal_rasterizeGDAL1.8.0
Parameters: - xRes – the x size of a pixel in the output dataset must be a positive value
- yRes – the y size of a pixel in the output dataset must be a positive value
- format – gdal GDT pixel type
- nodata – the output nodata value
- rasterFile (string) – URI to file location for raster
- shp – vector shapefile to base extent of output raster on
Returns: - blank raster whose bounds fit within `shp`s bounding box
and features are equivalent to the passed in data
Return type: raster
-
natcap.invest.pygeoprocessing_0_3_3.geoprocessing.
create_raster_from_vector_extents_uri
(shapefile_uri, pixel_size, gdal_format, nodata_out_value, output_uri)¶ Create a blank raster based on a vector file extent.
A wrapper for create_raster_from_vector_extents
Parameters: - shapefile_uri (string) – uri to an OGR datasource to use as the extents of the raster
- pixel_size – size of output pixels in the projected units of shapefile_uri
- gdal_format – the raster pixel format, something like gdal.GDT_Float32
- nodata_out_value – the output nodata value
- output_uri (string) – the URI to write the gdal dataset
Returns: gdal dataset
Return type: dataset (gdal.Dataset)
-
natcap.invest.pygeoprocessing_0_3_3.geoprocessing.
create_rat
(dataset, attr_dict, column_name)¶ Create a raster attribute table.
Parameters: - dataset – a GDAL raster dataset to create the RAT for (…)
- attr_dict (dict) – a dictionary with keys that point to a primitive type {integer_id_1: value_1, … integer_id_n: value_n}
- column_name (string) – a string for the column name that maps the values
Returns: a GDAL raster dataset with an updated RAT
Return type: dataset (gdal.Dataset)
-
natcap.invest.pygeoprocessing_0_3_3.geoprocessing.
create_rat_uri
(dataset_uri, attr_dict, column_name)¶ Create a raster attribute table.
URI wrapper for create_rat.
Parameters: - dataset_uri (string) – a GDAL raster dataset to create the RAT for (…)
- attr_dict (dict) – a dictionary with keys that point to a primitive type {integer_id_1: value_1, … integer_id_n: value_n}
- column_name (string) – a string for the column name that maps the values
-
natcap.invest.pygeoprocessing_0_3_3.geoprocessing.
dictionary_to_point_shapefile
(dict_data, layer_name, output_uri)¶ Create a point shapefile from a dictionary.
- The point shapefile created is not projected and uses latitude and
- longitude for its geometry.
Parameters: - dict_data (dict) – a python dictionary with keys being unique id’s that point to sub-dictionarys that have key-value pairs. These inner key-value pairs will represent the field-value pair for the point features. At least two fields are required in the sub-dictionaries, All the keys in the sub dictionary should have the same name and order. All the values in the sub dictionary should have the same type ‘lati’ and ‘long’. These fields determine the geometry of the point 0 : {‘lati’:97, ‘long’:43, ‘field_a’:6.3, ‘field_b’:’Forest’,…}, 1 : {‘lati’:55, ‘long’:51, ‘field_a’:6.2, ‘field_b’:’Crop’,…}, 2 : {‘lati’:73, ‘long’:47, ‘field_a’:6.5, ‘field_b’:’Swamp’,…}
- layer_name (string) – a python string for the name of the layer
- output_uri (string) – a uri for the output path of the point shapefile
Returns: None
-
natcap.invest.pygeoprocessing_0_3_3.geoprocessing.
distance_transform_edt
(input_mask_uri, output_distance_uri, process_pool=None)¶ Find the Euclidean distance transform on input_mask_uri and output the result as raster.
Parameters: - input_mask_uri (string) – a gdal raster to calculate distance from the non 0 value pixels
- output_distance_uri (string) – will make a float raster w/ same dimensions and projection as input_mask_uri where all zero values of input_mask_uri are equal to the euclidean distance to the closest non-zero pixel.
Keyword Arguments: process_pool – (description)
Returns: None
-
natcap.invest.pygeoprocessing_0_3_3.geoprocessing.
extract_datasource_table_by_key
(datasource_uri, key_field)¶ Return vector attribute table of first layer as dictionary.
Create a dictionary lookup table of the features in the attribute table of the datasource referenced by datasource_uri.
Parameters: - datasource_uri (string) – a uri to an OGR datasource
- key_field – a field in datasource_uri that refers to a key value for each row such as a polygon id.
Returns: - returns a dictionary of the
form {key_field_0: {field_0: value0, field_1: value1}…}
Return type: attribute_dictionary (dict)
-
natcap.invest.pygeoprocessing_0_3_3.geoprocessing.
get_bounding_box
(dataset_uri)¶ Get bounding box where coordinates are in projected units.
Parameters: dataset_uri (string) – a uri to a GDAL dataset Returns: [upper_left_x, upper_left_y, lower_right_x, lower_right_y] in projected coordinates Return type: bounding_box (list)
-
natcap.invest.pygeoprocessing_0_3_3.geoprocessing.
get_cell_size_from_uri
(dataset_uri)¶ Get the cell size of a dataset in units of meters.
Raises an exception if the raster is not square since this’ll break most of the pygeoprocessing algorithms.
Parameters: dataset_uri (string) – uri to a gdal dataset Returns: cell size of the dataset in meters Return type: size_meters
-
natcap.invest.pygeoprocessing_0_3_3.geoprocessing.
get_dataset_projection_wkt_uri
(dataset_uri)¶ Get the projection of a GDAL dataset as well known text (WKT).
Parameters: dataset_uri (string) – a URI for the GDAL dataset Returns: WKT describing the GDAL dataset project Return type: proj_wkt (string)
-
natcap.invest.pygeoprocessing_0_3_3.geoprocessing.
get_datasource_bounding_box
(datasource_uri)¶ Get datasource bounding box where coordinates are in projected units.
Parameters: dataset_uri (string) – a uri to a GDAL dataset Returns: [upper_left_x, upper_left_y, lower_right_x, lower_right_y] in projected coordinates Return type: bounding_box (list)
-
natcap.invest.pygeoprocessing_0_3_3.geoprocessing.
get_datatype_from_uri
(dataset_uri)¶ Return datatype for first band in gdal dataset.
Parameters: dataset_uri (string) – a uri to a gdal dataset Returns: datatype for dataset band 1 Return type: datatype
-
natcap.invest.pygeoprocessing_0_3_3.geoprocessing.
get_geotransform_uri
(dataset_uri)¶ Get the geotransform from a gdal dataset.
Parameters: dataset_uri (string) – a URI for the dataset Returns: a dataset geotransform list Return type: geotransform
-
natcap.invest.pygeoprocessing_0_3_3.geoprocessing.
get_lookup_from_csv
(csv_table_uri, key_field)¶ Read CSV table file in as dictionary.
Creates a python dictionary to look up the rest of the fields in a csv table indexed by the given key_field
Parameters: - csv_table_uri (string) – a URI to a csv file containing at least the header key_field
- key_field – (description)
Returns: - returns a dictionary of the form {key_field_0:
{header_1: val_1_0, header_2: val_2_0, etc.} depending on the values of those fields
Return type: lookup_dict (dict)
-
natcap.invest.pygeoprocessing_0_3_3.geoprocessing.
get_lookup_from_table
(table_uri, key_field)¶ Read table file in as dictionary.
Creates a python dictionary to look up the rest of the fields in a table file indexed by the given key_field. This function is case insensitive to field header names and returns a lookup table with lowercase keys.
Parameters: - table_uri (string) – a URI to a dbf or csv file containing at least the header key_field
- key_field – (description)
Returns: - a dictionary of the form {key_field_0:
{header_1: val_1_0, header_2: val_2_0, etc.} where key_field_n is the lowercase version of the column name.
Return type: lookup_dict (dict)
-
natcap.invest.pygeoprocessing_0_3_3.geoprocessing.
get_nodata_from_uri
(dataset_uri)¶ Return nodata value from first band in gdal dataset cast as numpy datatype.
Parameters: dataset_uri (string) – a uri to a gdal dataset Returns: nodata value for dataset band 1 Return type: nodata
-
natcap.invest.pygeoprocessing_0_3_3.geoprocessing.
get_raster_properties
(dataset)¶ Get width, height, X size, and Y size of the dataset as dictionary.
This function can be expanded to return more properties if needed
Parameters: dataset (gdal.Dataset) – a GDAL raster dataset to get the properties from Returns: - a dictionary with the properties stored
- under relevant keys. The current list of things returned is: width (w-e pixel resolution), height (n-s pixel resolution), XSize, YSize
Return type: dataset_dict (dictionary)
-
natcap.invest.pygeoprocessing_0_3_3.geoprocessing.
get_raster_properties_uri
(dataset_uri)¶ Get width, height, X size, and Y size of the dataset as dictionary.
Wrapper function for get_raster_properties() that passes in the dataset URI instead of the datasets itself
Parameters: dataset_uri (string) – a URI to a GDAL raster dataset Returns: - a dictionary with the properties stored under
- relevant keys. The current list of things returned is: width (w-e pixel resolution), height (n-s pixel resolution), XSize, YSize
Return type: value (dictionary)
-
natcap.invest.pygeoprocessing_0_3_3.geoprocessing.
get_rat_as_dictionary
(dataset)¶ Get Raster Attribute Table of the first band of dataset as a dictionary.
Parameters: dataset (gdal.Dataset) – a GDAL dataset that has a RAT associated with the first band Returns: - a 2D dictionary where the first key is the
- column name and second is the row number
Return type: rat_dictionary (dictionary)
-
natcap.invest.pygeoprocessing_0_3_3.geoprocessing.
get_rat_as_dictionary_uri
(dataset_uri)¶ Get Raster Attribute Table of the first band of dataset as a dictionary.
Parameters: dataset (string) – a GDAL dataset that has a RAT associated with the first band Returns: - a 2D dictionary where the first key is the column
- name and second is the row number
Return type: value (dictionary)
-
natcap.invest.pygeoprocessing_0_3_3.geoprocessing.
get_row_col_from_uri
(dataset_uri)¶ Return number of rows and columns of given dataset uri as tuple.
Parameters: dataset_uri (string) – a uri to a gdal dataset Returns: 2-tuple (n_row, n_col) from dataset_uri Return type: rows_cols (tuple)
-
natcap.invest.pygeoprocessing_0_3_3.geoprocessing.
get_spatial_ref_uri
(datasource_uri)¶ Get the spatial reference of an OGR datasource.
Parameters: datasource_uri (string) – a URI to an ogr datasource Returns: a spatial reference Return type: spat_ref
-
natcap.invest.pygeoprocessing_0_3_3.geoprocessing.
get_statistics_from_uri
(dataset_uri)¶ Get the min, max, mean, stdev from first band in a GDAL Dataset.
Parameters: dataset_uri (string) – a uri to a gdal dataset Returns: min, max, mean, stddev Return type: statistics (tuple)
-
natcap.invest.pygeoprocessing_0_3_3.geoprocessing.
iterblocks
(raster_uri, band_list=None, largest_block=1048576, astype=None, offset_only=False)¶ Iterate across all the memory blocks in the input raster.
Result is a generator of block location information and numpy arrays.
This is especially useful when a single value needs to be derived from the pixel values in a raster, such as the sum total of all pixel values, or a sequence of unique raster values. In such cases, raster_local_op is overkill, since it writes out a raster.
As a generator, this can be combined multiple times with itertools.izip() to iterate ‘simultaneously’ over multiple rasters, though the user should be careful to do so only with prealigned rasters.
Parameters: - raster_uri (string) – The string filepath to the raster to iterate over.
- band_list=None (list of ints or None) – A list of the bands for which the matrices should be returned. The band number to operate on. Defaults to None, which will return all bands. Bands may be specified in any order, and band indexes may be specified multiple times. The blocks returned on each iteration will be in the order specified in this list.
- largest_block (int) – Attempts to iterate over raster blocks with this many elements. Useful in cases where the blocksize is relatively small, memory is available, and the function call overhead dominates the iteration. Defaults to 2**20. A value of anything less than the original blocksize of the raster will result in blocksizes equal to the original size.
- astype (list of numpy types) – If none, output blocks are in the native type of the raster bands. Otherwise this parameter is a list of len(band_list) length that contains the desired output types that iterblock generates for each band.
- offset_only (boolean) – defaults to False, if True iterblocks only returns offset dictionary and doesn’t read any binary data from the raster. This can be useful when iterating over writing to an output.
Returns: If offset_only is false, on each iteration, a tuple containing a dict of block data and n 2-dimensional numpy arrays are returned, where n is the number of bands requested via band_list. The dict of block data has these attributes:
- data[‘xoff’] - The X offset of the upper-left-hand corner of the
block.
- data[‘yoff’] - The Y offset of the upper-left-hand corner of the
block.
data[‘win_xsize’] - The width of the block. data[‘win_ysize’] - The height of the block.
- If offset_only is True, the function returns only the block data and
does not attempt to read binary data from the raster.
-
natcap.invest.pygeoprocessing_0_3_3.geoprocessing.
load_memory_mapped_array
(dataset_uri, memory_file, array_type=None)¶ Get the first band of a dataset as a memory mapped array.
Parameters: - dataset_uri (string) – the GDAL dataset to load into a memory mapped array
- memory_uri (string) – a path to a file OR a file-like object that will be used to hold the memory map. It is up to the caller to create and delete this file.
Keyword Arguments: array_type – the type of the resulting array, if None defaults to the type of the raster band in the dataset
Returns: - a memmap numpy array of the data
contained in the first band of dataset_uri
Return type: memory_array (memmap numpy array)
-
natcap.invest.pygeoprocessing_0_3_3.geoprocessing.
make_constant_raster_from_base_uri
(base_dataset_uri, constant_value, out_uri, nodata_value=None, dataset_type=<Mock id='140530832174160'>)¶ Create new gdal raster filled with uniform values.
A helper function that creates a new gdal raster from base, and fills it with the constant value provided.
Parameters: - base_dataset_uri (string) – the gdal base raster
- constant_value – the value to set the new base raster to
- out_uri (string) – the uri of the output raster
Keyword Arguments: - nodata_value – the value to set the constant raster’s nodata value to. If not specified, it will be set to constant_value - 1.0
- dataset_type – the datatype to set the dataset to, default will be a float 32 value.
Returns: None
-
natcap.invest.pygeoprocessing_0_3_3.geoprocessing.
new_raster
(cols, rows, projection, geotransform, format, nodata, datatype, bands, outputURI)¶ Create a new raster with the given properties.
Parameters: - cols (int) – number of pixel columns
- rows (int) – number of pixel rows
- projection – the datum
- geotransform – the coordinate system
- format (string) – a string representing the GDAL file format of the output raster. See http://gdal.org/formats_list.html for a list of available formats. This parameter expects the format code, such as ‘GTiff’ or ‘MEM’
- nodata – a value that will be set as the nodata value for the output raster. Should be the same type as ‘datatype’
- datatype – the pixel datatype of the output raster, for example gdal.GDT_Float32. See the following header file for supported pixel types: http://www.gdal.org/gdal_8h.html#22e22ce0a55036a96f652765793fb7a4
- bands (int) – the number of bands in the raster
- outputURI (string) – the file location for the outputed raster. If format is ‘MEM’ this can be an empty string
Returns: a new GDAL raster with the parameters as described above
Return type: dataset
-
natcap.invest.pygeoprocessing_0_3_3.geoprocessing.
new_raster_from_base
(base, output_uri, gdal_format, nodata, datatype, fill_value=None, n_rows=None, n_cols=None, dataset_options=None)¶ Create a new, empty GDAL raster dataset with the spatial references, geotranforms of the base GDAL raster dataset.
Parameters: - base – a the GDAL raster dataset to base output size, and transforms on
- output_uri (string) – a string URI to the new output raster dataset.
- gdal_format (string) – a string representing the GDAL file format of the output raster. See http://gdal.org/formats_list.html for a list of available formats. This parameter expects the format code, such as ‘GTiff’ or ‘MEM’
- nodata – a value that will be set as the nodata value for the output raster. Should be the same type as ‘datatype’
- datatype – the pixel datatype of the output raster, for example gdal.GDT_Float32. See the following header file for supported pixel types: http://www.gdal.org/gdal_8h.html#22e22ce0a55036a96f652765793fb7a4
Keyword Arguments: - fill_value – the value to fill in the raster on creation
- n_rows – if set makes the resulting raster have n_rows in it if not, the number of rows of the outgoing dataset are equal to the base.
- n_cols – similar to n_rows, but for the columns.
- dataset_options – a list of dataset options that gets passed to the gdal creation driver, overrides defaults
Returns: a new GDAL raster dataset.
Return type: dataset
-
natcap.invest.pygeoprocessing_0_3_3.geoprocessing.
new_raster_from_base_uri
(base_uri, output_uri, gdal_format, nodata, datatype, fill_value=None, n_rows=None, n_cols=None, dataset_options=None)¶ Create a new, empty GDAL raster dataset with the spatial references, geotranforms of the base GDAL raster dataset.
A wrapper for the function new_raster_from_base that opens up the base_uri before passing it to new_raster_from_base.
Parameters: - base_uri (string) – a URI to a GDAL dataset on disk.
- output_uri (string) – a string URI to the new output raster dataset.
- gdal_format (string) – a string representing the GDAL file format of the output raster. See http://gdal.org/formats_list.html for a list of available formats. This parameter expects the format code, such as ‘GTiff’ or ‘MEM’
- nodata – a value that will be set as the nodata value for the output raster. Should be the same type as ‘datatype’
- datatype – the pixel datatype of the output raster, for example gdal.GDT_Float32. See the following header file for supported pixel types: http://www.gdal.org/gdal_8h.html#22e22ce0a55036a96f652765793fb7a4
Keyword Arguments: - fill_value – the value to fill in the raster on creation
- n_rows – if set makes the resulting raster have n_rows in it if not, the number of rows of the outgoing dataset are equal to the base.
- n_cols – similar to n_rows, but for the columns.
- dataset_options – a list of dataset options that gets passed to the gdal creation driver, overrides defaults
Returns: nothing
-
natcap.invest.pygeoprocessing_0_3_3.geoprocessing.
pixel_size_based_on_coordinate_transform
(dataset, coord_trans, point)¶ Get width and height of cell in meters.
Calculates the pixel width and height in meters given a coordinate transform and reference point on the dataset that’s close to the transform’s projected coordinate sytem. This is only necessary if dataset is not already in a meter coordinate system, for example dataset may be in lat/long (WGS84).
Parameters: - dataset (gdal.Dataset) – a projected GDAL dataset in the form of lat/long decimal degrees
- coord_trans (osr.CoordinateTransformation) – an OSR coordinate transformation from dataset coordinate system to meters
- point (tuple) – a reference point close to the coordinate transform coordinate system. must be in the same coordinate system as dataset.
Returns: - a 2-tuple containing (pixel width in meters, pixel
height in meters)
Return type: pixel_diff (tuple)
-
natcap.invest.pygeoprocessing_0_3_3.geoprocessing.
pixel_size_based_on_coordinate_transform_uri
(dataset_uri, *args, **kwargs)¶ Get width and height of cell in meters.
A wrapper for pixel_size_based_on_coordinate_transform that takes a dataset uri as an input and opens it before sending it along.
Parameters: - dataset_uri (string) – a URI to a gdal dataset
- other parameters pass along (All) –
Returns: (pixel_width_meters, pixel_height_meters)
Return type: result (tuple)
-
natcap.invest.pygeoprocessing_0_3_3.geoprocessing.
rasterize_layer_uri
(raster_uri, shapefile_uri, burn_values=[], option_list=[])¶ Rasterize datasource layer.
Burn the layer from ‘shapefile_uri’ onto the raster from ‘raster_uri’. Will burn ‘burn_value’ onto the raster unless ‘field’ is not None, in which case it will burn the value from shapefiles field.
Parameters: - raster_uri (string) – a URI to a gdal dataset
- shapefile_uri (string) – a URI to an ogr datasource
Keyword Arguments: - burn_values (list) – the equivalent value for burning into a polygon. If empty uses the Z values.
- option_list (list) – a Python list of options for the operation. Example: [“ATTRIBUTE=NPV”, “ALL_TOUCHED=TRUE”]
Returns: None
-
natcap.invest.pygeoprocessing_0_3_3.geoprocessing.
reclassify_dataset_uri
(dataset_uri, value_map, raster_out_uri, out_datatype, out_nodata, exception_flag='values_required', assert_dataset_projected=True)¶ Reclassify values in a dataset.
A function to reclassify values in dataset to any output type. By default the values except for nodata must be in value_map.
Parameters: - dataset_uri (string) – a uri to a gdal dataset
- value_map (dictionary) – a dictionary of values of {source_value: dest_value, …} where source_value’s type is a postive integer type and dest_value is of type out_datatype.
- raster_out_uri (string) – the uri for the output raster
- out_datatype (gdal type) – the type for the output dataset
- out_nodata (numerical type) – the nodata value for the output raster. Must be the same type as out_datatype
Keyword Arguments: - exception_flag (string) – either ‘none’ or ‘values_required’. If ‘values_required’ raise an exception if there is a value in the raster that is not found in value_map
- assert_dataset_projected (boolean) – if True this operation will test if the input dataset is not projected and raise an exception if so.
Returns: nothing
Raises: Exception
– if exception_flag == ‘values_required’ and the value from ‘key_raster’ is not a key in ‘attr_dict’
-
natcap.invest.pygeoprocessing_0_3_3.geoprocessing.
reproject_dataset_uri
(original_dataset_uri, pixel_spacing, output_wkt, resampling_method, output_uri)¶ Reproject and resample GDAL dataset.
A function to reproject and resample a GDAL dataset given an output pixel size and output reference. Will use the datatype and nodata value from the original dataset.
Parameters: - original_dataset_uri (string) – a URI to a gdal Dataset to written to disk
- pixel_spacing – output dataset pixel size in projected linear units
- output_wkt – output project in Well Known Text
- resampling_method (string) – a string representing the one of the following resampling methods: “nearest|bilinear|cubic|cubic_spline|lanczos”
- output_uri (string) – location on disk to dump the reprojected dataset
Returns: None
-
natcap.invest.pygeoprocessing_0_3_3.geoprocessing.
reproject_datasource
(original_datasource, output_wkt, output_uri)¶ Reproject OGR DataSource object.
Changes the projection of an ogr datasource by creating a new shapefile based on the output_wkt passed in. The new shapefile then copies all the features and fields of the original_datasource as its own.
Parameters: - original_datasource – an ogr datasource
- output_wkt – the desired projection as Well Known Text (by layer.GetSpatialRef().ExportToWkt())
- output_uri (string) – the filepath to the output shapefile
Returns: the reprojected shapefile.
Return type: output_datasource
-
natcap.invest.pygeoprocessing_0_3_3.geoprocessing.
reproject_datasource_uri
(original_dataset_uri, output_wkt, output_uri)¶ Reproject OGR DataSource file.
URI wrapper for reproject_datasource that takes in the uri for the datasource that is to be projected instead of the datasource itself. This function directly calls reproject_datasource.
Parameters: - original_dataset_uri (string) – a uri to an ogr datasource
- output_wkt – the desired projection as Well Known Text (by layer.GetSpatialRef().ExportToWkt())
- output_uri (string) – the path to where the new shapefile should be written to disk.
Returns: None
-
natcap.invest.pygeoprocessing_0_3_3.geoprocessing.
resize_and_resample_dataset_uri
(original_dataset_uri, bounding_box, out_pixel_size, output_uri, resample_method)¶ Resize and resample the given dataset.
Parameters: - original_dataset_uri (string) – a GDAL dataset
- bounding_box (list) – [upper_left_x, upper_left_y, lower_right_x, lower_right_y]
- out_pixel_size – the pixel size in projected linear units
- output_uri (string) – the location of the new resampled GDAL dataset
- resample_method (string) – the resampling technique, one of “nearest|bilinear|cubic|cubic_spline|lanczos”
Returns: None
-
natcap.invest.pygeoprocessing_0_3_3.geoprocessing.
temporary_filename
(suffix='')¶ Get path to new temporary file that will be deleted on program exit.
Returns a temporary filename using mkstemp. The file is deleted on exit using the atexit register.
Keyword Arguments: suffix (string) – the suffix to be appended to the temporary file Returns: a unique temporary filename Return type: fname
-
natcap.invest.pygeoprocessing_0_3_3.geoprocessing.
temporary_folder
()¶ Get path to new temporary folder that will be deleted on program exit.
Returns a temporary folder using mkdtemp. The folder is deleted on exit using the atexit register.
Returns: an absolute, unique and temporary folder path. Return type: path (string)
-
natcap.invest.pygeoprocessing_0_3_3.geoprocessing.
tile_dataset_uri
(in_uri, out_uri, blocksize)¶ - Resample gdal dataset into tiled raster with blocks of blocksize X
- blocksize.
Parameters: - in_uri (string) – dataset to base data from
- out_uri (string) – output dataset
- blocksize (int) – defines the side of the square for the raster, this seems to have a lower limit of 16, but is untested
Returns: None
-
natcap.invest.pygeoprocessing_0_3_3.geoprocessing.
transform_bounding_box
(bounding_box, base_ref_wkt, new_ref_wkt, edge_samples=11)¶ Transform input bounding box to output projection.
This transform accounts for the fact that the reprojected square bounding box might be warped in the new coordinate system. To account for this, the function samples points along the original bounding box edges and attempts to make the largest bounding box around any transformed point on the edge whether corners or warped edges.
Parameters: - bounding_box (list) – a list of 4 coordinates in base_epsg coordinate system describing the bound in the order [xmin, ymin, xmax, ymax]
- base_ref_wkt (string) – the spatial reference of the input coordinate system in Well Known Text.
- new_ref_wkt (string) – the EPSG code of the desired output coordinate system in Well Known Text.
- edge_samples (int) – the number of interpolated points along each bounding box edge to sample along. A value of 2 will sample just the corners while a value of 3 will also sample the corners and the midpoint.
Returns: A list of the form [xmin, ymin, xmax, ymax] that describes the largest fitting bounding box around the original warped bounding box in new_epsg coordinate system.
-
natcap.invest.pygeoprocessing_0_3_3.geoprocessing.
unique_raster_values
(dataset)¶ Get list of unique integer values within given dataset.
Parameters: dataset – a gdal dataset of some integer type Returns: a list of dataset’s unique non-nodata values Return type: unique_list (list)
-
natcap.invest.pygeoprocessing_0_3_3.geoprocessing.
unique_raster_values_count
(dataset_uri, ignore_nodata=True)¶ Return a dict from unique int values in the dataset to their frequency.
Parameters: dataset_uri (string) – uri to a gdal dataset of some integer type Keyword Arguments: ignore_nodata (boolean) – if set to false, the nodata count is also included in the result Returns: values to count. Return type: itemfreq (dict)
-
natcap.invest.pygeoprocessing_0_3_3.geoprocessing.
unique_raster_values_uri
(dataset_uri)¶ Get list of unique integer values within given dataset.
Parameters: dataset_uri (string) – a uri to a gdal dataset of some integer type Returns: a list of dataset’s unique non-nodata values Return type: value (list)
-
natcap.invest.pygeoprocessing_0_3_3.geoprocessing.
vectorize_datasets
(dataset_uri_list, dataset_pixel_op, dataset_out_uri, datatype_out, nodata_out, pixel_size_out, bounding_box_mode, resample_method_list=None, dataset_to_align_index=None, dataset_to_bound_index=None, aoi_uri=None, assert_datasets_projected=True, process_pool=None, vectorize_op=True, datasets_are_pre_aligned=False, dataset_options=None, all_touched=False)¶ Apply local raster operation on stack of datasets.
This function applies a user defined function across a stack of datasets. It has functionality align the output dataset grid with one of the input datasets, output a dataset that is the union or intersection of the input dataset bounding boxes, and control over the interpolation techniques of the input datasets, if necessary. The datasets in dataset_uri_list must be in the same projection; the function will raise an exception if not.
Parameters: - dataset_uri_list (list) – a list of file uris that point to files that can be opened with gdal.Open.
- dataset_pixel_op (function) – arguments as there are elements in dataset_uri_list. The arguments can be treated as interpolated or actual pixel values from the input datasets and the function should calculate the output value for that pixel stack. The function is a parallel paradigmn and does not know the spatial position of the pixels in question at the time of the call. If the bounding_box_mode parameter is “union” then the values of input dataset pixels that may be outside their original range will be the nodata values of those datasets. Known bug: if dataset_pixel_op does not return a value in some cases the output dataset values are undefined even if the function does not crash or raise an exception.
- dataset_out_uri (string) – the uri of the output dataset. The projection will be the same as the datasets in dataset_uri_list.
- datatype_out – the GDAL output type of the output dataset
- nodata_out – the nodata value of the output dataset.
- pixel_size_out – the pixel size of the output dataset in projected coordinates.
- bounding_box_mode (string) – one of “union” or “intersection”, “dataset”. If union the output dataset bounding box will be the union of the input datasets. Will be the intersection otherwise. An exception is raised if the mode is “intersection” and the input datasets have an empty intersection. If dataset it will make a bounding box as large as the given dataset, if given dataset_to_bound_index must be defined.
Keyword Arguments: - resample_method_list (list) – a list of resampling methods for each output uri in dataset_out_uri list. Each element must be one of “nearest|bilinear|cubic|cubic_spline|lanczos”. If None, the default is “nearest” for all input datasets.
- dataset_to_align_index (int) – an int that corresponds to the position in one of the dataset_uri_lists that, if positive aligns the output rasters to fix on the upper left hand corner of the output datasets. If negative, the bounding box aligns the intersection/ union without adjustment.
- dataset_to_bound_index – if mode is “dataset” this indicates which dataset should be the output size.
- aoi_uri (string) – a URI to an OGR datasource to be used for the aoi. Irrespective of the mode input, the aoi will be used to intersect the final bounding box.
- assert_datasets_projected (boolean) – if True this operation will test if any datasets are not projected and raise an exception if so.
- process_pool – a process pool for multiprocessing
- vectorize_op (boolean) – if true the model will try to numpy.vectorize dataset_pixel_op. If dataset_pixel_op is designed to use maximize array broadcasting, set this parameter to False, else it may inefficiently invoke the function on individual elements.
- datasets_are_pre_aligned (boolean) – If this value is set to False this operation will first align and interpolate the input datasets based on the rules provided in bounding_box_mode, resample_method_list, dataset_to_align_index, and dataset_to_bound_index, if set to True the input dataset list must be aligned, probably by raster_utils.align_dataset_list
- dataset_options – this is an argument list that will be passed to the GTiff driver. Useful for blocksizes, compression, etc.
- all_touched (boolean) – if true the clip uses the option ALL_TOUCHED=TRUE when calling RasterizeLayer for AOI masking.
Returns: None
Raises: ValueError
– invalid input provided
-
natcap.invest.pygeoprocessing_0_3_3.geoprocessing.
vectorize_points
(shapefile, datasource_field, dataset, randomize_points=False, mask_convex_hull=False, interpolation='nearest')¶ Interpolate values in shapefile onto given raster.
Takes a shapefile of points and a field defined in that shapefile and interpolate the values in the points onto the given raster
Parameters: - shapefile – ogr datasource of points
- datasource_field – a field in shapefile
- dataset – a gdal dataset must be in the same projection as shapefile
Keyword Arguments: - randomize_points (boolean) – (description)
- mask_convex_hull (boolean) – (description)
- interpolation (string) – the interpolation method to use for scipy.interpolate.griddata(). Default is ‘nearest’
Returns: None
-
natcap.invest.pygeoprocessing_0_3_3.geoprocessing.
vectorize_points_uri
(shapefile_uri, field, output_uri, interpolation='nearest')¶ Interpolate values in shapefile onto given raster.
A wrapper function for pygeoprocessing.vectorize_points, that allows for uri passing.
Parameters: - shapefile_uri (string) – a uri path to an ogr shapefile
- field (string) – a string for the field name
- output_uri (string) – a uri path for the output raster
- interpolation (string) – interpolation method to use on points, default is ‘nearest’
Returns: None
__init__ module for pygeprocessing, imports all the geoprocessing functions into the pygeoprocessing namespace
-
natcap.invest.pygeoprocessing_0_3_3.
aggregate_raster_values_uri
(raster_uri, shapefile_uri, shapefile_field=None, ignore_nodata=True, all_touched=False, polygons_might_overlap=True)¶ Collect stats on pixel values which lie within shapefile polygons.
Parameters: - raster_uri (string) – a uri to a raster. In order for hectare mean values to be accurate, this raster must be projected in meter units.
- shapefile_uri (string) – a uri to a OGR datasource that should overlap raster; raises an exception if not.
Keyword Arguments: - shapefile_field (string) – a string indicating which key in shapefile to associate the output dictionary values with whose values are associated with ints; if None dictionary returns a value over the entire shapefile region that intersects the raster.
- ignore_nodata – if operation == ‘mean’ then it does not account for nodata pixels when determining the pixel_mean, otherwise all pixels in the AOI are used for calculation of the mean. This does not affect hectare_mean which is calculated from the geometrical area of the feature.
- all_touched (boolean) – if true will account for any pixel whose geometry passes through the pixel, not just the center point
- polygons_might_overlap (boolean) – if True the function calculates aggregation coverage close to optimally by rasterizing sets of polygons that don’t overlap. However, this step can be computationally expensive for cases where there are many polygons. Setting this flag to False directs the function rasterize in one step.
Returns: - named tuple of the form
- (‘aggregate_values’, ‘total pixel_mean hectare_mean n_pixels
pixel_min pixel_max’)
Each of [sum pixel_mean hectare_mean] contains a dictionary that maps shapefile_field value to the total, pixel mean, hecatare mean, pixel max, and pixel min of the values under that feature. ‘n_pixels’ contains the total number of valid pixels used in that calculation. hectare_mean is None if raster_uri is unprojected.
Return type: result_tuple (tuple)
Raises: AttributeError
TypeError
OSError
-
natcap.invest.pygeoprocessing_0_3_3.
align_dataset_list
(dataset_uri_list, dataset_out_uri_list, resample_method_list, out_pixel_size, mode, dataset_to_align_index, dataset_to_bound_index=None, aoi_uri=None, assert_datasets_projected=True, all_touched=False)¶ - Create a new list of datasets that are aligned based on a list of
- inputted datasets.
Take a list of dataset uris and generates a new set that is completely aligned with identical projections and pixel sizes.
Parameters: - dataset_uri_list (list) – a list of input dataset uris
- dataset_out_uri_list (list) – a parallel dataset uri list whose positions correspond to entries in dataset_uri_list
- resample_method_list (list) – a list of resampling methods for each output uri in dataset_out_uri list. Each element must be one of “nearest|bilinear|cubic|cubic_spline|lanczos”
- out_pixel_size – the output pixel size
- mode (string) – one of “union”, “intersection”, or “dataset” which defines how the output output extents are defined as either the union or intersection of the input datasets or to have the same bounds as an existing raster. If mode is “dataset” then dataset_to_bound_index must be defined
- dataset_to_align_index (int) – an int that corresponds to the position in one of the dataset_uri_lists that, if positive aligns the output rasters to fix on the upper left hand corner of the output datasets. If negative, the bounding box aligns the intersection/ union without adjustment.
- all_touched (boolean) – if True and an AOI is passed, the ALL_TOUCHED=TRUE option is passed to the RasterizeLayer function when determining the mask of the AOI.
Keyword Arguments: - dataset_to_bound_index – if mode is “dataset” then this index is used to indicate which dataset to define the output bounds of the dataset_out_uri_list
- aoi_uri (string) – a URI to an OGR datasource to be used for the aoi. Irrespective of the mode input, the aoi will be used to intersect the final bounding box.
Returns: None
-
natcap.invest.pygeoprocessing_0_3_3.
assert_datasets_in_same_projection
(dataset_uri_list)¶ Assert that provided datasets are all in the same projection.
Tests if datasets represented by their uris are projected and in the same projection and raises an exception if not.
Parameters: dataset_uri_list (list) – (description)
Returns: True (otherwise exception raised)
Return type: is_true (boolean)
Raises: DatasetUnprojected
– if one of the datasets is unprojected.DifferentProjections
– if at least one of the datasets is in a different projection
-
natcap.invest.pygeoprocessing_0_3_3.
assert_file_existance
(dataset_uri_list)¶ Assert that provided uris exist in filesystem.
Verify that the uris passed in the argument exist on the filesystem if not, raise an exeception indicating which files do not exist
Parameters: dataset_uri_list (list) – a list of relative or absolute file paths to validate Returns: None Raises: IOError
– if any files are not found
-
natcap.invest.pygeoprocessing_0_3_3.
calculate_disjoint_polygon_set
(shapefile_uri)¶ Create a list of sets of polygons that don’t overlap.
Determining the minimal number of those sets is an np-complete problem so this is an approximation that builds up sets of maximal subsets.
Parameters: shapefile_uri (string) – a uri to an OGR shapefile to process Returns: list of sets of FIDs from shapefile_uri Return type: subset_list (list)
-
natcap.invest.pygeoprocessing_0_3_3.
calculate_intersection_rectangle
(dataset_list, aoi=None)¶ Return bounding box of the intersection of all rasters in the list.
Parameters: dataset_list (list) – a list of GDAL datasets in the same projection and coordinate system Keyword Arguments: aoi – an OGR polygon datasource which may optionally also restrict the extents of the intersection rectangle based on its own extents. Returns: - a 4 element list that bounds the intersection of
- all the rasters in dataset_list. [left, top, right, bottom]
Return type: bounding_box (list) Raises: SpatialExtentOverlapException
– in cases where the dataset list and aoi don’t overlap.
-
natcap.invest.pygeoprocessing_0_3_3.
calculate_raster_stats_uri
(dataset_uri)¶ Calculate min, max, stdev, and mean for all bands in dataset.
Parameters: dataset_uri (string) – a uri to a GDAL raster dataset that will be modified by having its band statistics set Returns: None
-
natcap.invest.pygeoprocessing_0_3_3.
calculate_slope
(dem_dataset_uri, slope_uri, aoi_uri=None, process_pool=None)¶ Create slope raster from DEM raster.
Follows the algorithm described here: http://webhelp.esri.com/arcgiSDEsktop/9.3/index.cfm?TopicName=How%20Slope%20works
Parameters: - dem_dataset_uri (string) – a URI to a single band raster of z values.
- slope_uri (string) – a path to the output slope uri in percent.
Keyword Arguments: - aoi_uri (string) – a uri to an AOI input
- process_pool – a process pool for multiprocessing
Returns: None
-
natcap.invest.pygeoprocessing_0_3_3.
clip_dataset_uri
(source_dataset_uri, aoi_datasource_uri, out_dataset_uri, assert_projections=True, process_pool=None, all_touched=False)¶ Clip raster dataset to bounding box of provided vector datasource aoi.
This function will clip source_dataset to the bounding box of the polygons in aoi_datasource and mask out the values in source_dataset outside of the AOI with the nodata values in source_dataset.
Parameters: - source_dataset_uri (string) – uri to single band GDAL dataset to clip
- aoi_datasource_uri (string) – uri to ogr datasource
- out_dataset_uri (string) – path to disk for the clipped datset
Keyword Arguments: - assert_projections (boolean) – a boolean value for whether the dataset needs to be projected
- process_pool – a process pool for multiprocessing
- all_touched (boolean) – if true the clip uses the option ALL_TOUCHED=TRUE when calling RasterizeLayer for AOI masking.
Returns: None
-
natcap.invest.pygeoprocessing_0_3_3.
convolve_2d_uri
(signal_path, kernel_path, output_path)¶ Convolve 2D kernel over 2D signal.
Convolves the raster in kernel_path over signal_path. Nodata values are treated as 0.0 during the convolution and masked to nodata for the output result where signal_path has nodata.
Parameters: - signal_path (string) – a filepath to a gdal dataset that’s the source input.
- kernel_path (string) – a filepath to a gdal dataset that’s the source input.
- output_path (string) – a filepath to the gdal dataset that’s the convolution output of signal and kernel that is the same size and projection of signal_path. Any nodata pixels that align with signal_path will be set to nodata.
Returns: None
-
natcap.invest.pygeoprocessing_0_3_3.
copy_datasource_uri
(shape_uri, copy_uri)¶ Create a copy of an ogr shapefile.
Parameters: - shape_uri (string) – a uri path to the ogr shapefile that is to be copied
- copy_uri (string) – a uri path for the destination of the copied shapefile
Returns: None
-
natcap.invest.pygeoprocessing_0_3_3.
create_directories
(directory_list)¶ Make directories provided in list of path strings.
This function will create any of the directories in the directory list if possible and raise exceptions if something exception other than the directory previously existing occurs.
Parameters: directory_list (list) – a list of string uri paths Returns: None
-
natcap.invest.pygeoprocessing_0_3_3.
create_raster_from_vector_extents
(xRes, yRes, format, nodata, rasterFile, shp)¶ Create a blank raster based on a vector file extent.
This code is adapted from http://trac.osgeo.org/gdal/wiki/FAQRaster#HowcanIcreateablankrasterbasedonavectorfilesextentsforusewithgdal_rasterizeGDAL1.8.0
Parameters: - xRes – the x size of a pixel in the output dataset must be a positive value
- yRes – the y size of a pixel in the output dataset must be a positive value
- format – gdal GDT pixel type
- nodata – the output nodata value
- rasterFile (string) – URI to file location for raster
- shp – vector shapefile to base extent of output raster on
Returns: - blank raster whose bounds fit within `shp`s bounding box
and features are equivalent to the passed in data
Return type: raster
-
natcap.invest.pygeoprocessing_0_3_3.
create_raster_from_vector_extents_uri
(shapefile_uri, pixel_size, gdal_format, nodata_out_value, output_uri)¶ Create a blank raster based on a vector file extent.
A wrapper for create_raster_from_vector_extents
Parameters: - shapefile_uri (string) – uri to an OGR datasource to use as the extents of the raster
- pixel_size – size of output pixels in the projected units of shapefile_uri
- gdal_format – the raster pixel format, something like gdal.GDT_Float32
- nodata_out_value – the output nodata value
- output_uri (string) – the URI to write the gdal dataset
Returns: gdal dataset
Return type: dataset (gdal.Dataset)
-
natcap.invest.pygeoprocessing_0_3_3.
create_rat
(dataset, attr_dict, column_name)¶ Create a raster attribute table.
Parameters: - dataset – a GDAL raster dataset to create the RAT for (…)
- attr_dict (dict) – a dictionary with keys that point to a primitive type {integer_id_1: value_1, … integer_id_n: value_n}
- column_name (string) – a string for the column name that maps the values
Returns: a GDAL raster dataset with an updated RAT
Return type: dataset (gdal.Dataset)
-
natcap.invest.pygeoprocessing_0_3_3.
create_rat_uri
(dataset_uri, attr_dict, column_name)¶ Create a raster attribute table.
URI wrapper for create_rat.
Parameters: - dataset_uri (string) – a GDAL raster dataset to create the RAT for (…)
- attr_dict (dict) – a dictionary with keys that point to a primitive type {integer_id_1: value_1, … integer_id_n: value_n}
- column_name (string) – a string for the column name that maps the values
-
natcap.invest.pygeoprocessing_0_3_3.
dictionary_to_point_shapefile
(dict_data, layer_name, output_uri)¶ Create a point shapefile from a dictionary.
- The point shapefile created is not projected and uses latitude and
- longitude for its geometry.
Parameters: - dict_data (dict) – a python dictionary with keys being unique id’s that point to sub-dictionarys that have key-value pairs. These inner key-value pairs will represent the field-value pair for the point features. At least two fields are required in the sub-dictionaries, All the keys in the sub dictionary should have the same name and order. All the values in the sub dictionary should have the same type ‘lati’ and ‘long’. These fields determine the geometry of the point 0 : {‘lati’:97, ‘long’:43, ‘field_a’:6.3, ‘field_b’:’Forest’,…}, 1 : {‘lati’:55, ‘long’:51, ‘field_a’:6.2, ‘field_b’:’Crop’,…}, 2 : {‘lati’:73, ‘long’:47, ‘field_a’:6.5, ‘field_b’:’Swamp’,…}
- layer_name (string) – a python string for the name of the layer
- output_uri (string) – a uri for the output path of the point shapefile
Returns: None
-
natcap.invest.pygeoprocessing_0_3_3.
distance_transform_edt
(input_mask_uri, output_distance_uri, process_pool=None)¶ Find the Euclidean distance transform on input_mask_uri and output the result as raster.
Parameters: - input_mask_uri (string) – a gdal raster to calculate distance from the non 0 value pixels
- output_distance_uri (string) – will make a float raster w/ same dimensions and projection as input_mask_uri where all zero values of input_mask_uri are equal to the euclidean distance to the closest non-zero pixel.
Keyword Arguments: process_pool – (description)
Returns: None
-
natcap.invest.pygeoprocessing_0_3_3.
extract_datasource_table_by_key
(datasource_uri, key_field)¶ Return vector attribute table of first layer as dictionary.
Create a dictionary lookup table of the features in the attribute table of the datasource referenced by datasource_uri.
Parameters: - datasource_uri (string) – a uri to an OGR datasource
- key_field – a field in datasource_uri that refers to a key value for each row such as a polygon id.
Returns: - returns a dictionary of the
form {key_field_0: {field_0: value0, field_1: value1}…}
Return type: attribute_dictionary (dict)
-
natcap.invest.pygeoprocessing_0_3_3.
get_bounding_box
(dataset_uri)¶ Get bounding box where coordinates are in projected units.
Parameters: dataset_uri (string) – a uri to a GDAL dataset Returns: [upper_left_x, upper_left_y, lower_right_x, lower_right_y] in projected coordinates Return type: bounding_box (list)
-
natcap.invest.pygeoprocessing_0_3_3.
get_cell_size_from_uri
(dataset_uri)¶ Get the cell size of a dataset in units of meters.
Raises an exception if the raster is not square since this’ll break most of the pygeoprocessing algorithms.
Parameters: dataset_uri (string) – uri to a gdal dataset Returns: cell size of the dataset in meters Return type: size_meters
-
natcap.invest.pygeoprocessing_0_3_3.
get_dataset_projection_wkt_uri
(dataset_uri)¶ Get the projection of a GDAL dataset as well known text (WKT).
Parameters: dataset_uri (string) – a URI for the GDAL dataset Returns: WKT describing the GDAL dataset project Return type: proj_wkt (string)
-
natcap.invest.pygeoprocessing_0_3_3.
get_datasource_bounding_box
(datasource_uri)¶ Get datasource bounding box where coordinates are in projected units.
Parameters: dataset_uri (string) – a uri to a GDAL dataset Returns: [upper_left_x, upper_left_y, lower_right_x, lower_right_y] in projected coordinates Return type: bounding_box (list)
-
natcap.invest.pygeoprocessing_0_3_3.
get_datatype_from_uri
(dataset_uri)¶ Return datatype for first band in gdal dataset.
Parameters: dataset_uri (string) – a uri to a gdal dataset Returns: datatype for dataset band 1 Return type: datatype
-
natcap.invest.pygeoprocessing_0_3_3.
get_geotransform_uri
(dataset_uri)¶ Get the geotransform from a gdal dataset.
Parameters: dataset_uri (string) – a URI for the dataset Returns: a dataset geotransform list Return type: geotransform
-
natcap.invest.pygeoprocessing_0_3_3.
get_lookup_from_csv
(csv_table_uri, key_field)¶ Read CSV table file in as dictionary.
Creates a python dictionary to look up the rest of the fields in a csv table indexed by the given key_field
Parameters: - csv_table_uri (string) – a URI to a csv file containing at least the header key_field
- key_field – (description)
Returns: - returns a dictionary of the form {key_field_0:
{header_1: val_1_0, header_2: val_2_0, etc.} depending on the values of those fields
Return type: lookup_dict (dict)
-
natcap.invest.pygeoprocessing_0_3_3.
get_lookup_from_table
(table_uri, key_field)¶ Read table file in as dictionary.
Creates a python dictionary to look up the rest of the fields in a table file indexed by the given key_field. This function is case insensitive to field header names and returns a lookup table with lowercase keys.
Parameters: - table_uri (string) – a URI to a dbf or csv file containing at least the header key_field
- key_field – (description)
Returns: - a dictionary of the form {key_field_0:
{header_1: val_1_0, header_2: val_2_0, etc.} where key_field_n is the lowercase version of the column name.
Return type: lookup_dict (dict)
-
natcap.invest.pygeoprocessing_0_3_3.
get_nodata_from_uri
(dataset_uri)¶ Return nodata value from first band in gdal dataset cast as numpy datatype.
Parameters: dataset_uri (string) – a uri to a gdal dataset Returns: nodata value for dataset band 1 Return type: nodata
-
natcap.invest.pygeoprocessing_0_3_3.
get_raster_properties
(dataset)¶ Get width, height, X size, and Y size of the dataset as dictionary.
This function can be expanded to return more properties if needed
Parameters: dataset (gdal.Dataset) – a GDAL raster dataset to get the properties from Returns: - a dictionary with the properties stored
- under relevant keys. The current list of things returned is: width (w-e pixel resolution), height (n-s pixel resolution), XSize, YSize
Return type: dataset_dict (dictionary)
-
natcap.invest.pygeoprocessing_0_3_3.
get_raster_properties_uri
(dataset_uri)¶ Get width, height, X size, and Y size of the dataset as dictionary.
Wrapper function for get_raster_properties() that passes in the dataset URI instead of the datasets itself
Parameters: dataset_uri (string) – a URI to a GDAL raster dataset Returns: - a dictionary with the properties stored under
- relevant keys. The current list of things returned is: width (w-e pixel resolution), height (n-s pixel resolution), XSize, YSize
Return type: value (dictionary)
-
natcap.invest.pygeoprocessing_0_3_3.
get_rat_as_dictionary
(dataset)¶ Get Raster Attribute Table of the first band of dataset as a dictionary.
Parameters: dataset (gdal.Dataset) – a GDAL dataset that has a RAT associated with the first band Returns: - a 2D dictionary where the first key is the
- column name and second is the row number
Return type: rat_dictionary (dictionary)
-
natcap.invest.pygeoprocessing_0_3_3.
get_rat_as_dictionary_uri
(dataset_uri)¶ Get Raster Attribute Table of the first band of dataset as a dictionary.
Parameters: dataset (string) – a GDAL dataset that has a RAT associated with the first band Returns: - a 2D dictionary where the first key is the column
- name and second is the row number
Return type: value (dictionary)
-
natcap.invest.pygeoprocessing_0_3_3.
get_row_col_from_uri
(dataset_uri)¶ Return number of rows and columns of given dataset uri as tuple.
Parameters: dataset_uri (string) – a uri to a gdal dataset Returns: 2-tuple (n_row, n_col) from dataset_uri Return type: rows_cols (tuple)
-
natcap.invest.pygeoprocessing_0_3_3.
get_spatial_ref_uri
(datasource_uri)¶ Get the spatial reference of an OGR datasource.
Parameters: datasource_uri (string) – a URI to an ogr datasource Returns: a spatial reference Return type: spat_ref
-
natcap.invest.pygeoprocessing_0_3_3.
get_statistics_from_uri
(dataset_uri)¶ Get the min, max, mean, stdev from first band in a GDAL Dataset.
Parameters: dataset_uri (string) – a uri to a gdal dataset Returns: min, max, mean, stddev Return type: statistics (tuple)
-
natcap.invest.pygeoprocessing_0_3_3.
iterblocks
(raster_uri, band_list=None, largest_block=1048576, astype=None, offset_only=False)¶ Iterate across all the memory blocks in the input raster.
Result is a generator of block location information and numpy arrays.
This is especially useful when a single value needs to be derived from the pixel values in a raster, such as the sum total of all pixel values, or a sequence of unique raster values. In such cases, raster_local_op is overkill, since it writes out a raster.
As a generator, this can be combined multiple times with itertools.izip() to iterate ‘simultaneously’ over multiple rasters, though the user should be careful to do so only with prealigned rasters.
Parameters: - raster_uri (string) – The string filepath to the raster to iterate over.
- band_list=None (list of ints or None) – A list of the bands for which the matrices should be returned. The band number to operate on. Defaults to None, which will return all bands. Bands may be specified in any order, and band indexes may be specified multiple times. The blocks returned on each iteration will be in the order specified in this list.
- largest_block (int) – Attempts to iterate over raster blocks with this many elements. Useful in cases where the blocksize is relatively small, memory is available, and the function call overhead dominates the iteration. Defaults to 2**20. A value of anything less than the original blocksize of the raster will result in blocksizes equal to the original size.
- astype (list of numpy types) – If none, output blocks are in the native type of the raster bands. Otherwise this parameter is a list of len(band_list) length that contains the desired output types that iterblock generates for each band.
- offset_only (boolean) – defaults to False, if True iterblocks only returns offset dictionary and doesn’t read any binary data from the raster. This can be useful when iterating over writing to an output.
Returns: If offset_only is false, on each iteration, a tuple containing a dict of block data and n 2-dimensional numpy arrays are returned, where n is the number of bands requested via band_list. The dict of block data has these attributes:
- data[‘xoff’] - The X offset of the upper-left-hand corner of the
block.
- data[‘yoff’] - The Y offset of the upper-left-hand corner of the
block.
data[‘win_xsize’] - The width of the block. data[‘win_ysize’] - The height of the block.
- If offset_only is True, the function returns only the block data and
does not attempt to read binary data from the raster.
-
natcap.invest.pygeoprocessing_0_3_3.
load_memory_mapped_array
(dataset_uri, memory_file, array_type=None)¶ Get the first band of a dataset as a memory mapped array.
Parameters: - dataset_uri (string) – the GDAL dataset to load into a memory mapped array
- memory_uri (string) – a path to a file OR a file-like object that will be used to hold the memory map. It is up to the caller to create and delete this file.
Keyword Arguments: array_type – the type of the resulting array, if None defaults to the type of the raster band in the dataset
Returns: - a memmap numpy array of the data
contained in the first band of dataset_uri
Return type: memory_array (memmap numpy array)
-
natcap.invest.pygeoprocessing_0_3_3.
make_constant_raster_from_base_uri
(base_dataset_uri, constant_value, out_uri, nodata_value=None, dataset_type=<Mock id='140530832174160'>)¶ Create new gdal raster filled with uniform values.
A helper function that creates a new gdal raster from base, and fills it with the constant value provided.
Parameters: - base_dataset_uri (string) – the gdal base raster
- constant_value – the value to set the new base raster to
- out_uri (string) – the uri of the output raster
Keyword Arguments: - nodata_value – the value to set the constant raster’s nodata value to. If not specified, it will be set to constant_value - 1.0
- dataset_type – the datatype to set the dataset to, default will be a float 32 value.
Returns: None
-
natcap.invest.pygeoprocessing_0_3_3.
new_raster
(cols, rows, projection, geotransform, format, nodata, datatype, bands, outputURI)¶ Create a new raster with the given properties.
Parameters: - cols (int) – number of pixel columns
- rows (int) – number of pixel rows
- projection – the datum
- geotransform – the coordinate system
- format (string) – a string representing the GDAL file format of the output raster. See http://gdal.org/formats_list.html for a list of available formats. This parameter expects the format code, such as ‘GTiff’ or ‘MEM’
- nodata – a value that will be set as the nodata value for the output raster. Should be the same type as ‘datatype’
- datatype – the pixel datatype of the output raster, for example gdal.GDT_Float32. See the following header file for supported pixel types: http://www.gdal.org/gdal_8h.html#22e22ce0a55036a96f652765793fb7a4
- bands (int) – the number of bands in the raster
- outputURI (string) – the file location for the outputed raster. If format is ‘MEM’ this can be an empty string
Returns: a new GDAL raster with the parameters as described above
Return type: dataset
-
natcap.invest.pygeoprocessing_0_3_3.
new_raster_from_base
(base, output_uri, gdal_format, nodata, datatype, fill_value=None, n_rows=None, n_cols=None, dataset_options=None)¶ Create a new, empty GDAL raster dataset with the spatial references, geotranforms of the base GDAL raster dataset.
Parameters: - base – a the GDAL raster dataset to base output size, and transforms on
- output_uri (string) – a string URI to the new output raster dataset.
- gdal_format (string) – a string representing the GDAL file format of the output raster. See http://gdal.org/formats_list.html for a list of available formats. This parameter expects the format code, such as ‘GTiff’ or ‘MEM’
- nodata – a value that will be set as the nodata value for the output raster. Should be the same type as ‘datatype’
- datatype – the pixel datatype of the output raster, for example gdal.GDT_Float32. See the following header file for supported pixel types: http://www.gdal.org/gdal_8h.html#22e22ce0a55036a96f652765793fb7a4
Keyword Arguments: - fill_value – the value to fill in the raster on creation
- n_rows – if set makes the resulting raster have n_rows in it if not, the number of rows of the outgoing dataset are equal to the base.
- n_cols – similar to n_rows, but for the columns.
- dataset_options – a list of dataset options that gets passed to the gdal creation driver, overrides defaults
Returns: a new GDAL raster dataset.
Return type: dataset
-
natcap.invest.pygeoprocessing_0_3_3.
new_raster_from_base_uri
(base_uri, output_uri, gdal_format, nodata, datatype, fill_value=None, n_rows=None, n_cols=None, dataset_options=None)¶ Create a new, empty GDAL raster dataset with the spatial references, geotranforms of the base GDAL raster dataset.
A wrapper for the function new_raster_from_base that opens up the base_uri before passing it to new_raster_from_base.
Parameters: - base_uri (string) – a URI to a GDAL dataset on disk.
- output_uri (string) – a string URI to the new output raster dataset.
- gdal_format (string) – a string representing the GDAL file format of the output raster. See http://gdal.org/formats_list.html for a list of available formats. This parameter expects the format code, such as ‘GTiff’ or ‘MEM’
- nodata – a value that will be set as the nodata value for the output raster. Should be the same type as ‘datatype’
- datatype – the pixel datatype of the output raster, for example gdal.GDT_Float32. See the following header file for supported pixel types: http://www.gdal.org/gdal_8h.html#22e22ce0a55036a96f652765793fb7a4
Keyword Arguments: - fill_value – the value to fill in the raster on creation
- n_rows – if set makes the resulting raster have n_rows in it if not, the number of rows of the outgoing dataset are equal to the base.
- n_cols – similar to n_rows, but for the columns.
- dataset_options – a list of dataset options that gets passed to the gdal creation driver, overrides defaults
Returns: nothing
-
natcap.invest.pygeoprocessing_0_3_3.
pixel_size_based_on_coordinate_transform
(dataset, coord_trans, point)¶ Get width and height of cell in meters.
Calculates the pixel width and height in meters given a coordinate transform and reference point on the dataset that’s close to the transform’s projected coordinate sytem. This is only necessary if dataset is not already in a meter coordinate system, for example dataset may be in lat/long (WGS84).
Parameters: - dataset (gdal.Dataset) – a projected GDAL dataset in the form of lat/long decimal degrees
- coord_trans (osr.CoordinateTransformation) – an OSR coordinate transformation from dataset coordinate system to meters
- point (tuple) – a reference point close to the coordinate transform coordinate system. must be in the same coordinate system as dataset.
Returns: - a 2-tuple containing (pixel width in meters, pixel
height in meters)
Return type: pixel_diff (tuple)
-
natcap.invest.pygeoprocessing_0_3_3.
pixel_size_based_on_coordinate_transform_uri
(dataset_uri, *args, **kwargs)¶ Get width and height of cell in meters.
A wrapper for pixel_size_based_on_coordinate_transform that takes a dataset uri as an input and opens it before sending it along.
Parameters: - dataset_uri (string) – a URI to a gdal dataset
- other parameters pass along (All) –
Returns: (pixel_width_meters, pixel_height_meters)
Return type: result (tuple)
-
natcap.invest.pygeoprocessing_0_3_3.
rasterize_layer_uri
(raster_uri, shapefile_uri, burn_values=[], option_list=[])¶ Rasterize datasource layer.
Burn the layer from ‘shapefile_uri’ onto the raster from ‘raster_uri’. Will burn ‘burn_value’ onto the raster unless ‘field’ is not None, in which case it will burn the value from shapefiles field.
Parameters: - raster_uri (string) – a URI to a gdal dataset
- shapefile_uri (string) – a URI to an ogr datasource
Keyword Arguments: - burn_values (list) – the equivalent value for burning into a polygon. If empty uses the Z values.
- option_list (list) – a Python list of options for the operation. Example: [“ATTRIBUTE=NPV”, “ALL_TOUCHED=TRUE”]
Returns: None
-
natcap.invest.pygeoprocessing_0_3_3.
reclassify_dataset_uri
(dataset_uri, value_map, raster_out_uri, out_datatype, out_nodata, exception_flag='values_required', assert_dataset_projected=True)¶ Reclassify values in a dataset.
A function to reclassify values in dataset to any output type. By default the values except for nodata must be in value_map.
Parameters: - dataset_uri (string) – a uri to a gdal dataset
- value_map (dictionary) – a dictionary of values of {source_value: dest_value, …} where source_value’s type is a postive integer type and dest_value is of type out_datatype.
- raster_out_uri (string) – the uri for the output raster
- out_datatype (gdal type) – the type for the output dataset
- out_nodata (numerical type) – the nodata value for the output raster. Must be the same type as out_datatype
Keyword Arguments: - exception_flag (string) – either ‘none’ or ‘values_required’. If ‘values_required’ raise an exception if there is a value in the raster that is not found in value_map
- assert_dataset_projected (boolean) – if True this operation will test if the input dataset is not projected and raise an exception if so.
Returns: nothing
Raises: Exception
– if exception_flag == ‘values_required’ and the value from ‘key_raster’ is not a key in ‘attr_dict’
-
natcap.invest.pygeoprocessing_0_3_3.
reproject_dataset_uri
(original_dataset_uri, pixel_spacing, output_wkt, resampling_method, output_uri)¶ Reproject and resample GDAL dataset.
A function to reproject and resample a GDAL dataset given an output pixel size and output reference. Will use the datatype and nodata value from the original dataset.
Parameters: - original_dataset_uri (string) – a URI to a gdal Dataset to written to disk
- pixel_spacing – output dataset pixel size in projected linear units
- output_wkt – output project in Well Known Text
- resampling_method (string) – a string representing the one of the following resampling methods: “nearest|bilinear|cubic|cubic_spline|lanczos”
- output_uri (string) – location on disk to dump the reprojected dataset
Returns: None
-
natcap.invest.pygeoprocessing_0_3_3.
reproject_datasource
(original_datasource, output_wkt, output_uri)¶ Reproject OGR DataSource object.
Changes the projection of an ogr datasource by creating a new shapefile based on the output_wkt passed in. The new shapefile then copies all the features and fields of the original_datasource as its own.
Parameters: - original_datasource – an ogr datasource
- output_wkt – the desired projection as Well Known Text (by layer.GetSpatialRef().ExportToWkt())
- output_uri (string) – the filepath to the output shapefile
Returns: the reprojected shapefile.
Return type: output_datasource
-
natcap.invest.pygeoprocessing_0_3_3.
reproject_datasource_uri
(original_dataset_uri, output_wkt, output_uri)¶ Reproject OGR DataSource file.
URI wrapper for reproject_datasource that takes in the uri for the datasource that is to be projected instead of the datasource itself. This function directly calls reproject_datasource.
Parameters: - original_dataset_uri (string) – a uri to an ogr datasource
- output_wkt – the desired projection as Well Known Text (by layer.GetSpatialRef().ExportToWkt())
- output_uri (string) – the path to where the new shapefile should be written to disk.
Returns: None
-
natcap.invest.pygeoprocessing_0_3_3.
resize_and_resample_dataset_uri
(original_dataset_uri, bounding_box, out_pixel_size, output_uri, resample_method)¶ Resize and resample the given dataset.
Parameters: - original_dataset_uri (string) – a GDAL dataset
- bounding_box (list) – [upper_left_x, upper_left_y, lower_right_x, lower_right_y]
- out_pixel_size – the pixel size in projected linear units
- output_uri (string) – the location of the new resampled GDAL dataset
- resample_method (string) – the resampling technique, one of “nearest|bilinear|cubic|cubic_spline|lanczos”
Returns: None
-
natcap.invest.pygeoprocessing_0_3_3.
temporary_filename
(suffix='')¶ Get path to new temporary file that will be deleted on program exit.
Returns a temporary filename using mkstemp. The file is deleted on exit using the atexit register.
Keyword Arguments: suffix (string) – the suffix to be appended to the temporary file Returns: a unique temporary filename Return type: fname
-
natcap.invest.pygeoprocessing_0_3_3.
temporary_folder
()¶ Get path to new temporary folder that will be deleted on program exit.
Returns a temporary folder using mkdtemp. The folder is deleted on exit using the atexit register.
Returns: an absolute, unique and temporary folder path. Return type: path (string)
-
natcap.invest.pygeoprocessing_0_3_3.
tile_dataset_uri
(in_uri, out_uri, blocksize)¶ - Resample gdal dataset into tiled raster with blocks of blocksize X
- blocksize.
Parameters: - in_uri (string) – dataset to base data from
- out_uri (string) – output dataset
- blocksize (int) – defines the side of the square for the raster, this seems to have a lower limit of 16, but is untested
Returns: None
-
natcap.invest.pygeoprocessing_0_3_3.
transform_bounding_box
(bounding_box, base_ref_wkt, new_ref_wkt, edge_samples=11)¶ Transform input bounding box to output projection.
This transform accounts for the fact that the reprojected square bounding box might be warped in the new coordinate system. To account for this, the function samples points along the original bounding box edges and attempts to make the largest bounding box around any transformed point on the edge whether corners or warped edges.
Parameters: - bounding_box (list) – a list of 4 coordinates in base_epsg coordinate system describing the bound in the order [xmin, ymin, xmax, ymax]
- base_ref_wkt (string) – the spatial reference of the input coordinate system in Well Known Text.
- new_ref_wkt (string) – the EPSG code of the desired output coordinate system in Well Known Text.
- edge_samples (int) – the number of interpolated points along each bounding box edge to sample along. A value of 2 will sample just the corners while a value of 3 will also sample the corners and the midpoint.
Returns: A list of the form [xmin, ymin, xmax, ymax] that describes the largest fitting bounding box around the original warped bounding box in new_epsg coordinate system.
-
natcap.invest.pygeoprocessing_0_3_3.
unique_raster_values
(dataset)¶ Get list of unique integer values within given dataset.
Parameters: dataset – a gdal dataset of some integer type Returns: a list of dataset’s unique non-nodata values Return type: unique_list (list)
-
natcap.invest.pygeoprocessing_0_3_3.
unique_raster_values_count
(dataset_uri, ignore_nodata=True)¶ Return a dict from unique int values in the dataset to their frequency.
Parameters: dataset_uri (string) – uri to a gdal dataset of some integer type Keyword Arguments: ignore_nodata (boolean) – if set to false, the nodata count is also included in the result Returns: values to count. Return type: itemfreq (dict)
-
natcap.invest.pygeoprocessing_0_3_3.
unique_raster_values_uri
(dataset_uri)¶ Get list of unique integer values within given dataset.
Parameters: dataset_uri (string) – a uri to a gdal dataset of some integer type Returns: a list of dataset’s unique non-nodata values Return type: value (list)
-
natcap.invest.pygeoprocessing_0_3_3.
vectorize_datasets
(dataset_uri_list, dataset_pixel_op, dataset_out_uri, datatype_out, nodata_out, pixel_size_out, bounding_box_mode, resample_method_list=None, dataset_to_align_index=None, dataset_to_bound_index=None, aoi_uri=None, assert_datasets_projected=True, process_pool=None, vectorize_op=True, datasets_are_pre_aligned=False, dataset_options=None, all_touched=False)¶ Apply local raster operation on stack of datasets.
This function applies a user defined function across a stack of datasets. It has functionality align the output dataset grid with one of the input datasets, output a dataset that is the union or intersection of the input dataset bounding boxes, and control over the interpolation techniques of the input datasets, if necessary. The datasets in dataset_uri_list must be in the same projection; the function will raise an exception if not.
Parameters: - dataset_uri_list (list) – a list of file uris that point to files that can be opened with gdal.Open.
- dataset_pixel_op (function) – arguments as there are elements in dataset_uri_list. The arguments can be treated as interpolated or actual pixel values from the input datasets and the function should calculate the output value for that pixel stack. The function is a parallel paradigmn and does not know the spatial position of the pixels in question at the time of the call. If the bounding_box_mode parameter is “union” then the values of input dataset pixels that may be outside their original range will be the nodata values of those datasets. Known bug: if dataset_pixel_op does not return a value in some cases the output dataset values are undefined even if the function does not crash or raise an exception.
- dataset_out_uri (string) – the uri of the output dataset. The projection will be the same as the datasets in dataset_uri_list.
- datatype_out – the GDAL output type of the output dataset
- nodata_out – the nodata value of the output dataset.
- pixel_size_out – the pixel size of the output dataset in projected coordinates.
- bounding_box_mode (string) – one of “union” or “intersection”, “dataset”. If union the output dataset bounding box will be the union of the input datasets. Will be the intersection otherwise. An exception is raised if the mode is “intersection” and the input datasets have an empty intersection. If dataset it will make a bounding box as large as the given dataset, if given dataset_to_bound_index must be defined.
Keyword Arguments: - resample_method_list (list) – a list of resampling methods for each output uri in dataset_out_uri list. Each element must be one of “nearest|bilinear|cubic|cubic_spline|lanczos”. If None, the default is “nearest” for all input datasets.
- dataset_to_align_index (int) – an int that corresponds to the position in one of the dataset_uri_lists that, if positive aligns the output rasters to fix on the upper left hand corner of the output datasets. If negative, the bounding box aligns the intersection/ union without adjustment.
- dataset_to_bound_index – if mode is “dataset” this indicates which dataset should be the output size.
- aoi_uri (string) – a URI to an OGR datasource to be used for the aoi. Irrespective of the mode input, the aoi will be used to intersect the final bounding box.
- assert_datasets_projected (boolean) – if True this operation will test if any datasets are not projected and raise an exception if so.
- process_pool – a process pool for multiprocessing
- vectorize_op (boolean) – if true the model will try to numpy.vectorize dataset_pixel_op. If dataset_pixel_op is designed to use maximize array broadcasting, set this parameter to False, else it may inefficiently invoke the function on individual elements.
- datasets_are_pre_aligned (boolean) – If this value is set to False this operation will first align and interpolate the input datasets based on the rules provided in bounding_box_mode, resample_method_list, dataset_to_align_index, and dataset_to_bound_index, if set to True the input dataset list must be aligned, probably by raster_utils.align_dataset_list
- dataset_options – this is an argument list that will be passed to the GTiff driver. Useful for blocksizes, compression, etc.
- all_touched (boolean) – if true the clip uses the option ALL_TOUCHED=TRUE when calling RasterizeLayer for AOI masking.
Returns: None
Raises: ValueError
– invalid input provided
-
natcap.invest.pygeoprocessing_0_3_3.
vectorize_points
(shapefile, datasource_field, dataset, randomize_points=False, mask_convex_hull=False, interpolation='nearest')¶ Interpolate values in shapefile onto given raster.
Takes a shapefile of points and a field defined in that shapefile and interpolate the values in the points onto the given raster
Parameters: - shapefile – ogr datasource of points
- datasource_field – a field in shapefile
- dataset – a gdal dataset must be in the same projection as shapefile
Keyword Arguments: - randomize_points (boolean) – (description)
- mask_convex_hull (boolean) – (description)
- interpolation (string) – the interpolation method to use for scipy.interpolate.griddata(). Default is ‘nearest’
Returns: None
-
natcap.invest.pygeoprocessing_0_3_3.
vectorize_points_uri
(shapefile_uri, field, output_uri, interpolation='nearest')¶ Interpolate values in shapefile onto given raster.
A wrapper function for pygeoprocessing.vectorize_points, that allows for uri passing.
Parameters: - shapefile_uri (string) – a uri path to an ogr shapefile
- field (string) – a string for the field name
- output_uri (string) – a uri path for the output raster
- interpolation (string) – interpolation method to use on points, default is ‘nearest’
Returns: None
Buffered file manager module.
-
class
natcap.invest.recreation.buffered_numpy_disk_map.
BufferedNumpyDiskMap
(manager_filename, max_bytes_to_buffer)¶ Bases:
object
Persistent object to append and read numpy arrays to unique keys.
This object is abstractly a key/value pair map where the operations are to append, read, and delete numpy arrays associated with those keys. The object attempts to keep data in RAM as much as possible and saves data to files on disk to manage memory and persist between instantiations.
-
append
(array_id, array_data)¶ Append data to the file.
Parameters: - array_id (int) – unique key to identify the array node
- array_data (numpy.ndarray) – data to append to node.
Returns: None
-
delete
(array_id)¶ Delete node array_id from disk and cache.
-
flush
()¶ Method to flush data in memory to disk.
-
read
(array_id)¶ Read the entirety of the file.
Internally this might mean that part of the file is read from disk and the end from the buffer or any combination of those.
Parameters: array_id (string) – unique node id to read Returns: contents of node as a numpy.ndarray.
-
InVEST Recreation Client.
-
natcap.invest.recreation.recmodel_client.
delay_op
(last_time, time_delay, func)¶ Execute func if last_time + time_delay >= current time.
Parameters: - last_time (float) – last time in seconds that func was triggered
- time_delay (float) – time to wait in seconds since last_time before triggering func
- func (function) – parameterless function to invoke if current_time >= last_time + time_delay
Returns: If func was triggered, return the time which it was triggered in seconds, otherwise return last_time.
-
natcap.invest.recreation.recmodel_client.
execute
(args)¶ Recreation.
Execute recreation client model on remote server.
Parameters: - args['workspace_dir'] (string) – path to workspace directory
- args['aoi_path'] (string) – path to AOI vector
- args['hostname'] (string) – FQDN to recreation server
- args['port'] (string or int) – port on hostname for recreation server
- args['start_year'] (string) – start year in form YYYY. This year is the inclusive lower bound to consider points in the PUD and regression.
- args['end_year'] (string) – end year in form YYYY. This year is the inclusive upper bound to consider points in the PUD and regression.
- args['grid_aoi'] (boolean) – if true the polygon vector in args[‘aoi_path’] should be gridded into a new vector and the recreation model should be executed on that
- args['grid_type'] (string) – optional, but must exist if args[‘grid_aoi’] is True. Is one of ‘hexagon’ or ‘square’ and indicates the style of gridding.
- args['cell_size'] (string/float) – optional, but must exist if args[‘grid_aoi’] is True. Indicates the cell size of square pixels and the width of the horizontal axis for the hexagonal cells.
- args['compute_regression'] (boolean) – if True, then process the predictor table and scenario table (if present).
- args['predictor_table_path'] (string) –
required if args[‘compute_regression’] is True. Path to a table that describes the regression predictors, their IDs and types. Must contain the fields ‘id’, ‘path’, and ‘type’ where:
- ’id’: is a <=10 character length ID that is used to uniquely describe the predictor. It will be added to the output result shapefile attribute table which is an ESRI Shapefile, thus limited to 10 characters.
- ’path’: an absolute or relative (to this table) path to the predictor dataset, either a vector or raster type.
- ’type’: one of the following,
- ’raster_mean’: mean of values in the raster under the response polygon
- ’raster_sum’: sum of values in the raster under the response polygon
- ’point_count’: count of the points contained in the response polygon
- ’point_nearest_distance’: distance to the nearest point from the response polygon
- ’line_intersect_length’: length of lines that intersect with the response polygon in projected units of AOI
- ’polygon_area’: area of the polygon contained within response polygon in projected units of AOI
- args['scenario_predictor_table_path'] (string) – (optional) if present runs the scenario mode of the recreation model with the datasets described in the table on this path. Field headers are identical to args[‘predictor_table_path’] and ids in the table are required to be identical to the predictor list.
- args['results_suffix'] (string) – optional, if exists is appended to any output file paths.
Returns: None
InVEST recreation workspace fetcher.
-
natcap.invest.recreation.recmodel_workspace_fetcher.
execute
(args)¶ Fetch workspace from remote server.
After the call a .zip file exists at args[‘workspace_dir’] named args[‘workspace_id’] + ‘.zip’ and contains the zipped workspace of that model run.
Parameters: - args['workspace_dir'] (string) – path to workspace directory
- args['hostname'] (string) – FQDN to recreation server
- args['port'] (string or int) – port on hostname for recreation server
- args['workspace_id'] (string) – workspace identifier
Returns: None
Utilities for creating simple HTML report files.
-
class
natcap.invest.reporting.html.
Element
(tag, content='', end_tag=True, **attrs)¶ Bases:
object
Represents a generic HTML element.
Any Element object can be passed to HTMLDocument.add()
Example
doc = html.HTMLDocument(…) details_elem = doc.add(html.Element(‘details’)) details_elem.add(
html.Element(‘img’, src=’images/my_pic.png’, end_tag=False))-
add
(elem)¶ Add a child element (which is returned for convenience).
-
html
()¶ Returns an HTML string for this element (and its children).
-
-
class
natcap.invest.reporting.html.
HTMLDocument
(uri, title, header)¶ Bases:
object
Utility class for creating simple HTML files.
- Example usage:
# Create the document object. doc = html.HTMLDocument(‘myfile.html’, ‘My Page’, ‘A Page About Me’)
# Add some text. doc.write_header(‘My Early Life’) doc.write_paragraph(‘I lived in a small barn.’)
# Add a table. table = doc.add(html.Table()) table.add_row([‘Age’, ‘Weight’], is_header=True) table.add_row([‘1 year’, ‘20 pounds’]) table.add_row([‘2 years’, ‘40 pounds’])
# Add an arbitrary HTML element. # Note that the HTML ‘img’ element doesn’t have an end tag. doc.add(html.Element(‘img’, src=’images/my_pic.png’, end_tag=False))
# Create the file. doc.flush()
-
add
(elem)¶ Add an arbitrary element to the body of the document.
elem - any object that has a method html() to output HTML markup
Return the added element for convenience.
-
flush
()¶ Create a file with the contents of this document.
-
insert_table_of_contents
(max_header_level=2)¶ Insert an auto-generated table of contents.
The table of contents is based on the headers in the document.
-
write_header
(text, level=2)¶ Convenience method to write a header.
-
write_paragraph
(text)¶ Convenience method to write a paragraph.
-
class
natcap.invest.reporting.html.
Table
(**attr)¶ Bases:
object
Represents and renders HTML tables.
-
add_row
(cells, is_header=False, cell_attr=None, do_formatting=True)¶ Writes a table row with the given cell data.
- cell_attr - attributes for each cell. If provided, it must be the
- same length as cells. Each entry should be a dictionary mapping attribute key to value.
-
add_two_level_header
(outer_headers, inner_headers, row_id_header)¶ Adds a two level header to the table.
In this header, each outer header appears on the top row, and each inner header appears once beneath each outer header.
For example, the following code:
- table.add_two_level_header(
- outer_headers=[‘Weight’, ‘Value’], inner_headers=[‘Mean, Standard deviation’], row_id_header=’Farm ID’)
produces the following header:
Weight ValueFarm ID Mean Standard Deviation Mean Standard deviation
-
html
()¶ Return the HTML string for the table.
-
-
natcap.invest.reporting.html.
cell_format
(data)¶ Formats the data to put in a table cell.
A helper module for generating html tables that are represented as Strings
-
natcap.invest.reporting.table_generator.
add_checkbox_column
(col_list, row_list, checkbox_pos=1)¶ Insert a new column into the list of column dictionaries so that it is the second column dictionary found in the list. Also add the checkbox column header to the list of row dictionaries and subsequent checkbox value
- ‘col_list’- a list of dictionaries that defines the column
structure for the table (required). The order of the columns from left to right is depicted by the index of the column dictionary in the list. Each dictionary in the list has the following keys and values:
‘name’ - a string for the column name (required) ‘total’ - a boolean for whether the column should be
totaled (required)- ‘row_list’ - a list of dictionaries that represent the rows. Each
dictionaries keys should match the column names found in ‘col_list’ (required) Example: [{col_name_1: value, col_name_2: value, …},
{col_name_1: value, col_name_2: value, …}, …]- checkbox_pos - an integer for the position of the checkbox
- column. Defaulted at 1 (optional)
- returns - a tuple of the updated column and rows list of dictionaries
- in that order
-
natcap.invest.reporting.table_generator.
add_totals_row
(col_headers, total_list, total_name, checkbox_total, tdata_tuples)¶ Construct a totals row as an html string. Creates one row element with data where the row gets a class name and the data get a class name if the corresponding column is a totalable column
col_headers - a list of the column headers in order (required)
- total_list - a list of booleans that corresponds to ‘col_headers’ and
- indicates whether a column should be totaled (required)
- total_name - a string for the name of the total row, ex: ‘Total’, ‘Sum’
- (required)
- checkbox_total - a boolean value that distinguishes whether a checkbox
- total row is being added or a regular total row. Checkbox total row is True. This will determine the row class name and row data class name (required)
- tdata_tuples - a list of tuples where the first index in the tuple is a
- boolean which indicates if a table data element has a attribute class. The second index is the String value of that class or None (required)
- return - a string representing the html contents of a row which should
- later be used in a ‘tfoot’ element
-
natcap.invest.reporting.table_generator.
generate_table
(table_dict, attributes=None)¶ Takes in a dictionary representation of a table and generates a String of the the table in the form of hmtl
- table_dict - a dictionary with the following arguments:
- ‘cols’- a list of dictionaries that defines the column
structure for the table (required). The order of the columns from left to right is depicted by the index of the column dictionary in the list. Each dictionary in the list has the following keys and values:
‘name’ - a string for the column name (required) ‘total’ - a boolean for whether the column should be
totaled (required)- ‘attr’ - a dictionary that has key value pairs for
- optional tag attributes (optional). Ex: ‘attr’: {‘class’: ‘offsets’}
- ‘td_class’ - a String to assign as a class name to
- the table data tags under the column. Each table data tag under the column will have a class attribute assigned to ‘td_class’ value (optional)
- ‘rows’ - a list of dictionaries that represent the rows. Each
dictionaries keys should match the column names found in ‘cols’ (possibly empty list) (required) Example: [{col_name_1: value, col_name_2: value, …},
{col_name_1: value, col_name_2: value, …}, …]- ‘checkbox’ - a boolean value for whether there should be a
- checkbox column. If True a ‘selected total’ row will be added to the bottom of the table that will show the total of the columns selected (optional)
- ‘checkbox_pos’ - an integer value for in which column
- position the the checkbox column should appear (optional)
- ‘total’- a boolean value for whether there should be a constant
- total row at the bottom of the table that sums the column values (optional)
- ‘attributes’ - a dictionary of html table attributes. The attribute
- name is the key which gets set to the value of the key. (optional) Example: {‘class’: ‘sorttable’, ‘id’: ‘parcel_table’}
returns - a string representing an html table
-
natcap.invest.reporting.table_generator.
get_dictionary_values_ordered
(dict_list, key_name)¶ Generate a list, with values from ‘key_name’ found in each dictionary in the list of dictionaries ‘dict_list’. The order of the values in the returned list match the order they are retrieved from ‘dict_list’
- dict_list - a list of dictionaries where each dictionary has the same
- keys. Each dictionary should have at least one key:value pair with the key being ‘key_name’ (required)
- key_name - a String or Int for the key name of interest in the
- dictionaries (required)
- return - a list of values from ‘key_name’ in ascending order based
- on ‘dict_list’ keys
-
natcap.invest.reporting.table_generator.
get_row_data
(row_list, col_headers)¶ Construct the rows in a 2D List from the list of dictionaries, using col_headers to properly order the row data.
- ‘row_list’ - a list of dictionaries that represent the rows. Each
dictionaries keys should match the column names found in ‘col_headers’. The rows will be ordered the same as they are found in the dictionary list (required) Example: [{‘col_name_1’:‘9/13’, ‘col_name_3’:’expensive’,
‘col_name_2’:’chips’},- {‘col_name_1’:‘3/13’, ‘col_name_2’:’cheap’,
- ‘col_name_3’:’peanuts’},
- {‘col_name_1’:‘5/12’, ‘col_name_2’:’moderate’,
- ‘col_name_3’:’mints’}]
- col_headers - a List of the names of the column headers in order
- example : [col_name_1, col_name_2, col_name_3…]
return - a 2D list with each inner list representing a row
natcap.invest.reporting package.
-
natcap.invest.reporting.
add_head_element
(param_args)¶ Generates a string that represents a valid element in the head section of an html file. Currently handles ‘style’ and ‘script’ elements, where both the script and style are locally embedded
param_args - a dictionary that holds the following arguments:
- param_args[‘format’] - a string representing the type of element to
- be added. Currently : ‘script’, ‘style’ (required)
- param_args[‘data_src’] - a string URI path for the external source
- of the element OR a String representing the html (DO NOT include html tags, tags are automatically generated). If a URI the file is read in as a String. (required)
- param_args[‘input_type’] - ‘Text’ or ‘File’. Determines how the
- input from ‘data_src’ is handled (required)
- ‘attributes’ - a dictionary that has key value pairs for
- optional tag attributes (optional). Ex: ‘attributes’: {‘class’: ‘offsets’}
returns - a string representation of the html head element
-
natcap.invest.reporting.
add_text_element
(param_args)¶ Generates a string that represents a html text block. The input string should be wrapped in proper html tags
param_args - a dictionary with the following arguments:
param_args[‘text’] - a stringreturns - a string
-
natcap.invest.reporting.
build_table
(param_args)¶ Generates a string representing a table in html format.
- param_args - a dictionary that has the parameters for building up the
html table. The dictionary includes the following:
- ‘attributes’ - a dictionary of html table attributes. The attribute
- name is the key which gets set to the value of the key. (optional) Example: {‘class’: ‘sorttable’, ‘id’: ‘parcel_table’}
- param_args[‘sortable’] - a boolean value that determines whether the
- table should be sortable (required)
- param_args[‘data_type’] - a string depicting the type of input to
- build the table from. Either ‘shapefile’, ‘csv’, or ‘dictionary’ (required)
- param_args[‘data’] - a URI to a csv or shapefile OR a list of
dictionaries. If a list of dictionaries the data should be represented in the following format: (required)
- [{col_name_1: value, col_name_2: value, …},
- {col_name_1: value, col_name_2: value, …}, …]
- param_args[‘key’] - a string that depicts which column (csv) or
- field (shapefile) will be the unique key to use in extracting the data into a dictionary. (required for ‘data_type’ ‘shapefile’ and ‘csv’)
- param_args[‘columns’] - a list of dictionaries that defines the
column structure for the table (required). The order of the columns from left to right is depicted by the index of the column dictionary in the list. Each dictionary in the list has the following keys and values:
‘name’ - a string for the column name (required) ‘total’ - a boolean for whether the column should be
totaled (required)- ‘attr’ - a dictionary that has key value pairs for
- optional tag attributes (optional). Ex: ‘attr’: {‘class’: ‘offsets’}
- ‘td_class’ - a String to assign as a class name to
- the table data tags under the column. Each table data tag under the column will have a class attribute assigned to ‘td_class’ value (optional)
- param_args[‘total’] - a boolean value where if True a constant
- total row will be placed at the bottom of the table that sums the columns (required)
returns - a string that represents an html table
-
natcap.invest.reporting.
data_dict_to_list
(data_dict)¶ Abstract out inner dictionaries from data_dict into a list, where the inner dictionaries are added to the list in the order of their sorted keys
- data_dict - a dictionary with unique keys pointing to dictionaries.
- Could be empty (required)
returns - a list of dictionaries, or empty list if data_dict is empty
-
natcap.invest.reporting.
generate_report
(args)¶ Generate an html page from the arguments given in ‘reporting_args’
- reporting_args[title] - a string for the title of the html page
- (required)
- reporting_args[sortable] - a boolean value indicating whether
- the sorttable.js library should be added for table sorting functionality (optional)
- reporting_args[totals] - a boolean value indicating whether
- the totals_function.js script should be added for table totals functionality (optional)
- reporting_args[out_uri] - a URI to the output destination for the html
- page (required)
- reporting_args[elements] - a list of dictionaries that represent html
elements to be added to the html page. (required) If no elements are provided (list is empty) a blank html page will be generated. The 3 main element types are ‘table’, ‘head’, and ‘text’. All elements share the following arguments:
- ‘type’ - a string that depicts the type of element being add.
- Currently ‘table’, ‘head’, and ‘text’ are defined (required)
- ‘section’ - a string that depicts whether the element belongs
- in the body or head of the html page. Values: ‘body’ | ‘head’ (required)
Table element dictionary has at least the following additional arguments:
- ‘attributes’ - a dictionary of html table attributes. The
- attribute name is the key which gets set to the value of the key. (optional) Example: {‘class’: ‘sorttable’, ‘id’: ‘parcel_table’}
- ‘sortable’ - a boolean value for whether the tables columns
- should be sortable (required)
- ‘checkbox’ - a boolean value for whether there should be a
- checkbox column. If True a ‘selected total’ row will be added to the bottom of the table that will show the total of the columns selected (optional)
- ‘checkbox_pos’ - an integer value for in which column
- position the the checkbox column should appear (optional)
- ‘data_type’ - one of the following string values:
- ‘shapefile’|’hg csv’|’dictionary’. Depicts the type of data structure to build the table from (required)
- ‘data’ - either a list of dictionaries if ‘data_type’ is
‘dictionary’ or a URI to a CSV table or shapefile if ‘data_type’ is ‘shapefile’ or ‘csv’ (required). If a list of dictionaries, each dictionary should have keys that represent the columns, where each dictionary is a row (list could be empty) How the rows are ordered are defined by their index in the list. Formatted example: [{col_name_1: value, col_name_2: value, …},
{col_name_1: value, col_name_2: value, …}, …]- ‘key’ - a string that defines which column or field should be
- used as the keys for extracting data from a shapefile or csv table ‘key_field’. (required for ‘data_type’ = ‘shapefile’ | ‘csv’)
- ‘columns’- a list of dictionaries that defines the column
structure for the table (required). The order of the columns from left to right is depicted by the index of the column dictionary in the list. Each dictionary in the list has the following keys and values:
‘name’ - a string for the column name (required) ‘total’ - a boolean for whether the column should be
totaled (required)- ‘attr’ - a dictionary that has key value pairs for
- optional tag attributes (optional). Ex: ‘attr’: {‘class’: ‘offsets’}
- ‘td_class’ - a String to assign as a class name to
- the table data tags under the column. Each table data tag under the column will have a class attribute assigned to ‘td_class’ value (optional)
- ‘total’- a boolean value for whether there should be a constant
- total row at the bottom of the table that sums the column values (optional)
Head element dictionary has at least the following additional arguments:
- ‘format’ - a string representing the type of head element being
- added. Currently ‘script’ (javascript) and ‘style’ (css style) accepted (required)
- ‘data_src’- a URI to the location of the external file for
- either the ‘script’ or the ‘style’ OR a String representing the html script or style (DO NOT include the tags) (required)
- ‘input_type’ - a String, ‘File’ or ‘Text’ that refers to how
- ‘data_src’ is being passed in (URI vs String) (required).
- ‘attributes’ - a dictionary that has key value pairs for
- optional tag attributes (optional). Ex: ‘attributes’: {‘id’: ‘muni_data’}
Text element dictionary has at least the following additional arguments:
- ‘text’- a string to add as a paragraph element in the html page
- (required)
returns - nothing
-
natcap.invest.reporting.
u
(string)¶
-
natcap.invest.reporting.
write_html
(html_obj, out_uri)¶ Write an html file to ‘out_uri’ from html element represented as strings in ‘html_obj’
- html_obj - a dictionary with two keys, ‘head’ and ‘body’, that point to
lists. The list for each key is a list of the htmls elements as strings (required) example: {‘head’:[‘elem_1’, ‘elem_2’,…],
‘body’:[‘elem_1’, ‘elem_2’,…]}
out_uri - a URI for the output html file
returns - nothing
A class for a Qt-enabled python Thread.
Module to that provides functions for usage logging.
-
natcap.invest.ui.usage.
log_run
(*args, **kwds)¶ Context manager to log an InVEST model run and exit status.
Parameters: - module (string) – The string module name that identifies the model.
- args (dict) – The full args dictionary.
Returns: None
Functions to assist with remote logging of InVEST usage.
-
class
natcap.invest.ui.usage_logger.
LoggingServer
¶ Bases:
object
RPC server for logging invest runs and getting database summaries.
-
log_invest_run
¶
-
-
natcap.invest.ui.usage_logger.
execute
(args)¶ Function to start a remote procedure call server.
Parameters: - args['hostname'] (string) – network interface to bind to
- args['port'] (int) – TCP port to bind to
Returns: never
Carbon Storage and Sequestration.
-
natcap.invest.carbon.
execute
(args)¶ InVEST Carbon Model.
Calculate the amount of carbon stocks given a landscape, or the difference due to a future change, and/or the tradeoffs between that and a REDD scenario, and calculate economic valuation on those scenarios.
The model can operate on a single scenario, a combined present and future scenario, as well as an additional REDD scenario.
Parameters: - args['workspace_dir'] (string) – a path to the directory that will write output and other temporary files during calculation.
- args['results_suffix'] (string) – appended to any output file name.
- args['lulc_cur_path'] (string) – a path to a raster representing the current carbon stocks.
- args['calc_sequestration'] (bool) – if true, sequestration should be calculated and ‘lulc_fut_path’ and ‘do_redd’ should be defined.
- args['lulc_fut_path'] (string) – a path to a raster representing future landcover scenario. Optional, but if present and well defined will trigger a sequestration calculation.
- args['do_redd'] (bool) – if true, REDD analysis should be calculated and ‘lulc_redd_path’ should be defined
- args['lulc_redd_path'] (string) – a path to a raster representing the alternative REDD scenario which is only possible if the args[‘lulc_fut_path’] is present and well defined.
- args['carbon_pools_path'] (string) – path to CSV or that indexes carbon storage density to lulc codes. (required if ‘do_uncertainty’ is false)
- args['lulc_cur_year'] (int/string) – an integer representing the year of args[‘lulc_cur_path’] used if args[‘calc_sequestration’] is True.
- args['lulc_fut_year'] (int/string) – an integer representing the year of args[‘lulc_fut_path’] used in valuation if it exists. Required if args[‘do_valuation’] is True and args[‘lulc_fut_path’] is present and well defined.
- args['do_valuation'] (bool) – if true then run the valuation model on available outputs. At a minimum will run on carbon stocks, if sequestration with a future scenario is done and/or a REDD scenario calculate NPV for either and report in final HTML document.
- args['price_per_metric_ton_of_c'] (float) – Is the present value of carbon per metric ton. Used if args[‘do_valuation’] is present and True.
- args['discount_rate'] (float) – Discount rate used if NPV calculations are required. Used if args[‘do_valuation’] is present and True.
- args['rate_change'] (float) – Annual rate of change in price of carbon as a percentage. Used if args[‘do_valuation’] is present and True.
- args['n_workers'] (int) – (optional) The number of worker processes to use for processing this model. If omitted, computation will take place in the current process.
Returns: None.
Single entry point for all InVEST applications.
-
class
natcap.invest.cli.
ListModelsAction
(option_strings, dest, nargs=None, const=None, default=None, type=None, choices=None, required=False, help=None, metavar=None)¶ Bases:
argparse.Action
An argparse action to list the available models.
-
class
natcap.invest.cli.
SelectModelAction
(option_strings, dest, nargs=None, const=None, default=None, type=None, choices=None, required=False, help=None, metavar=None)¶ Bases:
argparse.Action
Given a possily-ambiguous model string, identify the model to run.
This is a subclass of
argparse.Action
and is executed when the argparse interface detects that the user has attempted to select a model by name.
-
natcap.invest.cli.
build_model_list_table
()¶ Build a table of model names, aliases and other details.
This table is a table only in the sense that its contents are aligned into columns, but are not separated by a delimited. This table is intended to be printed to stdout.
Returns: A string representation of the formatted table.
-
natcap.invest.cli.
main
()¶ CLI entry point for launching InVEST runs.
This command-line interface supports two methods of launching InVEST models from the command-line:
- through its GUI
- in headless mode, without its GUI.
Running in headless mode allows us to bypass all GUI functionality, so models may be run in this way wthout having GUI packages installed.
InVEST Coastal Vulnerability
-
natcap.invest.coastal_vulnerability.
aggregate_population_density
(base_shore_point_vector_path, base_population_raster_path, search_radius, model_resolution, workspace_dir, file_suffix, target_pickle_path)¶ Get population density within a search radius of points.
Parameters: - base_shore_point_vector_path (string) – path to point vector
- base_population_raster_path (string) – path to raster with population values
- search_radius (float) – radius in meters around each point to search for valid population pixels.
- model_resolution (float) – distance in meters of the shore_point spacing, used here as a target pixel size in warp raster.
- workspace_dir (string) – path to directory for intermediate files
- file_suffix (string) – appended to any output filename.
- target_pickle_path (string) – path to pickle file storing dict keyed by shore point fid.
Returns: None
-
natcap.invest.coastal_vulnerability.
assemble_results_and_calculate_exposure
(risk_id_path_list, habitat_protection_path, base_point_vector_path, target_intermediate_vector_path, target_output_vector_path, target_output_csv_path)¶ Calculate the final exposure score, with and without habitat protection.
Parameters: - risk_id_path_list (list) –
list of 3-tuples like: (‘relief.pickle’, True, ‘R_relief’)
- string: path to pickle with intermediate
- exposure values for a single variable
- bool:
- if True: variable contains values that need binning
- by percentile to convert to 1-5 ranks.
if False: variable is already on the 1-5 rank scale.
- string: if not None, this variable is included in the
- final exposure score, and ‘R_name’ becomes a fieldname in target_point_vector_path.
- habitat_protection_path (string) – path to csv file with the intermediate habitat ranks.
- base_point_vector_path (string) – path to shore point vector.
- target_intermediate_vector_path (string) – path to point vector populated with fields and raw values for all risk variables that require binning to ranks.
- target_output_vector_path (string) – path to point vector populated with fields and rank values for all risk variables, final exposure, exposure without habitats, and population density.
- target_output_csv_path (string) – path to a csv copy of target_output_vector_path.
Returns: None.
- risk_id_path_list (list) –
-
natcap.invest.coastal_vulnerability.
calculate_final_risk
(output_vector_path, output_csv_path)¶ Apply geometric mean calculation to variables at each shore point.
This function modifies the ‘output_vector_path’ file by adding new fields.
Parameters: - output_vector_path (string) – path to shore point vector populated with numeric fields named with the ‘R_’ prefix. These fields are included in the geometric mean.
- output_csv_path (string) – path to csv copy of the final ‘output_vector_path’.
Returns: None
-
natcap.invest.coastal_vulnerability.
calculate_geomorphology_exposure
(geomorphology_vector_path, geomorphology_fill_value, base_shore_point_vector_path, model_resolution, target_pickle_path)¶ Join geomorphology ranks to shore points by proximity.
Buffer each shore point by half the model_resolution and find all geomorphology types present around each point. Return the average of the geomorphology ranks, unless none are present, then return the
Parameters: - geomorphology_vector_path (string) – path to polyline vector with an integer attribute named RANK that contains values (1, 2, 3, 4, 5).
- geomorphology_fill_value (int) – integer in (1, 2, 3, 4, 5).
- base_shore_point_vector_path (string) – path to point vector
- model_resolution (float) – distance in meters of the shore_point spacing.
- target_pickle_path (string) – path to pickle file storing dict keyed by point fid.
Returns: None
-
natcap.invest.coastal_vulnerability.
calculate_habitat_rank
(habitat_pickle_list, target_habitat_protection_path)¶ Combine dicts of habitat ranks into a dataframe and calcuate Rhab.
Parameters: - habitat_pickle_list (list) – list of file paths to pickled dictionaries in the form of: {‘habitat_id’: {<fid0>: 5, <fid1>.: 5, <fid2>: 5}}
- target_habitat_protection_path (string) –
path to a csv file with a row for each shore point, and a header like:
’fid’,’kelp’,’eelgrass’,’coral’,’R_hab’
Returns: None
-
natcap.invest.coastal_vulnerability.
calculate_relief_exposure
(base_shore_point_vector_path, base_dem_path, dem_averaging_radius, model_resolution, workspace_dir, file_suffix, target_relief_pickle_path)¶ Calculate average of DEM pixels within a radius of shore points.
Parameters: - base_shore_point_vector_path (string) – path to a shore point vector.
- base_dem_path (string) – path to a DEM raster.
- dem_averaging_radius (float) – distance in meters
- model_resolution (float) – distance in meters of the shore_point spacing, used here as a target pixel size in warp raster.
- workspace_dir (string) – path to a directory for intermediate files
- file_suffix (string) – to be appended to output filenames
- target_relief_pickle_path (string) – path to pickle file storing dict keyed by shore point fid.
Returns: None.
-
natcap.invest.coastal_vulnerability.
calculate_surge_exposure
(base_shore_point_vector_path, shelf_contour_path, aoi_vector_path, target_surge_pickle_path)¶ Calculate surge potential as distance to nearest point on a contour.
Parameters: - base_shore_point_vector_path (string) – path to a point shapefile to for relief point analysis.
- shelf_contour_path (string) – path to a polyline vector.
- aoi_vector_path (string) – path to a polygon vector.
- target_surge_pickle_path (string) – path to pickle file storing dict keyed by shore point fid.
Returns: None.
-
natcap.invest.coastal_vulnerability.
calculate_wave_exposure
(base_fetch_point_vector_path, max_fetch_distance, target_wave_exposure_pickle_path)¶ Calculate wave exposure values at each shore point.
Parameters: - base_fetch_point_vector_path (string) – path to a point shapefile that contains 16 ‘WavP_[direction]’ fields, ‘WavPPCT[direction]’ fields, ‘fdist_[direction]’ fields, a single H, and a single T field.
- max_fetch_distance (float) – max fetch distance before a wind fetch ray is terminated.
- target_wave_exposure_pickle_path (string) – path to pickle file storing dict keyed by shore point fid.
Returns: None
-
natcap.invest.coastal_vulnerability.
calculate_wind_exposure
(base_shore_point_vector_path, landmass_polygon_pickle_path, landmass_line_rtree_path, landmass_lines_pickle_path, workspace_dir, file_suffix, max_fetch_distance, target_shore_point_vector_path, target_wind_exposure_pickle_path)¶ Calculate wind exposure for each shore point.
Parameters: - base_shore_point_vector_path (string) – path to a point vector with WWIII variables in the table.
- landmass_polygon_pickle_path (string) – path to pickle storing shapely polygon geometry of the landmass.
- landmass_line_rtree_path (string) – path to rtree file indexing bounds of line geometries.
- landmass_lines_pickle_path (string) – path to pickle storing list of shapely line geometries. List index must match index of landmass_line_rtree_path.
- workspace_dir (string) – path to a directory for intermediate files
- file_suffix (string) – to be appended to output filenames
- max_fetch_distance (float) – maximum fetch distance for a ray in meters.
- target_shore_point_vector_path (string) – path to target point file, will be a copy of base_shore_point_vector_path’s geometry with an ‘REI’ (relative exposure index) field added.
- target_wind_exposure_pickle_path (string) – path to pickle file storing dict keyed by shore point fid, with wind exposure values.
Returns: None
-
natcap.invest.coastal_vulnerability.
clip_and_project_raster
(base_raster_path, clipping_box, target_srs_wkt, model_resolution, working_dir, file_suffix, target_raster_path)¶ Clip a raster to a box in the raster’s native SRS, then reproject.
Parameters: - base_raster_path (string) – path to a gdal raster
- clipping_box (list) – sequence of floats that are coordinates in the target_srs [minx, miny, maxx, maxy]
- target_srs_wkt (string) – well-known-text spatial reference system
- model_resolution (float) – value for target pixel size
- working_dir (string) – path to directory for intermediate files
- file_suffix (string) – appended to any output filename.
- target_raster_path (string) – path to clipped and warped raster.
Returns: None
-
natcap.invest.coastal_vulnerability.
clip_and_project_vector
(base_vector_path, clipping_box, target_srs_wkt, tmp_vector_path, target_vector_path)¶ Clip a vector to a box in the vector’s native SRS, then reproject.
Parameters: - base_vector_path (string) – path to polygon or multipolygon type vector
- clipping_box (list) – sequence of floats that are coordinates in the target_srs [minx, miny, maxx, maxy]
- target_srs_wkt (string) – well-known-text spatial reference system
- tmp_vector_path (string) – path to clipped but unprojected .gpkg vector
- target_vector_path (string) – path to clipped and projected .gpkg vector
Returns: None
-
natcap.invest.coastal_vulnerability.
execute
(args)¶ InVEST Coastal Vulnerability Model.
For points along a coastline, evaluate the relative exposure of points to coastal hazards based on up to eight biophysical hazard indices. Also quantify the role of habitats in reducing the hazard. Optionally summarize the population density in proximity to each shore point.
Parameters: - args['workspace_dir'] (string) – (required) a path to the directory that will write output and other temporary files during calculation.
- args['results_suffix'] (string) – (optional) appended to any output filename.
- args['aoi_vector_path'] (string) – (required) path to a polygon vector that is projected in a coordinate system with units of meters. The polygon should intersect the landmass and the shelf contour line.
- args['model_resolution'] (string) – (required) distance in meters. Points are spaced along the coastline at intervals of this distance.
- args['landmass_vector_path'] (string) – (required) path to a polygon vector representing landmasses in the region of interest.
- args['wwiii_vector_path'] (string) – (required) path to a point vector containing wind and wave information across the region of interest.
- args['max_fetch_distance'] (string) – (required) maximum distance in meters to extend rays from shore points. Points with rays equal to this distance will accumulate ocean-driven wave exposure along those rays and local-wind-driven wave exposure along the shorter rays.
- args['shelf_contour_vector_path'] (string) – (required) path to a polyline vector delineating edges of the continental shelf or other bathymetry contour.
- args['dem_path'] (string) – (required) path to a raster representing the elevation on land in the region of interest.
- args['dem_averaging_radius'] (int or float) – (required) a value >= 0. The radius in meters around each shore point in which to compute the average elevation.
- args['habitat_table_path'] (string) –
(rqeuired) path to a CSV file with the following four fields:
’id’: unique string to represent each habitat ‘path’: absolute or relative path to a polygon vector ‘rank’: integer from 1 to 5 representing the relativeprotection offered by this habitat- ’protection distance (m)’: integer or float used as a
- search radius around each shore point.
- args['geomorphology_vector_path'] (string) – (optional) path to a polyline vector that has a field called “RANK” with values from 1 to 5 in the attribute table.
- args['geomorphology_fill_value'] (int) – (optional) a value from 1 to 5 that will be used as a geomorphology rank for any points not proximate to the geomorphology_vector_path.
- args['population_rater_path'] (string) – (optional) path a raster with values of total population per pixel.
- args['population_radius'] (int or float) – (optional) a value >= 0. The radius in meters around each shore point in which to compute the population density.
- args['slr_vector_path'] (string) – (optional) path to point vector containing the field args[‘slr_field’].
- args['slr_field'] (string) – name of a field in args[‘slr_vector_path’] containing numeric values.
- args['n_workers'] (int) – (optional) The number of worker processes to use for processing this model. If omitted, computation will take place in the current process.
Returns: None
-
natcap.invest.coastal_vulnerability.
geometry_to_lines
(geometry)¶ Convert a geometry object to a list of lines.
-
natcap.invest.coastal_vulnerability.
interpolate_sealevelrise_points
(base_shore_point_vector_path, slr_points_vector_path, slr_fieldname, target_pickle_path)¶ Spatial join of sea-level rise data to shore points.
Finds nearest sea-level points to each shore point and calculates a weighted average of values with inverse-distance weights.
Parameters: - base_shore_point_vector_path (string) – path to point vector
- slr_points_vector_path (string) – path to point vector containing the field slr_fieldname.
- slr_fieldname (string) – name of a field containing numeric values
- target_pickle_path (string) – path to pickle file storing dict keyed by point fid.
Returns: None
-
natcap.invest.coastal_vulnerability.
interpolate_shore_points
(aoi_vector_path, landmass_lines_pickle_path, model_resolution, target_vector_path)¶ Create points along a polyline geometry at a specified interval.
Parameters: - aoi_vector_path (string) – path to polygon vector used to define boundaries for adding points.
- landmass_lines_pickle_path (string) – path to pickle file containing list of shapely linestrings
- model_resolution (float) – distance in meters for the point’s spacing
- target_vector_path (string) – path to .gpkg point vector
Returns: None
-
natcap.invest.coastal_vulnerability.
interpolate_wwiii_to_shore
(base_shore_point_vector_path, wwiii_vector_path, target_shore_point_vector_path)¶ Spatial join of Wave Watch 3 data to shore points.
Finds nearest WW3 points to each shore point and calculates a weighted average of values with distance weights.
Parameters: - base_shore_point_vector_path (string) – path to point vector
- wwiii_vector_path (string) – path to point shapefile representing the Wave Watch III data.
- target_shore_point_vector_path (string) – path to point vector file with interpolated wwiii data.
Returns: None
-
natcap.invest.coastal_vulnerability.
polygon_to_lines
(geometry)¶ Return a list of shapely lines given higher order shapely geometry.
-
natcap.invest.coastal_vulnerability.
prepare_landmass_line_index
(landmass_vector_path, target_polygon_pickle_path, target_lines_pickle_path, target_rtree_path)¶ Prepare landmass polygon geometry for line operations.
Converts landmass polygons to lines used in shore point interpolation and saves spatial index of lines for fast intersections. Also saves unioned geometry polygons for fast point-in-polygon checks.
Parameters: - landmass_vector_path (string) – path to polygon vector
- target_polygon_pickle_path (string) – path to pickle storing shapely polygon geometry.
- target_lines_pickle_path (string) – path to pickle storing list of shapely line geometries
- target_rtree_path (string) – path to rtree file indexing bounds of line geometries.
Returns: None
-
natcap.invest.coastal_vulnerability.
search_for_habitat
(base_shore_point_vector_path, search_radius, habitat_rank, habitat_id, habitat_vector_path, target_habitat_pickle_path)¶ Search for habitat polygon within a radius of each shore point.
If habitat is present within the search radius, assign the habitat_rank to the shore point FID. If habitat is not present, assign rank of 5.
Parameters: - base_shore_point_vector_path (string) – path to a shore point vector.
- search_radius (integer) – distance around each point to search for habitat. units match units from base_shore_point_vector SRS.
- habitat_rank (integer) – from 1 to 5 representing the relative protection offered by this habitat (5 = no protection).
- habitat_id (string) – unique string to represent each habitat.
- habitat_vector_path (string) – path to a polygon vector.
- target_habitat_pickle_path (string) –
path to pickle file storing a dict keyed by shore point FID, nested in a dict keyed by habitat_id:
{‘habitat_id’: {<fid0>: 5, <fid1>.: 5, <fid2>: 5}}
Returns: None
InVEST Crop Production Percentile Model.
-
natcap.invest.crop_production_percentile.
aggregate_to_polygons
(base_aggregate_vector_path, target_aggregate_vector_path, landcover_raster_projection, crop_to_landcover_table, nutrient_table, yield_percentile_headers, output_dir, file_suffix, target_aggregate_table_path)¶ Write table with aggregate results of yield and nutrient values.
Use zonal statistics to summarize total observed and interpolated production and nutrient information for each polygon in base_aggregate_vector_path.
Parameters: - base_aggregate_vector_path (string) – path to polygon vector
- target_aggregate_vector_path (string) – path to re-projected copy of polygon vector
- landcover_raster_projection (string) – a WKT projection string
- crop_to_landcover_table (dict) – landcover codes keyed by crop names
- nutrient_table (dict) – a lookup of nutrient values by crop in the form of nutrient_table[<crop>][<nutrient>].
- yield_percentile_headers (list) – list of strings indicating percentiles at which yield was calculated.
- output_dir (string) – the file path to the output workspace.
- file_suffix (string) – string to appened to any output filenames.
- target_aggregate_table_path (string) – path to ‘aggregate_results.csv’ in the output workspace
Returns: None
-
natcap.invest.crop_production_percentile.
execute
(args)¶ Crop Production Percentile Model.
This model will take a landcover (crop cover?) map and produce yields, production, and observed crop yields, a nutrient table, and a clipped observed map.
Parameters: - args['workspace_dir'] (string) – output directory for intermediate, temporary, and final files
- args['results_suffix'] (string) – (optional) string to append to any output file names
- args['landcover_raster_path'] (string) – path to landcover raster
- args['landcover_to_crop_table_path'] (string) –
path to a table that converts landcover types to crop names that has two headers: * lucode: integer value corresponding to a landcover code in
args[‘landcover_raster_path’].- crop_name: a string that must match one of the crops in args[‘model_data_path’]/climate_bin_maps/[cropname]_* A ValueError is raised if strings don’t match.
- args['aggregate_polygon_path'] (string) – path to polygon shapefile that will be used to aggregate crop yields and total nutrient value. (optional, if value is None, then skipped)
- args['model_data_path'] (string) –
path to the InVEST Crop Production global data directory. This model expects that the following directories are subdirectories of this path * climate_bin_maps (contains [cropname]_climate_bin.tif files) * climate_percentile_yield (contains
[cropname]_percentile_yield_table.csv files)Please see the InVEST user’s guide chapter on crop production for details about how to download these data.
- args['n_workers'] (int) – (optional) The number of worker processes to use for processing this model. If omitted, computation will take place in the current process.
Returns: None.
-
natcap.invest.crop_production_percentile.
tabulate_results
(nutrient_table, yield_percentile_headers, crop_to_landcover_table, pixel_area_ha, landcover_raster_path, landcover_nodata, output_dir, file_suffix, target_table_path)¶ Write table with total yield and nutrient results by crop.
This function includes all the operations that write to results_table.csv.
Parameters: - nutrient_table (dict) – a lookup of nutrient values by crop in the form of nutrient_table[<crop>][<nutrient>].
- yield_percentile_headers (list) – list of strings indicating percentiles at which yield was calculated.
- crop_to_landcover_table (dict) – landcover codes keyed by crop names
- pixel_area_ha (float) – area of lulc raster cells (hectares)
- landcover_raster_path (string) – path to landcover raster
- landcover_nodata (float) – landcover raster nodata value
- output_dir (string) – the file path to the output workspace.
- file_suffix (string) – string to appened to any output filenames.
- target_table_path (string) – path to ‘result_table.csv’ in the output workspace
Returns: None
InVEST Crop Production Percentile Model.
-
natcap.invest.crop_production_regression.
aggregate_regression_results_to_polygons
(base_aggregate_vector_path, target_aggregate_vector_path, landcover_raster_projection, crop_to_landcover_table, nutrient_table, output_dir, file_suffix, target_aggregate_table_path)¶ Write table with aggregate results of yield and nutrient values.
Use zonal statistics to summarize total observed and interpolated production and nutrient information for each polygon in base_aggregate_vector_path.
Parameters: - base_aggregate_vector_path (string) – path to polygon vector
- target_aggregate_vector_path (string) – path to re-projected copy of polygon vector
- landcover_raster_projection (string) – a WKT projection string
- crop_to_landcover_table (dict) – landcover codes keyed by crop names
- nutrient_table (dict) – a lookup of nutrient values by crop in the form of nutrient_table[<crop>][<nutrient>].
- output_dir (string) – the file path to the output workspace.
- file_suffix (string) – string to appened to any output filenames.
- target_aggregate_table_path (string) – path to ‘aggregate_results.csv’ in the output workspace
Returns: None
-
natcap.invest.crop_production_regression.
execute
(args)¶ Crop Production Regression Model.
This model will take a landcover (crop cover?), N, P, and K map and produce modeled yields, and a nutrient table.
Parameters: - args['workspace_dir'] (string) – output directory for intermediate, temporary, and final files
- args['results_suffix'] (string) – (optional) string to append to any output file names
- args['landcover_raster_path'] (string) – path to landcover raster
- args['landcover_to_crop_table_path'] (string) –
path to a table that converts landcover types to crop names that has two headers: * lucode: integer value corresponding to a landcover code in
args[‘landcover_raster_path’].- crop_name: a string that must match one of the crops in args[‘model_data_path’]/climate_regression_yield_tables/[cropname]_* A ValueError is raised if strings don’t match.
- args['fertilization_rate_table_path'] (string) – path to CSV table that contains fertilization rates for the crops in the simulation, though it can contain additional crops not used in the simulation. The headers must be ‘crop_name’, ‘nitrogen_rate’, ‘phosphorous_rate’, and ‘potassium_rate’, where ‘crop_name’ is the name string used to identify crops in the ‘landcover_to_crop_table_path’, and rates are in units kg/Ha.
- args['aggregate_polygon_path'] (string) – path to polygon shapefile that will be used to aggregate crop yields and total nutrient value. (optional, if value is None, then skipped)
- args['model_data_path'] (string) –
path to the InVEST Crop Production global data directory. This model expects that the following directories are subdirectories of this path * climate_bin_maps (contains [cropname]_climate_bin.tif files) * climate_percentile_yield (contains
[cropname]_percentile_yield_table.csv files)Please see the InVEST user’s guide chapter on crop production for details about how to download these data.
Returns: None.
-
natcap.invest.crop_production_regression.
tabulate_regression_results
(nutrient_table, crop_to_landcover_table, pixel_area_ha, landcover_raster_path, landcover_nodata, output_dir, file_suffix, target_table_path)¶ Write table with total yield and nutrient results by crop.
This function includes all the operations that write to results_table.csv.
Parameters: - nutrient_table (dict) – a lookup of nutrient values by crop in the form of nutrient_table[<crop>][<nutrient>].
- crop_to_landcover_table (dict) – landcover codes keyed by crop names
- pixel_area_ha (float) – area of lulc raster cells (hectares)
- landcover_raster_path (string) – path to landcover raster
- landcover_nodata (float) – landcover raster nodata value
- output_dir (string) – the file path to the output workspace.
- file_suffix (string) – string to appened to any output filenames.
- target_table_path (string) – path to ‘result_table.csv’ in the output workspace
Returns: None
Functions for reading and writing InVEST model parameters.
A datastack for InVEST is a compressed archive that includes the arguments for a model, all of the data files referenced by the arguments, and a logfile with some extra information about how the archive was created. The resulting archive can then be extracted on a different computer and should have all of the information it needs to run an InVEST model in its entirity.
A parameter set for InVEST is a JSON-formatted text file that contains all of the parameters needed to run the current model, where the parameters are located on the local hard disk. Paths to files may be either relative or absolute. If paths are relative, they are interpreted as relative to the location of the parameter set file.
A logfile for InVEST is a text file that is written to disk for each model run.
-
class
natcap.invest.datastack.
ParameterSet
(args, model_name, invest_version)¶ Bases:
tuple
-
args
¶ Alias for field number 0
-
invest_version
¶ Alias for field number 2
-
model_name
¶ Alias for field number 1
-
-
natcap.invest.datastack.
build_datastack_archive
(args, model_name, datastack_path)¶ Build an InVEST datastack from an arguments dict.
Parameters: - args (dict) – The arguments dictionary to include in the datastack.
- model_name (string) – The python-importable module string of the model these args are for.
- datastack_path (string) – The path to where the datastack archive should be written.
Returns: None
-
natcap.invest.datastack.
build_parameter_set
(args, model_name, paramset_path, relative=False)¶ Record a parameter set to a file on disk.
Parameters: - args (dict) – The args dictionary to record to the parameter set.
- model_name (string) – An identifier string for the callable or InVEST model that would accept the arguments given.
- paramset_path (string) – The path to the file on disk where the parameters should be recorded.
- relative (bool) – Whether to save the paths as relative. If
True
, The datastack assumes that paths are relative to the parent directory ofparamset_path
.
Returns: None
-
natcap.invest.datastack.
extract_datastack_archive
(datastack_path, dest_dir_path)¶ Extract a datastack to a given folder.
Parameters: - datastack_path (string) – The path to a datastack archive on disk.
- dest_dir_path (string) – The path to a directory. The contents of the demonstration datastack archive will be extracted into this directory. If the directory does not exist, it will be created.
Returns: - A dictionary of arguments from the extracted
archive. Paths to files are absolute paths.
Return type: args
(dict)
-
natcap.invest.datastack.
extract_parameter_set
(paramset_path)¶ Extract and return attributes from a parameter set.
Any string values found will have environment variables expanded. See :py:ref:os.path.expandvars and :py:ref:os.path.expanduser for details.
Parameters: paramset_path (string) – The file containing a parameter set. Returns: A ParameterSet
namedtuple with these attributes:args (dict): The arguments dict for the callable invest_version (string): The version of InVEST used to record the parameter set. model_name (string): The name of the callable or model that these arguments are intended for.
-
natcap.invest.datastack.
extract_parameters_from_logfile
(logfile_path)¶ Parse an InVEST logfile for the parameters (args) dictionary.
Argument key-value pairs are parsed, one pair per line, starting the line after the line starting with
"Arguments"
, and ending with a blank line. If no such section exists within the logfile,ValueError
will be raised.If possible, the model name and InVEST version will be parsed from the same line as
"Arguments"
, but IUI-formatted logfiles (without model name and InVEST version information) are also supported.Parameters: logfile_path (string) – The path to an InVEST logfile on disk. Returns: An instance of the ParameterSet namedtuple. If a model name and InVEST version cannot be parsed from the Arguments section of the logfile, ParameterSet.model_name
andParameterSet.invest_version
will be set todatastack.UNKNOWN
.Raises: ValueError - when no arguments could be parsed from the logfile.
-
natcap.invest.datastack.
format_args_dict
(args_dict, model_name)¶ Nicely format an arguments dictionary for writing to a stream.
If printed to a console, the returned string will be aligned in two columns representing each key and value in the arg dict. Keys are in ascending, sorted order. Both columns are left-aligned.
Parameters: - args_dict (dict) – The args dictionary to format.
- model_name (string) – The model name (in python package import format)
Returns: A formatted, unicode string.
-
natcap.invest.datastack.
get_datastack_info
(filepath)¶ Get information about a datastack.
Parameters: filepath (string) – The path to a file on disk that can be extracted as a datastack, parameter set, or logfile. Returns: "archive"
when the file is a datastack archive."json"
when the file is a json parameter set."logfile"
when the file is a text logfile.
The second item of the tuple is a ParameterSet namedtuple with the raw parsed args, modelname and invest version that the file was built with.
Return type: A 2-tuple. The first item of the tuple is one of
InVEST Carbon Edge Effect Model.
An implementation of the model described in ‘Degradation in carbon stocks near tropical forest edges’, by Chaplin-Kramer et. al (2015).
-
natcap.invest.forest_carbon_edge_effect.
combine_carbon_maps
(*carbon_maps)¶ Combine the carbon maps and leave nodata where all inputs were nodata.
Parameters: *carbon_maps (array) – arrays of carbon stocks stored in different pool types. Returns: - an array consists of all the carbon stocks from
- different pool types.
Return type: result (array)
-
natcap.invest.forest_carbon_edge_effect.
execute
(args)¶ Forest Carbon Edge Effect.
InVEST Carbon Edge Model calculates the carbon due to edge effects in tropical forest pixels.
Parameters: - args['workspace_dir'] (string) – a path to the directory that will write output and other temporary files during calculation. (required)
- args['results_suffix'] (string) – a string to append to any output file name (optional)
- args['n_nearest_model_points'] (int) – number of nearest neighbor model points to search for
- args['aoi_vector_path'] (string) – (optional) if present, a path to a shapefile that will be used to aggregate carbon stock results at the end of the run.
- args['biophysical_table_path'] (string) –
a path to a CSV table that has at least the fields ‘lucode’ and ‘c_above’. If
args['compute_forest_edge_effects'] == True
, table must also contain an ‘is_tropical_forest’ field. Ifargs['pools_to_calculate'] == 'all'
, this table must contain the fields ‘c_below’, ‘c_dead’, and ‘c_soil’.lucode
: an integer that corresponds to landcover codes in the rasterargs['lulc_raster_path']
is_tropical_forest
: either 0 or 1 indicating whether the landcover type is forest (1) or not (0). If 1, the value inc_above
is ignored and instead calculated from the edge regression model.c_above
: floating point number indicating tons of above ground carbon per hectare for that landcover type{'c_below', 'c_dead', 'c_soil'}
: three other optional carbon pools that will statically map landcover types to the carbon densities in the table.
Example:
lucode,is_tropical_forest,c_above,c_soil,c_dead,c_below 0,0,32.8,5,5.2,2.1 1,1,n/a,2.5,0.0,0.0 2,1,n/a,1.8,1.0,0.0 16,0,28.1,4.3,0.0,2.0
Note the “n/a” in
c_above
are optional since that field is ignored whenis_tropical_forest==1
. - args['lulc_raster_path'] (string) – path to a integer landcover code raster
- args['pools_to_calculate'] (string) – if “all” then all carbon pools will be calculted. If any other value only above ground carbon pools will be calculated and expect only a ‘c_above’ header in the biophysical table. If “all” model expects ‘c_above’, ‘c_below’, ‘c_dead’, ‘c_soil’ in header of biophysical_table and will make a translated carbon map for each based off the landcover map.
- args['compute_forest_edge_effects'] (boolean) – if True, requires biophysical table to have ‘is_tropical_forest’ forest field, and any landcover codes that have a 1 in this column calculate carbon stocks using the Chaplin-Kramer et. al method and ignore ‘c_above’.
- args['tropical_forest_edge_carbon_model_vector_path'] (string) –
path to a shapefile that defines the regions for the local carbon edge models. Has at least the fields ‘method’, ‘theta1’, ‘theta2’, ‘theta3’. Where ‘method’ is an int between 1..3 describing the biomass regression model, and the thetas are floating point numbers that have different meanings depending on the ‘method’ parameter. Specifically,
- method 1 (asymptotic model):
biomass = theta1 - theta2 * exp(-theta3 * edge_dist_km)
- method 2 (logarithmic model):
# NOTE: theta3 is ignored for this method biomass = theta1 + theta2 * numpy.log(edge_dist_km)
- method 3 (linear regression):
biomass = theta1 + theta2 * edge_dist_km
- method 1 (asymptotic model):
- args['biomass_to_carbon_conversion_factor'] (string/float) – Number by which to multiply forest biomass to convert to carbon in the edge effect calculation.
- args['n_workers'] (int) – (optional) The number of worker processes to use for processing this model. If omitted, computation will take place in the current process.
Returns: None
GLOBIO InVEST Model.
-
natcap.invest.globio.
execute
(args)¶ GLOBIO.
The model operates in two modes. Mode (a) generates a landcover map based on a base landcover map and information about crop yields, infrastructure, and more. Mode (b) assumes the globio landcover map is generated. These modes are used below to describe input parameters.
Parameters: - args['workspace_dir'] (string) – output directory for intermediate, temporary, and final files
- args['predefined_globio'] (boolean) – if True then “mode (b)” else “mode (a)”
- args['results_suffix'] (string) – (optional) string to append to any output files
- args['lulc_path'] (string) – used in “mode (a)” path to a base landcover map with integer codes
- args['lulc_to_globio_table_path'] (string) –
used in “mode (a)” path to table that translates the land-cover args[‘lulc_path’] to intermediate GLOBIO classes, from which they will be further differentiated using the additional data in the model. Contains at least the following fields:
- ’lucode’: Land use and land cover class code of the dataset used. LULC codes match the ‘values’ column in the LULC raster of mode (b) and must be numeric and unique.
- ’globio_lucode’: The LULC code corresponding to the GLOBIO class to which it should be converted, using intermediate codes described in the example below.
- args['infrastructure_dir'] (string) – used in “mode (a) and (b)” a path to a folder containing maps of either gdal compatible rasters or OGR compatible shapefiles. These data will be used in the infrastructure to calculation of MSA.
- args['pasture_path'] (string) – used in “mode (a)” path to pasture raster
- args['potential_vegetation_path'] (string) – used in “mode (a)” path to potential vegetation raster
- args['pasture_threshold'] (float) – used in “mode (a)”
- args['intensification_fraction'] (float) – used in “mode (a)”; a value between 0 and 1 denoting proportion of total agriculture that should be classified as ‘high input’
- args['primary_threshold'] (float) – used in “mode (a)”
- args['msa_parameters_path'] (string) – path to MSA classification parameters
- args['aoi_path'] (string) – (optional) if it exists then final MSA raster is summarized by AOI
- args['globio_lulc_path'] (string) – used in “mode (b)” path to predefined globio raster.
- args['n_workers'] (int) – (optional) The number of worker processes to use for processing this model. If omitted, computation will take place in the current process.
Returns: None
-
natcap.invest.globio.
load_msa_parameter_table
(msa_parameter_table_filename, intensification_fraction)¶ Load parameter table to a dict that to define the MSA ranges.
Parameters: - msa_parameter_table_filename (string) – path to msa csv table
- intensification_fraction (float) – a number between 0 and 1 indicating what level between msa_lu 8 and 9 to define the general GLOBIO code “12” to.
- a dictionary of the form (returns) –
- {
- ‘msa_f’: {
- valuea: msa_f_value, … valueb: … ‘<’: (bound, msa_f_value), ‘>’: (bound, msa_f_value)}
- ’msa_i_other_table’: {
- valuea: msa_i_value, … valueb: … ‘<’: (bound, msa_i_other_value), ‘>’: (bound, msa_i_other_value)}
- ’msa_i_primary’: {
- valuea: msa_i_primary_value, … valueb: … ‘<’: (bound, msa_i_primary_value), ‘>’: (bound, msa_i_primary_value)}
- ’msa_lu’: {
- valuea: msa_lu_value, …
valueb: …
‘<’: (bound, msa_lu_value),
‘>’: (bound, msa_lu_value)
12: (msa_lu_8 * (1.0 - intensification_fraction) +msa_lu_9 * intensification_fraction}
}
-
natcap.invest.globio.
make_gaussian_kernel_path
(sigma, kernel_path)¶ Create a gaussian kernel raster.
InVEST Habitat Quality model.
-
natcap.invest.habitat_quality.
execute
(args)¶ Habitat Quality.
Open files necessary for the portion of the habitat_quality model.
Parameters: - workspace_dir (string) – a path to the directory that will write output and other temporary files (required)
- lulc_cur_path (string) – a path to an input land use/land cover raster (required)
- lulc_fut_path (string) – a path to an input land use/land cover raster (optional)
- lulc_bas_path (string) – a path to an input land use/land cover raster (optional, but required for rarity calculations)
- threat_folder (string) – a path to the directory that will contain all threat rasters (required)
- threats_table_path (string) – a path to an input CSV containing data of all the considered threats. Each row is a degradation source and each column a different attribute of the source with the following names: ‘THREAT’,’MAX_DIST’,’WEIGHT’ (required).
- access_vector_path (string) – a path to an input polygon shapefile containing data on the relative protection against threats (optional)
- sensitivity_table_path (string) – a path to an input CSV file of LULC types, whether they are considered habitat, and their sensitivity to each threat (required)
- half_saturation_constant (float) – a python float that determines the spread and central tendency of habitat quality scores (required)
- suffix (string) – a python string that will be inserted into all raster path paths just before the file extension.
Example Args Dictionary:
{ 'workspace_dir': 'path/to/workspace_dir', 'lulc_cur_path': 'path/to/lulc_cur_raster', 'lulc_fut_path': 'path/to/lulc_fut_raster', 'lulc_bas_path': 'path/to/lulc_bas_raster', 'threat_raster_folder': 'path/to/threat_rasters/', 'threats_table_path': 'path/to/threats_csv', 'access_vector_path': 'path/to/access_shapefile', 'sensitivity_table_path': 'path/to/sensitivity_csv', 'half_saturation_constant': 0.5, 'suffix': '_results', }
Returns: None
-
natcap.invest.habitat_quality.
make_linear_decay_kernel_path
(max_distance, kernel_path)¶ Create a linear decay kernel as a raster.
Pixels in raster are equal to d / max_distance where d is the distance to the center of the raster in number of pixels.
Parameters: - max_distance (int) – number of pixels out until the decay is 0.
- kernel_path (string) – path to output raster whose values are in (0,1) representing distance to edge. Size is (max_distance * 2 + 1)^2
Returns: None
-
natcap.invest.habitat_quality.
map_raster_to_dict_values
(key_raster_path, out_path, attr_dict, field, out_nodata, values_required)¶ Creates a new raster from ‘key_raster’ where the pixel values from ‘key_raster’ are the keys to a dictionary ‘attr_dict’. The values corresponding to those keys is what is written to the new raster. If a value from ‘key_raster’ does not appear as a key in ‘attr_dict’ then raise an Exception if ‘raise_error’ is True, otherwise return a ‘out_nodata’
- key_raster_path - a GDAL raster path dataset whose pixel values relate to
- the keys in ‘attr_dict’
out_path - a string for the output path of the created raster attr_dict - a dictionary representing a table of values we are interested
in making into a raster- field - a string of which field in the table or key in the dictionary
- to use as the new raster pixel values
out_nodata - a floating point value that is the nodata value. raise_error - a string that decides how to handle the case where the
value from ‘key_raster’ is not found in ‘attr_dict’. If ‘raise_error’ is ‘values_required’, raise Exception, if ‘none’, return ‘out_nodata’- returns - a GDAL raster, or raises an Exception and fail if:
- raise_error is True and
- the value from ‘key_raster’ is not a key in ‘attr_dict’
-
natcap.invest.habitat_quality.
raster_pixel_count
(raster_path)¶ Count unique pixel values in raster.
Parameters: raster_path (string) – path to a raster Returns: dict of pixel values to frequency.
-
natcap.invest.habitat_quality.
resolve_ambiguous_raster_path
(path, raise_error=True)¶ Determine real path when we don’t know true path extension.
Parameters: - path (string) – file path that includes the name of the file but not its extension
- raise_error (boolean) – if True then function will raise an ValueError if a valid raster file could not be found.
Returns: the full path, plus extension, to the valid raster.
Habitat risk assessment (HRA) model for InVEST.
-
natcap.invest.hra.
execute
(args)¶ InVEST Habitat Risk Assessment (HRA) Model.
Parameters: - args['workspace_dir'] (str) – a path to the output workspace folder. It will overwrite any files that exist if the path already exists.
- args['results_suffix'] (str) – a string appended to each output file path. (optional)
- args['info_table_path'] (str) – a path to the CSV or Excel file that
contains the name of the habitat (H) or stressor (s) on the
NAME
column that matches the names in criteria_table_path. Each H/S has its corresponding vector or raster path on thePATH
column. TheSTRESSOR BUFFER (meters)
column should have a buffer value if theTYPE
column is a stressor. - args['criteria_table_path'] (str) – a path to the CSV or Excel file that contains the set of criteria ranking of each stressor on each habitat.
- args['resolution'] (int) – a number representing the desired pixel dimensions of output rasters in meters.
- args['max_rating'] (str, int or float) – a number representing the highest potential value that should be represented in rating in the criteria scores table.
- args['risk_eq'] (str) – a string identifying the equation that should be used in calculating risk scores for each H-S overlap cell. This will be either ‘Euclidean’ or ‘Multiplicative’.
- args['decay_eq'] (str) – a string identifying the equation that should be used in calculating the decay of stressor buffer influence. This can be ‘None’, ‘Linear’, or ‘Exponential’.
- args['aoi_vector_path'] (str) – a path to the shapefile containing one or more planning regions used to get the average risk value for each habitat-stressor combination over each area. Optionally, if each of the shapefile features contain a ‘name’ field, it will be used as a way of identifying each individual shape.
- args['n_workers'] (int) – the number of worker processes to use for processing this model. If omitted, computation will take place in the current process. (optional)
- args['visualize_outputs'] (bool) – if True, create output GeoJSONs and save them in a visualization_outputs folder, so users can visualize results on the web app. Default to True if not specified. (optional)
Returns: None.
Pollinator service model for InVEST.
-
natcap.invest.pollination.
execute
(args)¶ InVEST Pollination Model.
Parameters: - args['workspace_dir'] (string) – a path to the output workspace folder. Will overwrite any files that exist if the path already exists.
- args['results_suffix'] (string) – string appended to each output file path.
- args['landcover_raster_path'] (string) – file path to a landcover raster.
- args['guild_table_path'] (string) –
file path to a table indicating the bee species to analyze in this model run. Table headers must include:
- ’species’: a bee species whose column string names will
- be referred to in other tables and the model will output analyses per species.
- one or more columns matching _NESTING_SUITABILITY_PATTERN
- with values in the range [0.0, 1.0] indicating the suitability of the given species to nest in a particular substrate.
- one or more columns matching _FORAGING_ACTIVITY_RE_PATTERN
- with values in the range [0.0, 1.0] indicating the relative level of foraging activity for that species during a particular season.
- _ALPHA_HEADER the sigma average flight distance of that bee
- species in meters.
- ’relative_abundance’: a weight indicating the relative
- abundance of the particular species with respect to the sum of all relative abundance weights in the table.
- args['landcover_biophysical_table_path'] (string) –
path to a table mapping landcover codes in args[‘landcover_path’] to indexes of nesting availability for each nesting substrate referenced in guilds table as well as indexes of abundance of floral resources on that landcover type per season in the bee activity columns of the guild table.
All indexes are in the range [0.0, 1.0].
- Columns in the table must be at least
- ’lucode’: representing all the unique landcover codes in
- the raster ast args[‘landcover_path’]
- For every nesting matching _NESTING_SUITABILITY_PATTERN in the guild stable, a column matching the pattern in _LANDCOVER_NESTING_INDEX_HEADER.
- For every season matching _FORAGING_ACTIVITY_RE_PATTERN in the guilds table, a column matching the pattern in _LANDCOVER_FLORAL_RESOURCES_INDEX_HEADER.
- args['farm_vector_path'] (string) –
(optional) path to a single layer polygon shapefile representing farms. If present will trigger the farm yield component of the model.
The layer must have at least the following fields:
- season (string): season in which the farm needs pollination
- crop_type (string): a text field to identify the crop type for
- summary statistics.
- half_sat (float): a real in the range [0.0, 1.0] representing
- the proportion of wild pollinators to achieve a 50% yield of that crop.
- p_dep (float): a number in the range [0.0, 1.0]
- representing the proportion of yield dependent on pollinators.
- p_managed (float): proportion of pollinators that come from
- non-native/managed hives.
- fr_[season] (float): one or more fields that match this pattern
- such that season also matches the season headers in the biophysical and guild table. Any areas that overlap the landcover map will replace seasonal floral resources with this value. Ranges from 0..1.
- n_[substrate] (float): One or more fields that match this
- pattern such that substrate also matches the nesting substrate headers in the biophysical and guild table. Any areas that overlap the landcover map will replace nesting substrate suitability with this value. Ranges from 0..1.
- args['n_workers'] (int) – (optional) The number of worker processes to use for processing this model. If omitted, computation will take place in the current process.
Returns: None
RouteDEM for exposing the natcap.invest’s routing package to UI.
-
natcap.invest.routedem.
execute
(args)¶ RouteDEM: Hydrological routing.
This model exposes the pygeoprocessing D8 and Multiple Flow Direction routing functionality as an InVEST model.
This tool will always fill pits on the input DEM.
Parameters: - args['workspace_dir'] (string) – output directory for intermediate, temporary, and final files
- args['results_suffix'] (string) – (optional) string to append to any output file names
- args['dem_path'] (string) – path to a digital elevation raster
- args['dem_band_index'] (int) – Optional. The band index to operate on. If not provided, band index 1 is assumed.
- args['algorithm'] (string) – The routing algorithm to use. Must be one of ‘D8’ or ‘MFD’ (case-insensitive). Required when calculating flow direction, flow accumulation, stream threshold, and downstream distance.
- args['calculate_flow_direction'] (bool) – If True, model will calculate flow direction for the filled DEM.
- args['calculate_flow_accumulation'] (bool) – If True, model will calculate a flow accumulation raster. Only applies when args[‘calculate_flow_direction’] is True.
- args['calculate_stream_threshold'] (bool) – if True, model will
calculate a stream classification layer by thresholding flow
accumulation to the provided value in
args['threshold_flow_accumulation']
. Only applies when args[‘calculate_flow_accumulation’] and args[‘calculate_flow_direction’] are True. - args['threshold_flow_accumulation'] (int) – The number of upstream cells that must flow into a cell before it’s classified as a stream.
- args['calculate_downstream_distance'] (bool) – If True, and a stream threshold is calculated, model will calculate a downstream distance raster in units of pixels. Only applies when args[‘calculate_flow_accumulation’], args[‘calculate_flow_direction’], and args[‘calculate_stream_threshold’] are all True.
- args['calculate_slope'] (bool) – If True, model will calculate a slope raster from the DEM.
- args['n_workers'] (int) – The
n_workers
parameter to pass to the task graph. The default is-1
if not provided.
Returns: None
Scenario Generation: Proximity Based.
-
natcap.invest.scenario_gen_proximity.
execute
(args)¶ Scenario Generator: Proximity-Based.
Main entry point for proximity based scenario generator model.
Parameters: - args['workspace_dir'] (string) – output directory for intermediate, temporary, and final files
- args['results_suffix'] (string) – (optional) string to append to any output files
- args['base_lulc_path'] (string) – path to the base landcover map
- args['replacment_lucode'] (string or int) – code to replace when converting pixels
- args['area_to_convert'] (string or float) – max area (Ha) to convert
- args['focal_landcover_codes'] (string) – a space separated string of landcover codes that are used to determine the proximity when refering to “towards” or “away” from the base landcover codes
- args['convertible_landcover_codes'] (string) – a space separated string of landcover codes that can be converted in the generation phase found in args[‘base_lulc_path’].
- args['n_fragmentation_steps'] (string) – an int as a string indicating the number of steps to take for the fragmentation conversion
- args['aoi_path'] (string) – (optional) path to a shapefile that indicates area of interest. If present, the expansion scenario operates only under that AOI and the output raster is clipped to that shape.
- args['convert_farthest_from_edge'] (boolean) – if True will run the conversion simulation starting from the furthest pixel from the edge and work inwards. Workspace will contain output files named ‘toward_base{suffix}.{tif,csv}.
- args['convert_nearest_to_edge'] (boolean) – if True will run the conversion simulation starting from the nearest pixel on the edge and work inwards. Workspace will contain output files named ‘toward_base{suffix}.{tif,csv}.
- args['n_workers'] (int) – (optional) The number of worker processes to use for processing this model. If omitted, computation will take place in the current process.
Returns: None.
InVEST Sediment Delivery Ratio (SDR) module.
- The SDR method in this model is based on:
- Winchell, M. F., et al. “Extension and validation of a geographic information system-based method for calculating the Revised Universal Soil Loss Equation length-slope factor for erosion risk assessments in large watersheds.” Journal of Soil and Water Conservation 63.3 (2008): 105-111.
-
natcap.invest.sdr.
execute
(args)¶ Sediment Delivery Ratio.
This function calculates the sediment export and retention of a landscape using the sediment delivery ratio model described in the InVEST user’s guide.
Parameters: - args['workspace_dir'] (string) – output directory for intermediate, temporary, and final files
- args['results_suffix'] (string) – (optional) string to append to any output file names
- args['dem_path'] (string) – path to a digital elevation raster
- args['erosivity_path'] (string) – path to rainfall erosivity index raster
- args['erodibility_path'] (string) – a path to soil erodibility raster
- args['lulc_path'] (string) – path to land use/land cover raster
- args['watersheds_path'] (string) – path to vector of the watersheds
- args['biophysical_table_path'] (string) – path to CSV file with biophysical information of each land use classes. contain the fields ‘usle_c’ and ‘usle_p’
- args['threshold_flow_accumulation'] (number) – number of upstream pixels on the dem to threshold to a stream.
- args['k_param'] (number) – k calibration parameter
- args['sdr_max'] (number) – max value the SDR
- args['ic_0_param'] (number) – ic_0 calibration parameter
- args['drainage_path'] (string) – (optional) path to drainage raster that is used to add additional drainage areas to the internally calculated stream layer
- args['n_workers'] (int) – if present, indicates how many worker processes should be used in parallel processing. -1 indicates single process mode, 0 is single process but non-blocking mode, and >= 1 is number of processes.
Returns: None.
InVEST specific code utils.
-
class
natcap.invest.utils.
ThreadFilter
(thread_name)¶ Bases:
logging.Filter
Filters out log messages issued by the given thread.
Any log messages generated by a thread with the name matching the threadname provided to the constructor will be excluded.
-
filter
(record)¶ Filter the given log record.
Parameters: record (log record) – The log record to filter. Returns: True if the record should be included, false if not.
-
-
natcap.invest.utils.
build_file_registry
(base_file_path_list, file_suffix)¶ Combine file suffixes with key names, base filenames, and directories.
Parameters: - base_file_tuple_list (list) – a list of (dict, path) tuples where the dictionaries have a ‘file_key’: ‘basefilename’ pair, or ‘file_key’: list of ‘basefilename’s. ‘path’ indicates the file directory path to prepend to the basefile name.
- file_suffix (string) – a string to append to every filename, can be empty string
Returns: dictionary of ‘file_keys’ from the dictionaries in base_file_tuple_list mapping to full file paths with suffixes or lists of file paths with suffixes depending on the original type of the ‘basefilename’ pair.
Raises: - ValueError if there are duplicate file keys or duplicate file paths.
- ValueError if a path is not a string or a list of strings.
-
natcap.invest.utils.
build_lookup_from_csv
(table_path, key_field, to_lower=True, warn_if_missing=True)¶ Read a CSV table into a dictionary indexed by key_field.
Creates a dictionary from a CSV whose keys are unique entries in the CSV table under the column named by key_field and values are dictionaries indexed by the other columns in table_path including key_field whose values are the values on that row of the CSV table.
Parameters: - table_path (string) – path to a CSV file containing at least the header key_field
- key_field – (string): a column in the CSV file at table_path that can uniquely identify each row in the table.
- to_lower (bool) – if True, converts all unicode in the CSV, including headers and values to lowercase, otherwise uses raw string values.
- warn_if_missing (bool) – If True, warnings are logged if there are empty headers or value rows.
Returns: - a dictionary of the form {
key_field_0: {csv_header_0: value0, csv_header_1: value1…}, key_field_1: {csv_header_0: valuea, csv_header_1: valueb…}
}
if to_lower all strings including key_fields and values are converted to lowercase unicode.
Return type: lookup_dict (dict)
-
natcap.invest.utils.
capture_gdal_logging
(*args, **kwds)¶ Context manager for logging GDAL errors with python logging.
GDAL error messages are logged via python’s logging system, at a severity that corresponds to a log level in
logging
. Error messages are logged with theosgeo.gdal
logger.Parameters: None – Returns: None
-
natcap.invest.utils.
exponential_decay_kernel_raster
(expected_distance, kernel_filepath)¶ Create a raster-based exponential decay kernel.
The raster created will be a tiled GeoTiff, with 256x256 memory blocks.
Parameters: - expected_distance (int or float) – The distance (in pixels) of the kernel’s radius, the distance at which the value of the decay function is equal to 1/e.
- kernel_filepath (string) – The path to the file on disk where this kernel should be stored. If this file exists, it will be overwritten.
Returns: None
-
natcap.invest.utils.
log_to_file
(*args, **kwds)¶ Log all messages within this context to a file.
Parameters: - logfile (string) – The path to where the logfile will be written. If there is already a file at this location, it will be overwritten.
- exclude_threads=None (list) – If None, logging from all threads will be included in the log. If a list, it must be a list of string thread names that should be excluded from logging in this file.
- logging_level=logging.NOTSET (int) – The logging threshold. Log
messages with a level less than this will be automatically
excluded from the logfile. The default value (
logging.NOTSET
) will cause all logging to be captured. - log_fmt=LOG_FMT (string) – The logging format string to use. If not
provided,
utils.LOG_FMT
will be used. - date_fmt (string) – The logging date format string to use. If not provided, ISO8601 format will be used.
Yields: handler
–- An instance of
logging.FileHandler
that represents the file that is being written to.
Returns: None
-
natcap.invest.utils.
make_directories
(directory_list)¶ Create directories in directory_list if they do not already exist.
-
natcap.invest.utils.
make_suffix_string
(args, suffix_key)¶ Make an InVEST appropriate suffix string.
Creates an InVEST appropriate suffix string given the args dictionary and suffix key. In general, prepends an ‘_’ when necessary and generates an empty string when necessary.
Parameters: - args (dict) – the classic InVEST model parameter dictionary that is passed to execute.
- suffix_key (string) – the key used to index the base suffix.
Returns: - If suffix_key is not in args, or args[‘suffix_key’] is “”
return “”,
- If args[‘suffix_key’] starts with ‘_’ return args[‘suffix_key’]
else return ‘_’+`args[‘suffix_key’]`
-
natcap.invest.utils.
mean_pixel_size_and_area
(pixel_size_tuple)¶ Convert to mean and raise Exception if they are not close.
- Parameter:
- pixel_size_tuple (tuple): a 2 tuple indicating the x/y size of a
- pixel.
Returns: tuple of (mean absolute average of pixel_size, area of pixel size) Raises: ValueError if the dimensions of pixel_size_tuple are not almost – square.
-
natcap.invest.utils.
prepare_workspace
(*args, **kwds)¶
-
natcap.invest.utils.
sandbox_tempdir
(*args, **kwds)¶ Create a temporary directory for this context and clean it up on exit.
Parameters are identical to those for
tempfile.mkdtemp()
.When the context manager exits, the created temporary directory is recursively removed.
Parameters: - suffix='' (string) – a suffix for the name of the directory.
- prefix='tmp' (string) – the prefix to use for the directory name.
- dir=None (string or None) – If a string, a directory that should be the parent directory of the new temporary directory. If None, tempfile will determine the appropriate tempdir to use as the parent folder.
Yields: sandbox
(string) – The path to the new folder on disk.Returns: None
Common validation utilities for InVEST models.
-
natcap.invest.validation.
CHECK_ALL_KEYS
= None¶ A flag to pass to the validation context manager indicating that all keys should be checked.
-
class
natcap.invest.validation.
ValidationContext
(args, limit_to)¶ Bases:
object
An object to represent a validation context.
A validation context reduces the amount of boilerplate code needed within an InVEST validation function to produce validation warnings that are consistent with the InVEST Validation API.
-
is_arg_complete
(key, require=False)¶ Test if a given argument is complete and should be validated.
An argument is complete if:
- The value associated with
key
is neither''
orNone
- The key-value pair is in
self.args
- The key should be validated (the key matches the value of
self.limit_to
orself.limit_to == None
)
If the argument is incomplete and
require == True
, a warning is recorded in the ValidationContext’s warnings list.Parameters: - key (string) – The key to test.
- require=False (bool) – Whether the parameter is required.
Returns: A
bool
, indicating whether the argument is complete.- The value associated with
-
warn
(message, keys)¶ Record a warning in the internal warnings list.
Parameters: - message (string) – The message of the warning to log.
- keys (iterable) – An iterable of string keys that the message refers to.
-
-
natcap.invest.validation.
invest_validator
(validate_func)¶ Decorator to enforce characteristics of validation inputs and outputs.
Attributes of inputs and outputs that are enforced are:
args
parameter tovalidate
must be adict
limit_to
parameter tovalidate
must be eitherNone
or a string (str
orunicode
) that exists in theargs
dict.All keys in
args
must be stringsDecorated
validate
func must return a list of 2-tuples, where each 2-tuple conforms to these rules:- The first element of the 2-tuple is an iterable of strings. It is an error for the first element to be a string.
- The second element of the 2-tuple is a string error message.
In addition, this validates the
n_workers
argument if it’s included.Raises: AssertionError when an invalid format is found. Example
from natcap.invest import validation @validation.invest_validator def validate(args, limit_to=None):
# do your validation here
InVEST Wave Energy Model Core Code
-
exception
natcap.invest.wave_energy.
IntersectionError
¶ Bases:
exceptions.Exception
A custom error message for when the AOI does not intersect any wave data points.
-
natcap.invest.wave_energy.
execute
(args)¶ Wave Energy.
Executes both the biophysical and valuation parts of the wave energy model (WEM). Files will be written on disk to the intermediate and output directories. The outputs computed for biophysical and valuation include: wave energy capacity raster, wave power raster, net present value raster, percentile rasters for the previous three, and a point shapefile of the wave points with attributes.
Parameters: - workspace_dir (str) – Where the intermediate and output folder/files will be saved. (required)
- wave_base_data_path (str) – Directory location of wave base data including WAVEWATCH III (WW3) data and analysis area shapefile. (required)
- analysis_area_path (str) – A string identifying the analysis area of interest. Used to determine wave data shapefile, wave data text file, and analysis area boundary shape. (required)
- aoi_path (str) – A polygon OGR vector outlining a more detailed area within the analysis area. This vector should be projected with linear units being in meters. (required to run Valuation model)
- machine_perf_path (str) – The path of a CSV file that holds the machine performance table. (required)
- machine_param_path (str) – The path of a CSV file that holds the machine parameter table. (required)
- dem_path (str) – The path of the Global Digital Elevation Model (DEM). (required)
- suffix (str) – A python string of characters to append to each output filename (optional)
- valuation_container (boolean) – Indicates whether the model includes valuation
- land_gridPts_path (str) – A CSV file path containing the Landing and Power Grid Connection Points table. (required for Valuation)
- machine_econ_path (str) – A CSV file path for the machine economic parameters table. (required for Valuation)
- number_of_machines (int) – An integer specifying the number of machines for a wave farm site. (required for Valuation)
- n_workers (int) – The number of worker processes to use for processing this model. If omitted, computation will take place in the current process. (optional)
Example Args Dictionary:
{ 'workspace_dir': 'path/to/workspace_dir', 'wave_base_data_path': 'path/to/base_data_dir', 'analysis_area_path': 'West Coast of North America and Hawaii', 'aoi_path': 'path/to/vector', 'machine_perf_path': 'path/to/csv', 'machine_param_path': 'path/to/csv', 'dem_path': 'path/to/raster', 'suffix': '_results', 'valuation_container': True, 'land_gridPts_path': 'path/to/csv', 'machine_econ_path': 'path/to/csv', 'number_of_machines': 28, }
InVEST Wind Energy model.
-
natcap.invest.wind_energy.
execute
(args)¶ Wind Energy.
This module handles the execution of the wind energy model given the following dictionary:
Parameters: - workspace_dir (str) – a path to the output workspace folder (required)
- wind_data_path (str) – path to a CSV file with the following header: [‘LONG’,’LATI’,’LAM’, ‘K’, ‘REF’]. Each following row is a location with at least the Longitude, Latitude, Scale (‘LAM’), Shape (‘K’), and reference height (‘REF’) at which the data was collected (required)
- aoi_vector_path (str) – a path to an OGR polygon vector that is projected in linear units of meters. The polygon specifies the area of interest for the wind data points. If limiting the wind farm bins by distance, then the aoi should also cover a portion of the land polygon that is of interest (optional for biophysical and no distance masking, required for biophysical and distance masking, required for valuation)
- bathymetry_path (str) – a path to a GDAL raster that has the depth values of the area of interest (required)
- land_polygon_vector_path (str) – a path to an OGR polygon vector that provides a coastline for determining distances from wind farm bins. Enabled by AOI and required if wanting to mask by distances or run valuation
- global_wind_parameters_path (str) – a float for the average distance in kilometers from a grid connection point to a land connection point (required for valuation if grid connection points are not provided)
- suffix (str) – a str to append to the end of the output files (optional)
- turbine_parameters_path (str) – a path to a CSV file that holds the turbines biophysical parameters as well as valuation parameters (required)
- number_of_turbines (int) – an integer value for the number of machines for the wind farm (required for valuation)
- min_depth (float) – a float value for the minimum depth for offshore wind farm installation (meters) (required)
- max_depth (float) – a float value for the maximum depth for offshore wind farm installation (meters) (required)
- min_distance (float) – a float value for the minimum distance from shore for offshore wind farm installation (meters) The land polygon must be selected for this input to be active (optional, required for valuation)
- max_distance (float) – a float value for the maximum distance from shore for offshore wind farm installation (meters) The land polygon must be selected for this input to be active (optional, required for valuation)
- valuation_container (boolean) – Indicates whether model includes valuation
- foundation_cost (float) – a float representing how much the foundation will cost for the specific type of turbine (required for valuation)
- discount_rate (float) – a float value for the discount rate (required for valuation)
- grid_points_path (str) – a path to a CSV file that specifies the landing and grid point locations (optional)
- avg_grid_distance (float) – a float for the average distance in kilometers from a grid connection point to a land connection point (required for valuation if grid connection points are not provided)
- price_table (boolean) – a bool indicating whether to use the wind energy price table or not (required)
- wind_schedule (str) – a path to a CSV file for the yearly prices of wind energy for the lifespan of the farm (required if ‘price_table’ is true)
- wind_price (float) – a float for the wind energy price at year 0 (required if price_table is false)
- rate_change (float) – a float as a percent for the annual rate of change in the price of wind energy. (required if price_table is false)
- n_workers (int) – The number of worker processes to use for processing this model. If omitted, computation will take place in the current process. (optional)
Example Args Dictionary:
{ 'workspace_dir': 'path/to/workspace_dir', 'wind_data_path': 'path/to/file', 'aoi_vector_path': 'path/to/shapefile', 'bathymetry_path': 'path/to/raster', 'land_polygon_vector_path': 'path/to/shapefile', 'global_wind_parameters_path': 'path/to/csv', 'suffix': '_results', 'turbine_parameters_path': 'path/to/csv', 'number_of_turbines': 10, 'min_depth': 3, 'max_depth': 60, 'min_distance': 0, 'max_distance': 200000, 'valuation_container': True, 'foundation_cost': 3.4, 'discount_rate': 7.0, 'grid_points_path': 'path/to/csv', 'avg_grid_distance': 4, 'price_table': True, 'wind_schedule': 'path/to/csv', 'wind_price': 0.4, 'rate_change': 0.0, }
Returns: None
init module for natcap.invest.
-
natcap.invest.
local_dir
(source_file)¶ Return the path to where source_file would be on disk.
If this is frozen (as with PyInstaller), this will be the folder with the executable in it. If not, it’ll just be the foldername of the source_file being passed in.
Module contents¶
Indices and tables¶
Ecosystem Service Analysis Tools¶
Coastal Protection Package¶
Coastal Vulnerability¶
Coastal Vulnerability Core¶
Coastal Vulnerability Cython Core¶
Coastal Vulnerability Post Processing¶
Module contents¶
InVEST Coastal Vulnerability
-
natcap.invest.coastal_vulnerability.
aggregate_population_density
(base_shore_point_vector_path, base_population_raster_path, search_radius, model_resolution, workspace_dir, file_suffix, target_pickle_path)¶ Get population density within a search radius of points.
Parameters: - base_shore_point_vector_path (string) – path to point vector
- base_population_raster_path (string) – path to raster with population values
- search_radius (float) – radius in meters around each point to search for valid population pixels.
- model_resolution (float) – distance in meters of the shore_point spacing, used here as a target pixel size in warp raster.
- workspace_dir (string) – path to directory for intermediate files
- file_suffix (string) – appended to any output filename.
- target_pickle_path (string) – path to pickle file storing dict keyed by shore point fid.
Returns: None
-
natcap.invest.coastal_vulnerability.
assemble_results_and_calculate_exposure
(risk_id_path_list, habitat_protection_path, base_point_vector_path, target_intermediate_vector_path, target_output_vector_path, target_output_csv_path)¶ Calculate the final exposure score, with and without habitat protection.
Parameters: - risk_id_path_list (list) –
list of 3-tuples like: (‘relief.pickle’, True, ‘R_relief’)
- string: path to pickle with intermediate
- exposure values for a single variable
- bool:
- if True: variable contains values that need binning
- by percentile to convert to 1-5 ranks.
if False: variable is already on the 1-5 rank scale.
- string: if not None, this variable is included in the
- final exposure score, and ‘R_name’ becomes a fieldname in target_point_vector_path.
- habitat_protection_path (string) – path to csv file with the intermediate habitat ranks.
- base_point_vector_path (string) – path to shore point vector.
- target_intermediate_vector_path (string) – path to point vector populated with fields and raw values for all risk variables that require binning to ranks.
- target_output_vector_path (string) – path to point vector populated with fields and rank values for all risk variables, final exposure, exposure without habitats, and population density.
- target_output_csv_path (string) – path to a csv copy of target_output_vector_path.
Returns: None.
- risk_id_path_list (list) –
-
natcap.invest.coastal_vulnerability.
calculate_final_risk
(output_vector_path, output_csv_path)¶ Apply geometric mean calculation to variables at each shore point.
This function modifies the ‘output_vector_path’ file by adding new fields.
Parameters: - output_vector_path (string) – path to shore point vector populated with numeric fields named with the ‘R_’ prefix. These fields are included in the geometric mean.
- output_csv_path (string) – path to csv copy of the final ‘output_vector_path’.
Returns: None
-
natcap.invest.coastal_vulnerability.
calculate_geomorphology_exposure
(geomorphology_vector_path, geomorphology_fill_value, base_shore_point_vector_path, model_resolution, target_pickle_path)¶ Join geomorphology ranks to shore points by proximity.
Buffer each shore point by half the model_resolution and find all geomorphology types present around each point. Return the average of the geomorphology ranks, unless none are present, then return the
Parameters: - geomorphology_vector_path (string) – path to polyline vector with an integer attribute named RANK that contains values (1, 2, 3, 4, 5).
- geomorphology_fill_value (int) – integer in (1, 2, 3, 4, 5).
- base_shore_point_vector_path (string) – path to point vector
- model_resolution (float) – distance in meters of the shore_point spacing.
- target_pickle_path (string) – path to pickle file storing dict keyed by point fid.
Returns: None
-
natcap.invest.coastal_vulnerability.
calculate_habitat_rank
(habitat_pickle_list, target_habitat_protection_path)¶ Combine dicts of habitat ranks into a dataframe and calcuate Rhab.
Parameters: - habitat_pickle_list (list) – list of file paths to pickled dictionaries in the form of: {‘habitat_id’: {<fid0>: 5, <fid1>.: 5, <fid2>: 5}}
- target_habitat_protection_path (string) –
path to a csv file with a row for each shore point, and a header like:
’fid’,’kelp’,’eelgrass’,’coral’,’R_hab’
Returns: None
-
natcap.invest.coastal_vulnerability.
calculate_relief_exposure
(base_shore_point_vector_path, base_dem_path, dem_averaging_radius, model_resolution, workspace_dir, file_suffix, target_relief_pickle_path)¶ Calculate average of DEM pixels within a radius of shore points.
Parameters: - base_shore_point_vector_path (string) – path to a shore point vector.
- base_dem_path (string) – path to a DEM raster.
- dem_averaging_radius (float) – distance in meters
- model_resolution (float) – distance in meters of the shore_point spacing, used here as a target pixel size in warp raster.
- workspace_dir (string) – path to a directory for intermediate files
- file_suffix (string) – to be appended to output filenames
- target_relief_pickle_path (string) – path to pickle file storing dict keyed by shore point fid.
Returns: None.
-
natcap.invest.coastal_vulnerability.
calculate_surge_exposure
(base_shore_point_vector_path, shelf_contour_path, aoi_vector_path, target_surge_pickle_path)¶ Calculate surge potential as distance to nearest point on a contour.
Parameters: - base_shore_point_vector_path (string) – path to a point shapefile to for relief point analysis.
- shelf_contour_path (string) – path to a polyline vector.
- aoi_vector_path (string) – path to a polygon vector.
- target_surge_pickle_path (string) – path to pickle file storing dict keyed by shore point fid.
Returns: None.
-
natcap.invest.coastal_vulnerability.
calculate_wave_exposure
(base_fetch_point_vector_path, max_fetch_distance, target_wave_exposure_pickle_path)¶ Calculate wave exposure values at each shore point.
Parameters: - base_fetch_point_vector_path (string) – path to a point shapefile that contains 16 ‘WavP_[direction]’ fields, ‘WavPPCT[direction]’ fields, ‘fdist_[direction]’ fields, a single H, and a single T field.
- max_fetch_distance (float) – max fetch distance before a wind fetch ray is terminated.
- target_wave_exposure_pickle_path (string) – path to pickle file storing dict keyed by shore point fid.
Returns: None
-
natcap.invest.coastal_vulnerability.
calculate_wind_exposure
(base_shore_point_vector_path, landmass_polygon_pickle_path, landmass_line_rtree_path, landmass_lines_pickle_path, workspace_dir, file_suffix, max_fetch_distance, target_shore_point_vector_path, target_wind_exposure_pickle_path)¶ Calculate wind exposure for each shore point.
Parameters: - base_shore_point_vector_path (string) – path to a point vector with WWIII variables in the table.
- landmass_polygon_pickle_path (string) – path to pickle storing shapely polygon geometry of the landmass.
- landmass_line_rtree_path (string) – path to rtree file indexing bounds of line geometries.
- landmass_lines_pickle_path (string) – path to pickle storing list of shapely line geometries. List index must match index of landmass_line_rtree_path.
- workspace_dir (string) – path to a directory for intermediate files
- file_suffix (string) – to be appended to output filenames
- max_fetch_distance (float) – maximum fetch distance for a ray in meters.
- target_shore_point_vector_path (string) – path to target point file, will be a copy of base_shore_point_vector_path’s geometry with an ‘REI’ (relative exposure index) field added.
- target_wind_exposure_pickle_path (string) – path to pickle file storing dict keyed by shore point fid, with wind exposure values.
Returns: None
-
natcap.invest.coastal_vulnerability.
clip_and_project_raster
(base_raster_path, clipping_box, target_srs_wkt, model_resolution, working_dir, file_suffix, target_raster_path)¶ Clip a raster to a box in the raster’s native SRS, then reproject.
Parameters: - base_raster_path (string) – path to a gdal raster
- clipping_box (list) – sequence of floats that are coordinates in the target_srs [minx, miny, maxx, maxy]
- target_srs_wkt (string) – well-known-text spatial reference system
- model_resolution (float) – value for target pixel size
- working_dir (string) – path to directory for intermediate files
- file_suffix (string) – appended to any output filename.
- target_raster_path (string) – path to clipped and warped raster.
Returns: None
-
natcap.invest.coastal_vulnerability.
clip_and_project_vector
(base_vector_path, clipping_box, target_srs_wkt, tmp_vector_path, target_vector_path)¶ Clip a vector to a box in the vector’s native SRS, then reproject.
Parameters: - base_vector_path (string) – path to polygon or multipolygon type vector
- clipping_box (list) – sequence of floats that are coordinates in the target_srs [minx, miny, maxx, maxy]
- target_srs_wkt (string) – well-known-text spatial reference system
- tmp_vector_path (string) – path to clipped but unprojected .gpkg vector
- target_vector_path (string) – path to clipped and projected .gpkg vector
Returns: None
-
natcap.invest.coastal_vulnerability.
execute
(args)¶ InVEST Coastal Vulnerability Model.
For points along a coastline, evaluate the relative exposure of points to coastal hazards based on up to eight biophysical hazard indices. Also quantify the role of habitats in reducing the hazard. Optionally summarize the population density in proximity to each shore point.
Parameters: - args['workspace_dir'] (string) – (required) a path to the directory that will write output and other temporary files during calculation.
- args['results_suffix'] (string) – (optional) appended to any output filename.
- args['aoi_vector_path'] (string) – (required) path to a polygon vector that is projected in a coordinate system with units of meters. The polygon should intersect the landmass and the shelf contour line.
- args['model_resolution'] (string) – (required) distance in meters. Points are spaced along the coastline at intervals of this distance.
- args['landmass_vector_path'] (string) – (required) path to a polygon vector representing landmasses in the region of interest.
- args['wwiii_vector_path'] (string) – (required) path to a point vector containing wind and wave information across the region of interest.
- args['max_fetch_distance'] (string) – (required) maximum distance in meters to extend rays from shore points. Points with rays equal to this distance will accumulate ocean-driven wave exposure along those rays and local-wind-driven wave exposure along the shorter rays.
- args['shelf_contour_vector_path'] (string) – (required) path to a polyline vector delineating edges of the continental shelf or other bathymetry contour.
- args['dem_path'] (string) – (required) path to a raster representing the elevation on land in the region of interest.
- args['dem_averaging_radius'] (int or float) – (required) a value >= 0. The radius in meters around each shore point in which to compute the average elevation.
- args['habitat_table_path'] (string) –
(rqeuired) path to a CSV file with the following four fields:
’id’: unique string to represent each habitat ‘path’: absolute or relative path to a polygon vector ‘rank’: integer from 1 to 5 representing the relativeprotection offered by this habitat- ’protection distance (m)’: integer or float used as a
- search radius around each shore point.
- args['geomorphology_vector_path'] (string) – (optional) path to a polyline vector that has a field called “RANK” with values from 1 to 5 in the attribute table.
- args['geomorphology_fill_value'] (int) – (optional) a value from 1 to 5 that will be used as a geomorphology rank for any points not proximate to the geomorphology_vector_path.
- args['population_rater_path'] (string) – (optional) path a raster with values of total population per pixel.
- args['population_radius'] (int or float) – (optional) a value >= 0. The radius in meters around each shore point in which to compute the population density.
- args['slr_vector_path'] (string) – (optional) path to point vector containing the field args[‘slr_field’].
- args['slr_field'] (string) – name of a field in args[‘slr_vector_path’] containing numeric values.
- args['n_workers'] (int) – (optional) The number of worker processes to use for processing this model. If omitted, computation will take place in the current process.
Returns: None
-
natcap.invest.coastal_vulnerability.
geometry_to_lines
(geometry)¶ Convert a geometry object to a list of lines.
-
natcap.invest.coastal_vulnerability.
interpolate_sealevelrise_points
(base_shore_point_vector_path, slr_points_vector_path, slr_fieldname, target_pickle_path)¶ Spatial join of sea-level rise data to shore points.
Finds nearest sea-level points to each shore point and calculates a weighted average of values with inverse-distance weights.
Parameters: - base_shore_point_vector_path (string) – path to point vector
- slr_points_vector_path (string) – path to point vector containing the field slr_fieldname.
- slr_fieldname (string) – name of a field containing numeric values
- target_pickle_path (string) – path to pickle file storing dict keyed by point fid.
Returns: None
-
natcap.invest.coastal_vulnerability.
interpolate_shore_points
(aoi_vector_path, landmass_lines_pickle_path, model_resolution, target_vector_path)¶ Create points along a polyline geometry at a specified interval.
Parameters: - aoi_vector_path (string) – path to polygon vector used to define boundaries for adding points.
- landmass_lines_pickle_path (string) – path to pickle file containing list of shapely linestrings
- model_resolution (float) – distance in meters for the point’s spacing
- target_vector_path (string) – path to .gpkg point vector
Returns: None
-
natcap.invest.coastal_vulnerability.
interpolate_wwiii_to_shore
(base_shore_point_vector_path, wwiii_vector_path, target_shore_point_vector_path)¶ Spatial join of Wave Watch 3 data to shore points.
Finds nearest WW3 points to each shore point and calculates a weighted average of values with distance weights.
Parameters: - base_shore_point_vector_path (string) – path to point vector
- wwiii_vector_path (string) – path to point shapefile representing the Wave Watch III data.
- target_shore_point_vector_path (string) – path to point vector file with interpolated wwiii data.
Returns: None
-
natcap.invest.coastal_vulnerability.
polygon_to_lines
(geometry)¶ Return a list of shapely lines given higher order shapely geometry.
-
natcap.invest.coastal_vulnerability.
prepare_landmass_line_index
(landmass_vector_path, target_polygon_pickle_path, target_lines_pickle_path, target_rtree_path)¶ Prepare landmass polygon geometry for line operations.
Converts landmass polygons to lines used in shore point interpolation and saves spatial index of lines for fast intersections. Also saves unioned geometry polygons for fast point-in-polygon checks.
Parameters: - landmass_vector_path (string) – path to polygon vector
- target_polygon_pickle_path (string) – path to pickle storing shapely polygon geometry.
- target_lines_pickle_path (string) – path to pickle storing list of shapely line geometries
- target_rtree_path (string) – path to rtree file indexing bounds of line geometries.
Returns: None
-
natcap.invest.coastal_vulnerability.
search_for_habitat
(base_shore_point_vector_path, search_radius, habitat_rank, habitat_id, habitat_vector_path, target_habitat_pickle_path)¶ Search for habitat polygon within a radius of each shore point.
If habitat is present within the search radius, assign the habitat_rank to the shore point FID. If habitat is not present, assign rank of 5.
Parameters: - base_shore_point_vector_path (string) – path to a shore point vector.
- search_radius (integer) – distance around each point to search for habitat. units match units from base_shore_point_vector SRS.
- habitat_rank (integer) – from 1 to 5 representing the relative protection offered by this habitat (5 = no protection).
- habitat_id (string) – unique string to represent each habitat.
- habitat_vector_path (string) – path to a polygon vector.
- target_habitat_pickle_path (string) –
path to pickle file storing a dict keyed by shore point FID, nested in a dict keyed by habitat_id:
{‘habitat_id’: {<fid0>: 5, <fid1>.: 5, <fid2>: 5}}
Returns: None
Final Ecosystem Services¶
Coastal Blue Carbon Package¶
Model Entry Point¶
Coastal Blue Carbon¶
Coastal Blue Carbon Model.
-
natcap.invest.coastal_blue_carbon.coastal_blue_carbon.
execute
(args)¶ Coastal Blue Carbon.
Parameters: - workspace_dir (str) – location into which all intermediate and output files should be placed.
- results_suffix (str) – a string to append to output filenames.
- lulc_lookup_uri (str) – filepath to a CSV table used to convert the lulc code to a name. Also used to determine if a given lulc type is a coastal blue carbon habitat.
- lulc_transition_matrix_uri (str) – generated by the preprocessor. This file must be edited before it can be used by the main model. The left-most column represents the source lulc class, and the top row represents the destination lulc class.
- carbon_pool_initial_uri (str) – the provided CSV table contains information related to the initial conditions of the carbon stock within each of the three pools of a habitat. Biomass includes carbon stored above and below ground. All non-coastal blue carbon habitat lulc classes are assumed to contain no carbon. The values for ‘biomass’, ‘soil’, and ‘litter’ should be given in terms of Megatonnes CO2 e/ ha.
- carbon_pool_transient_uri (str) – the provided CSV table contains information related to the transition of carbon into and out of coastal blue carbon pools. All non-coastal blue carbon habitat lulc classes are assumed to neither sequester nor emit carbon as a result of change. The ‘yearly_accumulation’ values should be given in terms of Megatonnes of CO2 e/ha-yr. The ‘half-life’ values must be given in terms of years. The ‘disturbance’ values must be given as a decimal (e.g. 0.5 for 50%) of stock distrubed given a transition occurs away from a lulc-class.
- lulc_baseline_map_uri (str) – a GDAL-supported raster representing the baseline landscape/seascape.
- lulc_baseline_year (int) – The year of the baseline snapshot.
- lulc_transition_maps_list (list) – a list of GDAL-supported rasters representing the landscape/seascape at particular points in time. Provided in chronological order.
- lulc_transition_years_list (list) – a list of years that respectively correspond to transition years of the rasters. Provided in chronological order.
- analysis_year (int) – optional. Indicates how many timesteps to run the transient analysis beyond the last transition year. Must come chronologically after the last transition year if provided. Otherwise, the final timestep of the model will be set to the last transition year.
- do_economic_analysis (bool) – boolean value indicating whether model should run economic analysis.
- do_price_table (bool) – boolean value indicating whether a price table is included in the arguments and to be used or a price and interest rate is provided and to be used instead.
- price (float) – the price per Megatonne CO2 e at the base year.
- inflation_rate (float) – the interest rate on the price per Megatonne CO2e, compounded yearly. Provided as a percentage (e.g. 3.0 for 3%).
- price_table_uri (bool) – if args[‘do_price_table’] is set to True the provided CSV table is used in place of the initial price and interest rate inputs. The table contains the price per Megatonne CO2e sequestered for a given year, for all years from the original snapshot to the analysis year, if provided.
- discount_rate (float) – the discount rate on future valuations of sequestered carbon, compounded yearly. Provided as a percentage (e.g. 3.0 for 3%).
Example Args:
args = { 'workspace_dir': 'path/to/workspace/', 'results_suffix': '', 'lulc_lookup_uri': 'path/to/lulc_lookup_uri', 'lulc_transition_matrix_uri': 'path/to/lulc_transition_uri', 'carbon_pool_initial_uri': 'path/to/carbon_pool_initial_uri', 'carbon_pool_transient_uri': 'path/to/carbon_pool_transient_uri', 'lulc_baseline_map_uri': 'path/to/baseline_map.tif', 'lulc_baseline_year': <int>, 'lulc_transition_maps_list': [raster1_uri, raster2_uri, ...], 'lulc_transition_years_list': [2000, 2005, ...], 'analysis_year': 2100, 'do_economic_analysis': '<boolean>', 'do_price_table': '<boolean>', 'price': '<float>', 'inflation_rate': '<float>', 'price_table_uri': 'path/to/price_table', 'discount_rate': '<float>' }
-
natcap.invest.coastal_blue_carbon.coastal_blue_carbon.
get_inputs
(args)¶ Get Inputs.
Parameters: - workspace_dir (str) – workspace directory
- results_suffix (str) – optional suffix appended to results
- lulc_lookup_uri (str) – lulc lookup table filepath
- lulc_transition_matrix_uri (str) – lulc transition table filepath
- carbon_pool_initial_uri (str) – initial conditions table filepath
- carbon_pool_transient_uri (str) – transient conditions table filepath
- lulc_baseline_map_uri (str) – baseline map filepath
- lulc_transition_maps_list (list) – ordered list of transition map filepaths
- lulc_transition_years_list (list) – ordered list of transition years
- analysis_year (int) – optional final year to extend the analysis beyond the last transition year
- do_economic_analysis (bool) – whether to run economic component of the analysis
- do_price_table (bool) – whether to use the price table for the economic component of the analysis
- price (float) – the price of net sequestered carbon
- inflation_rate (float) – the interest rate on the price of carbon
- price_table_uri (str) – price table filepath
- discount_rate (float) – the discount rate on future valuations of carbon
Returns: data dictionary.
Return type: d (dict)
- Example Returns:
- d = {
- ‘do_economic_analysis’: <bool>, ‘lulc_to_Sb’: <dict>, ‘lulc_to_Ss’: <dict> ‘lulc_to_L’: <dict>, ‘lulc_to_Yb’: <dict>, ‘lulc_to_Ys’: <dict>, ‘lulc_to_Hb’: <dict>, ‘lulc_to_Hs’: <dict>, ‘lulc_trans_to_Db’: <dict>, ‘lulc_trans_to_Ds’: <dict>, ‘C_r_rasters’: <list>, ‘transition_years’: <list>, ‘snapshot_years’: <list>, ‘timesteps’: <int>, ‘transitions’: <list>, ‘price_t’: <list>, ‘File_Registry’: <dict>
}
-
natcap.invest.coastal_blue_carbon.coastal_blue_carbon.
get_num_blocks
(raster_uri)¶ Get the number of blocks in a raster file.
Parameters: raster_uri (str) – filepath to raster Returns: number of blocks in raster Return type: num_blocks (int)
-
natcap.invest.coastal_blue_carbon.coastal_blue_carbon.
is_transition_year
(snapshot_years, transitions, timestep)¶ Check whether given timestep is a transition year.
Parameters: - snapshot_years (list) – list of snapshot years.
- transitions (int) – number of transitions.
- timestep (int) – current timestep.
Returns: - whether the year corresponding to the
timestep is a transition year.
Return type: is_transition_year (bool)
-
natcap.invest.coastal_blue_carbon.coastal_blue_carbon.
read_from_raster
(input_raster, offset_block)¶ Read numpy array from raster block.
Parameters: - input_raster (str) – filepath to input raster
- offset_block (dict) – dictionary of offset information
Returns: a blocked array of the input raster
Return type: array (numpy.array)
-
natcap.invest.coastal_blue_carbon.coastal_blue_carbon.
reclass
(array, d, out_dtype=None, nodata_mask=None)¶ Reclassify values in array.
If a nodata value is not provided, the function will return an array with NaN values in its place to mark cells that could not be reclassed.
Parameters: - array (numpy.array) – input data
- d (dict) – reclassification map
- out_dtype (numpy.dtype) – a numpy datatype for the reclass_array
- nodata_mask (number) – for floats, a nodata value that is set to numpy.nan if provided to make reclass_array nodata values consistent
Returns: reclassified array
Return type: reclass_array (numpy.array)
-
natcap.invest.coastal_blue_carbon.coastal_blue_carbon.
reclass_transition
(a_prev, a_next, trans_dict, out_dtype=None, nodata_mask=None)¶ Reclass arrays based on element-wise combinations between two arrays.
Parameters: - a_prev (numpy.array) – previous lulc array
- a_next (numpy.array) – next lulc array
- trans_dict (dict) – reclassification map
- out_dtype (numpy.dtype) – a numpy datatype for the reclass_array
- nodata_mask (number) – for floats, a nodata value that is set to numpy.nan if provided to make reclass_array nodata values consistent
Returns: reclassified array
Return type: reclass_array (numpy.array)
-
natcap.invest.coastal_blue_carbon.coastal_blue_carbon.
s_to_timestep
(snapshot_years, snapshot_idx)¶ Convert snapshot index position to timestep.
Parameters: - snapshot_years (list) – list of snapshot years.
- snapshot_idx (int) – index of snapshot
Returns: timestep of the snapshot
Return type: snapshot_timestep (int)
-
natcap.invest.coastal_blue_carbon.coastal_blue_carbon.
timestep_to_transition_idx
(snapshot_years, transitions, timestep)¶ Convert timestep to transition index.
Parameters: - snapshot_years (list) – a list of years corresponding to the provided rasters
- transitions (int) – the number of transitions in the scenario
- timestep (int) – the current timestep
Returns: the current transition
Return type: transition_idx (int)
-
natcap.invest.coastal_blue_carbon.coastal_blue_carbon.
write_to_raster
(output_raster, array, xoff, yoff)¶ Write numpy array to raster block.
Parameters: - output_raster (str) – filepath to output raster
- array (numpy.array) – block to save to raster
- xoff (int) – offset index for x-dimension
- yoff (int) – offset index for y-dimension
Preprocessor¶
Coastal Blue Carbon Preprocessor.
-
natcap.invest.coastal_blue_carbon.preprocessor.
execute
(args)¶ Coastal Blue Carbon Preprocessor.
The preprocessor accepts a list of rasters and checks for cell-transitions across the rasters. The preprocessor outputs a CSV file representing a matrix of land cover transitions, each cell prefilled with a string indicating whether carbon accumulates or is disturbed as a result of the transition, if a transition occurs.
Parameters: - workspace_dir (string) – directory path to workspace
- results_suffix (string) – append to outputs directory name if provided
- lulc_lookup_uri (string) – filepath of lulc lookup table
- lulc_snapshot_list (list) – a list of filepaths to lulc rasters
Example Args:
args = { 'workspace_dir': 'path/to/workspace_dir/', 'results_suffix': '', 'lulc_lookup_uri': 'path/to/lookup.csv', 'lulc_snapshot_list': ['path/to/raster1', 'path/to/raster2', ...] }
-
natcap.invest.coastal_blue_carbon.preprocessor.
read_from_raster
(input_raster, offset_block)¶ Read block from raster.
Parameters: - input_raster (str) – filepath to raster.
- offset_block (dict) – where the block is indexed.
Returns: the raster block.
Return type: a (np.array)
Module contents¶
Coastal Blue Carbon package.
Carbon Package¶
Model Entry Point¶
Carbon Combined¶
Carbon Biophysical¶
Carbon Valuation¶
Carbon Utilities¶
Module contents¶
Carbon Storage and Sequestration.
-
natcap.invest.carbon.
execute
(args)¶ InVEST Carbon Model.
Calculate the amount of carbon stocks given a landscape, or the difference due to a future change, and/or the tradeoffs between that and a REDD scenario, and calculate economic valuation on those scenarios.
The model can operate on a single scenario, a combined present and future scenario, as well as an additional REDD scenario.
Parameters: - args['workspace_dir'] (string) – a path to the directory that will write output and other temporary files during calculation.
- args['results_suffix'] (string) – appended to any output file name.
- args['lulc_cur_path'] (string) – a path to a raster representing the current carbon stocks.
- args['calc_sequestration'] (bool) – if true, sequestration should be calculated and ‘lulc_fut_path’ and ‘do_redd’ should be defined.
- args['lulc_fut_path'] (string) – a path to a raster representing future landcover scenario. Optional, but if present and well defined will trigger a sequestration calculation.
- args['do_redd'] (bool) – if true, REDD analysis should be calculated and ‘lulc_redd_path’ should be defined
- args['lulc_redd_path'] (string) – a path to a raster representing the alternative REDD scenario which is only possible if the args[‘lulc_fut_path’] is present and well defined.
- args['carbon_pools_path'] (string) – path to CSV or that indexes carbon storage density to lulc codes. (required if ‘do_uncertainty’ is false)
- args['lulc_cur_year'] (int/string) – an integer representing the year of args[‘lulc_cur_path’] used if args[‘calc_sequestration’] is True.
- args['lulc_fut_year'] (int/string) – an integer representing the year of args[‘lulc_fut_path’] used in valuation if it exists. Required if args[‘do_valuation’] is True and args[‘lulc_fut_path’] is present and well defined.
- args['do_valuation'] (bool) – if true then run the valuation model on available outputs. At a minimum will run on carbon stocks, if sequestration with a future scenario is done and/or a REDD scenario calculate NPV for either and report in final HTML document.
- args['price_per_metric_ton_of_c'] (float) – Is the present value of carbon per metric ton. Used if args[‘do_valuation’] is present and True.
- args['discount_rate'] (float) – Discount rate used if NPV calculations are required. Used if args[‘do_valuation’] is present and True.
- args['rate_change'] (float) – Annual rate of change in price of carbon as a percentage. Used if args[‘do_valuation’] is present and True.
- args['n_workers'] (int) – (optional) The number of worker processes to use for processing this model. If omitted, computation will take place in the current process.
Returns: None.
Crop Production Package¶
Table of Contents¶
Model Entry Point¶
Crop Production IO Module¶
Crop Production Model Module¶
Module contents¶
Finfish Aquaculture Package¶
Model Entry Point¶
-
natcap.invest.finfish_aquaculture.finfish_aquaculture.
execute
(args)¶ Finfish Aquaculture.
This function will take care of preparing files passed into the finfish aquaculture model. It will handle all files/inputs associated with biophysical and valuation calculations and manipulations. It will create objects to be passed to the aquaculture_core.py module. It may write log, warning, or error messages to stdout.
Parameters: - workspace_dir (string) – The directory in which to place all result files.
- ff_farm_loc (string) – URI that points to a shape file of fishery locations
- farm_ID (string) – column heading used to describe individual farms. Used to link GIS location data to later inputs.
- g_param_a (float) – Growth parameter alpha, used in modeling fish growth, should be an int or float.
- g_param_b (float) – Growth parameter beta, used in modeling fish growth, should be an int or float.
- g_param_tau (float) – Growth parameter tau, used in modeling fish growth, should be an int or float
- use_uncertainty (boolean) –
- g_param_a_sd (float) – (description)
- g_param_b_sd (float) – (description)
- num_monte_carlo_runs (int) –
- water_temp_tbl (string) – URI to a CSV table where daily water temperature values are stored from one year
- farm_op_tbl (string) – URI to CSV table of static variables for calculations
- outplant_buffer (int) – This value will allow the outplanting start day to be flexible plus or minus the number of days specified here.
- do_valuation (boolean) – Boolean that indicates whether or not valuation should be performed on the aquaculture model
- p_per_kg (float) – Market price per kilogram of processed fish
- frac_p (float) – Fraction of market price that accounts for costs rather than profit
- discount (float) – Daily market discount rate
Example Args Dictionary:
{ 'workspace_dir': 'path/to/workspace_dir', 'ff_farm_loc': 'path/to/shapefile', 'farm_ID': 'FarmID' 'g_param_a': 0.038, 'g_param_b': 0.6667, 'g_param_tau': 0.08, 'use_uncertainty': True, 'g_param_a_sd': 0.005, 'g_param_b_sd': 0.05, 'num_monte_carlo_runs': 1000, 'water_temp_tbl': 'path/to/water_temp_tbl', 'farm_op_tbl': 'path/to/farm_op_tbl', 'outplant_buffer': 3, 'do_valuation': True, 'p_per_kg': 2.25, 'frac_p': 0.3, 'discount': 0.000192, }
Finfish Aquaculture¶
inVEST finfish aquaculture filehandler for biophysical and valuation data
-
natcap.invest.finfish_aquaculture.finfish_aquaculture.
execute
(args) Finfish Aquaculture.
This function will take care of preparing files passed into the finfish aquaculture model. It will handle all files/inputs associated with biophysical and valuation calculations and manipulations. It will create objects to be passed to the aquaculture_core.py module. It may write log, warning, or error messages to stdout.
Parameters: - workspace_dir (string) – The directory in which to place all result files.
- ff_farm_loc (string) – URI that points to a shape file of fishery locations
- farm_ID (string) – column heading used to describe individual farms. Used to link GIS location data to later inputs.
- g_param_a (float) – Growth parameter alpha, used in modeling fish growth, should be an int or float.
- g_param_b (float) – Growth parameter beta, used in modeling fish growth, should be an int or float.
- g_param_tau (float) – Growth parameter tau, used in modeling fish growth, should be an int or float
- use_uncertainty (boolean) –
- g_param_a_sd (float) – (description)
- g_param_b_sd (float) – (description)
- num_monte_carlo_runs (int) –
- water_temp_tbl (string) – URI to a CSV table where daily water temperature values are stored from one year
- farm_op_tbl (string) – URI to CSV table of static variables for calculations
- outplant_buffer (int) – This value will allow the outplanting start day to be flexible plus or minus the number of days specified here.
- do_valuation (boolean) – Boolean that indicates whether or not valuation should be performed on the aquaculture model
- p_per_kg (float) – Market price per kilogram of processed fish
- frac_p (float) – Fraction of market price that accounts for costs rather than profit
- discount (float) – Daily market discount rate
Example Args Dictionary:
{ 'workspace_dir': 'path/to/workspace_dir', 'ff_farm_loc': 'path/to/shapefile', 'farm_ID': 'FarmID' 'g_param_a': 0.038, 'g_param_b': 0.6667, 'g_param_tau': 0.08, 'use_uncertainty': True, 'g_param_a_sd': 0.005, 'g_param_b_sd': 0.05, 'num_monte_carlo_runs': 1000, 'water_temp_tbl': 'path/to/water_temp_tbl', 'farm_op_tbl': 'path/to/farm_op_tbl', 'outplant_buffer': 3, 'do_valuation': True, 'p_per_kg': 2.25, 'frac_p': 0.3, 'discount': 0.000192, }
-
natcap.invest.finfish_aquaculture.finfish_aquaculture.
format_ops_table
(op_path, farm_ID, ff_aqua_args)¶ Takes in the path to the operating parameters table as well as the keyword to look for to identify the farm number to go with the parameters, and outputs a 2D dictionary that contains all parameters by farm and description. The outer key is farm number, and the inner key is a string description of the parameter.
- Input:
op_path: URI to CSV table of static variables for calculations farm_ID: The string to look for in order to identify the column in
which the farm numbers are stored. That column data will become the keys for the dictionary output.- ff_aqua_args: Dictionary of arguments being created in order to be
- passed to the aquaculture core function.
- Output:
- ff_aqua_args[‘farm_op_dict’]: A dictionary that is built up to store
- the static parameters for the aquaculture model run. This is a 2D dictionary, where the outer key is the farm ID number, and the inner keys are strings of parameter names.
Returns nothing.
-
natcap.invest.finfish_aquaculture.finfish_aquaculture.
format_temp_table
(temp_path, ff_aqua_args)¶ This function is doing much the same thing as format_ops_table- it takes in information from a temperature table, and is formatting it into a 2D dictionary as an output.
- Input:
- temp_path: URI to a CSV file containing temperature data for 365 days
- for the farms on which we will look at growth cycles.
- ff_aqua_args: Dictionary of arguments that we are building up in order
- to pass it to the aquaculture core module.
- Output:
- ff_aqua_args[‘water_temp_dict’]: A 2D dictionary containing temperature
- data for 365 days. The outer keys are days of the year from 0 to 364 (we need to be able to check the day modulo 365) which we manually shift down by 1, and the inner keys are farm ID numbers.
Returns nothing.
Finfish Aquaculture Core¶
Implementation of the aquaculture calculations, and subsequent outputs. This will pull from data passed in by finfish_aquaculture.
-
natcap.invest.finfish_aquaculture.finfish_aquaculture_core.
calc_farm_cycles
(outplant_buffer, a, b, tau, water_temp_dict, farm_op_dict, dur)¶ - Input:
- outplant_buffer: The number of days surrounding the outplant day during
- which the fish growth cycle can still be started.
- a: Growth parameter alpha. Float used as a scaler in the fish growth
- equation.
- b: Growth paramater beta. Float used as an exponential multiplier in
- the fish growth equation.
- water_temp_dict: 2D dictionary which contains temperature values for
- farms. The outer keys are calendar days as strings, and the inner are farm numbers as strings.
- farm_op_dict: 2D dictionary which contains individual operating
- parameters for each farm. The outer key is farm number as a string, and the inner is string descriptors of each parameter.
- dur: Float which describes the length for the growth simulation to run
- in years.
Returns cycle_history where:
- cycle_history: Dictionary which contains mappings from farms to a
history of growth for each cycle completed on that farm. These entries are formatted as follows…
- Farm->List of Type (day of outplanting,day of harvest, fish weight
- (grams))
-
natcap.invest.finfish_aquaculture.finfish_aquaculture_core.
calc_hrv_weight
(farm_op_dict, frac, mort, cycle_history)¶ - Input:
- farm_op_dict: 2D dictionary which contains individual operating
- parameters for each farm. The outer key is farm number as a string, and the inner is string descriptors of each parameter.
- frac: A float representing the fraction of the fish that remains after
- processing.
- mort: A float referring to the daily mortality rate of fishes on an
- aquaculture farm.
- cycle_history: Farm->List of Type (day of outplanting,
- day of harvest, fish weight (grams))
- Returns a tuple (curr_cycle_totals,indiv_tpw_totals) where:
- curr_cycle_totals_: dictionary which will hold a mapping from every
- farm (as identified by farm_ID) to the total processed weight of each farm
- indiv_tpw_totals: dictionary which will hold a farm->list mapping,
- where the list holds the individual tpw for all cycles that the farm completed
-
natcap.invest.finfish_aquaculture.finfish_aquaculture_core.
compute_uncertainty_data
(args, output_dir)¶ Does uncertainty analysis via a Monte Carlo simulation.
Returns a tuple with two 2D dicts. -a dict containing relative file paths to produced histograms -a dict containining statistical results (mean and std deviation) Each dict has farm IDs as outer keys, and result types (e.g. ‘value’, ‘weight’, and ‘cycles’) as inner keys.
-
natcap.invest.finfish_aquaculture.finfish_aquaculture_core.
create_HTML_table
(output_dir, args, cycle_history, sum_hrv_weight, hrv_weight, farms_npv, value_history, histogram_paths, uncertainty_stats)¶ - Inputs:
- output_dir: The directory in which we will be creating our .html file
- output.
- cycle_history: dictionary mapping farm ID->list of tuples, each of
- which contains 3 things- (day of outplanting, day of harvest,
- harvest weight of a single fish in grams)
- sum_hrv_weight: dictionary which holds a mapping from farm ID->total
- processed weight of each farm
- hrv_weight: dictionary which holds a farm->list mapping, where the list
- holds the individual tpw for all cycles that the farm completed
- do_valuation: boolean variable that says whether or not valuation is
- desired
- farms_npv: dictionary with a farm-> float mapping, where each float is
- the net processed value of the fish processed on that farm, in $1000s of dollars.
- value_history: dictionary which holds a farm->list mapping, where the
- list holds tuples containing (Net Revenue, Net Present Value) for each cycle completed by that farm
- Output:
- HTML file: contains 3 tables that summarize inputs and outputs for the
duration of the model. - Input Table: Farm Operations provided data, including Farm ID #,
Cycle Number, weight of fish at start, weight of fish at harvest, number of fish in farm, start day for growing, and length of fallowing period- Output Table 1: Farm Harvesting data, including a summary table
- for each harvest cycle of each farm. Will show Farm ID, cycle number, days since outplanting date, harvested weight, net revenue, outplant day, and year.
- Output Table 2: Model outputs for each farm, including Farm ID,
- net present value, number of completed harvest cycles, and total volume harvested.
Returns nothing.
-
natcap.invest.finfish_aquaculture.finfish_aquaculture_core.
do_monte_carlo_simulation
(args)¶ Performs a Monte Carlo simulation and returns the results.
-
natcap.invest.finfish_aquaculture.finfish_aquaculture_core.
execute
(args)¶ ‘ Runs the biophysical and valuation parts of the finfish aquaculture model. This will output: 1. a shape file showing farm locations w/ addition of # of harvest cycles,
total processed weight at that farm, and if valuation is true, total discounted net revenue at each farm location.- Three HTML tables summarizing all model I/O- summary of user-provided
- data, summary of each harvest cycle, and summary of the outputs/farm
- A .txt file that is named according to the date and time the model is
- run, which lists the values used during that run
Data in args should include the following: –Biophysical Arguments– args: a python dictionary containing the following data: args[‘workspace_dir’]- The directory in which to place all result files. args[‘ff_farm_file’]- An open shape file containing the locations of
individual fisheries- args[‘farm_ID’]- column heading used to describe individual farms. Used to
- link GIS location data to later inputs.
- args[‘g_param_a’]- Growth parameter alpha, used in modeling fish growth,
- should be int or a float.
- args[‘g_param_b’]- Growth parameter beta, used in modeling fish growth,
- should be int or a float.
- args[‘water_temp_dict’]- A dictionary which links a specific date to the
farm numbers, and their temperature values on that day. (Note: in this case, the outer keys 1 and 2 are calendar days out of 365, starting with January 1 (day 0), and the inner 1, 2, and 3 are farm numbers.)
- Format: {‘0’: ‘{‘1’: ‘8.447, ‘2’: ‘8.447’, ‘3’:‘8.947’, …}’ ,
- ‘1’: ‘{‘1’: ‘8.406, ‘2’: ‘8.406’, ‘3’:‘8.906’, …}’ ,
. . . . . . . . . }
- args[‘farm_op_dict’]- Dictionary which links a specific farm ID # to
another dictionary containing operating parameters mapped to their value for that particular farm (Note: in this case, the 1 and 2 are farm ID’s, not dates out of 365.)
- Format: {‘1’: ‘{‘Wt of Fish’: ‘0.06’, ‘Tar Weight’: ‘5.4’, …}’,
- ‘2’: ‘{‘Wt of Fish’: ‘0.06’, ‘Tar Weight’: ‘5.4’, …}’, . . . . . . . . . }
- args[‘frac_post_process’]- the fraction of edible fish left after
- processing is done to remove undesirable parts
- args[‘mort_rate_daily’]- mortality rate among fish in a year, divided by
- 365
args[‘duration’]- duration of the simulation, in years args[‘outplant_buffer’] - This value will allow the outplant start day to
be flexible plus or minus the number of days specified here.–Valuation arguments– args[‘do_valuation’]- boolean indicating whether or not to run the
valuation processargs[‘p_per_kg’]: Market price per kilogram of processed fish args[‘frac_p’]: Fraction of market price that accounts for costs rather
than profitargs[‘discount’]: Daily market discount rate
returns nothing
-
natcap.invest.finfish_aquaculture.finfish_aquaculture_core.
make_histograms
(farm, results, output_dir, total_num_runs)¶ Makes a histogram for the given farm and data.
Returns a dict mapping type (e.g. ‘value’, ‘weight’) to the relative file path for the respective histogram.
-
natcap.invest.finfish_aquaculture.finfish_aquaculture_core.
valuation
(price_per_kg, frac_mrkt_price, discount, hrv_weight, cycle_history)¶ This performs the valuation calculations, and returns tuple containing a dictionary with a farm-> float mapping, where each float is the net processed value of the fish processed on that farm, in $1000s of dollars, and a dictionary containing a farm-> list mapping, where each entry in the list is a tuple of (Net Revenue, Net Present Value) for every cycle on that farm.
- Inputs:
- price_per_kg: Float representing the price per kilogram of finfish for
- valuation purposes.
- frac_mrkt_price: Float that represents the fraction of market price
- that is attributable to costs.
discount: Float that is the daily market discount rate. cycle_hisory: Farm->List of Type (day of outplanting,
day of harvest, fish weight (grams))hrv_weight: Farm->List of TPW for each cycle (kilograms)
- Returns a tuple (val_history, valuations):
- val_history: dictionary which will hold a farm->list mapping, where the
- list holds tuples containing (Net Revenue, Net Present Value) for each cycle completed by that farm
- valuations: dictionary with a farm-> float mapping, where each float is
- the net processed value of the fish processed on that farm
Module contents¶
Fisheries Package¶
Table of Contents¶
Fisheries Model Entry Point¶
-
natcap.invest.fisheries.fisheries.
execute
(args, create_outputs=True)¶ Fisheries.
- args[‘workspace_dir’] (str): location into which all intermediate
- and output files should be placed.
args[‘results_suffix’] (str): a string to append to output filenames
- args[‘aoi_vector_path’] (str): location of shapefile which will be
- used as subregions for calculation. Each region must contain a ‘Name’ attribute (case-sensitive) matching the given name in the population parameters csv file.
- args[‘timesteps’] (int): represents the number of time steps that
- the user desires the model to run.
- args[‘population_type’] (str): specifies whether the model
- is age-specific or stage-specific. Options will be either “Age Specific” or “Stage Specific” and will change which equation is used in modeling growth.
- args[‘sexsp’] (str): specifies whether or not the age and stage
- classes are distinguished by sex.
- args[‘harvest_units’] (str): specifies how the user wants to get
- the harvest data. Options are either “Individuals” or “Weight”, and will change the harvest equation used in core. (Required if args[‘val_cont’] is True)
- args[‘do_batch’] (bool): specifies whether program will perform a
- single model run or a batch (set) of model runs.
- args[‘population_csv_path’] (str): location of the population
- parameters csv. This will contain all age and stage specific parameters. (Required if args[‘do_batch’] is False)
- args[‘population_csv_dir’] (str): location of the directory that
- contains the Population Parameters CSV files for batch processing (Required if args[‘do_batch’] is True)
- args[‘spawn_units’] (str): specifies whether the spawner abundance used in
- the recruitment function should be calculated in terms of number of individuals (‘Individuals’) or in terms of biomass (‘Weight’). If ‘Weight’ is selected, the user must provide a ‘Weight’ vector alongside the survival matrix in the Population Parameters CSV File. The ‘alpha’ and ‘beta’ parameters provided by the user should correspond to the selected choice.
- args[‘total_init_recruits’] (float): represents the initial number of
- recruits that will be used in calculation of population on a per area basis.
- args[‘recruitment_type’] (str): name corresponding to one of the built-in
- recruitment functions {‘Beverton-Holt’, ‘Ricker’, ‘Fecundity’, Fixed}, or ‘Other’, meaning that the user is passing in their own recruitment function as an anonymous python function via the optional dictionary argument ‘recruitment_func’.
- args[‘recruitment_func’] (function): Required if args[‘recruitment_type’]
- is set to ‘Other’. See below for instructions on how to create a user- defined recruitment function.
- args[‘alpha’] (float): must exist within args for BH or Ricker Recruitment.
- Parameter that will be used in calculation of recruitment.
- args[‘beta’] (float): must exist within args for BH or Ricker Recruitment.
- Parameter that will be used in calculation of recruitment.
- args[‘total_recur_recruits’] (float): must exist within args for Fixed
- Recruitment. Parameter that will be used in calculation of recruitment.
args[‘migr_cont’] (bool): if True, model uses migration.
- args[‘migration_dir’] (str): if this parameter exists, it means migration
- is desired. This is the location of the parameters folder containing files for migration. There should be one file for every age class which migrates. (Required if args[‘migr_cont’] is True)
args[‘val_cont’] (bool): if True, model computes valuation.
- args[‘frac_post_process’] (float): represents the fraction of the species
- remaining after processing of the whole carcass is complete. This will exist only if valuation is desired for the particular species. (Required if args[‘val_cont’] is True)
- args[‘unit_price’] (float): represents the price for a single unit of
- harvest. Exists only if valuation is desired. (Required if args[‘val_cont’] is True)
Example Args:
args = { 'workspace_dir': 'path/to/workspace_dir/', 'results_suffix': 'scenario_name', 'aoi_vector_path': 'path/to/aoi_vector_path', 'total_timesteps': 100, 'population_type': 'Stage-Based', 'sexsp': 'Yes', 'harvest_units': 'Individuals', 'do_batch': False, 'population_csv_path': 'path/to/csv_path', 'population_csv_dir': '', 'spawn_units': 'Weight', 'total_init_recruits': 100000.0, 'recruitment_type': 'Ricker', 'alpha': 32.4, 'beta': 54.2, 'total_recur_recruits': 92.1, 'migr_cont': True, 'migration_dir': 'path/to/mig_dir/', 'val_cont': True, 'frac_post_process': 0.5, 'unit_price': 5.0, }
Creating a User-Defined Recruitment Function
An optional argument has been created in the Fisheries Model to allow users proficient in Python to pass their own recruitment function into the program via the args dictionary.
Using the Beverton-Holt recruitment function as an example, here’s how a user might create and pass in their own recruitment function:
import natcap.invest import numpy as np # define input data Matu = np.array([...]) # the Maturity vector in the Population Parameters File Weight = np.array([...]) # the Weight vector in the Population Parameters File LarvDisp = np.array([...]) # the LarvalDispersal vector in the Population Parameters File alpha = 2.0 # scalar value beta = 10.0 # scalar value sexsp = 2 # 1 = not sex-specific, 2 = sex-specific # create recruitment function def spawners(N_prev): return (N_prev * Matu * Weight).sum() def rec_func_BH(N_prev): N_0 = (LarvDisp * ((alpha * spawners( N_prev) / (beta + spawners(N_prev)))) / sexsp) return (N_0, spawners(N_prev)) # fill out args dictionary args = {} # ... define other arguments ... args['recruitment_type'] = 'Other' # lets program know to use user- defined function args['recruitment_func'] = rec_func_BH # pass recruitment function as 'anonymous' Python function # run model natcap.invest.fisheries.fisheries.execute(args)
Conditions that a new recruitment function must meet to run properly:
- The function must accept as an argument: a single numpy three-
- dimensional array (N_prev) representing the state of the population at the previous time step. N_prev has three dimensions: the indices of the first dimension correspond to the region (must be in same order as provided in the Population Parameters File), the indices of the second dimension represent the sex if it is specific (i.e. two indices representing female, then male if the model is ‘sex-specific’, else just a single zero index representing the female and male populations aggregated together), and the indicies of the third dimension represent age/stage in ascending order.
- The function must return: a tuple of two values. The first value
- (N_0) being a single numpy one-dimensional array representing the youngest age of the population for the next time step. The indices of the array correspond to the regions of the population (outputted in same order as provided). If the model is sex-specific, it is currently assumed that males and females are produced in equal number and that the returned array has been already been divided by 2 in the recruitment function. The second value (spawners) is the number or weight of the spawners created by the population from the previous time step, provided as a scalar float value (non-negative).
Example of How Recruitment Function Operates within Fisheries Model:
# input data N_prev_xsa = [[[region0-female-age0, region0-female-age1], [region0-male-age0, region1-male-age1]], [[region1-female-age0, region1-female-age1], [region1-male-age0], [region1-male-age1]]] # execute function N_0_x, spawners = rec_func(N_prev_xsa) # output data - where N_0 contains information about the youngest # age/stage of the population for the next time step: N_0_x = [region0-age0, region1-age0] # if sex-specific, rec_func should divide by two before returning type(spawners) is float
Fisheries IO Module¶
The Fisheries IO module contains functions for handling inputs and outputs
-
exception
natcap.invest.fisheries.fisheries_io.
MissingParameter
¶ Bases:
exceptions.ValueError
An exception class that may be raised when a necessary parameter is not provided by the user.
-
natcap.invest.fisheries.fisheries_io.
create_outputs
(vars_dict)¶ Creates outputs from variables generated in the run_population_model() function in the fisheries_model module
Creates the following:
- Results CSV File
- Results HTML Page
- Results Shapefile (if provided)
- Intermediate CSV File
Parameters: vars_dict (dictionary) – contains variables generated by model run
-
natcap.invest.fisheries.fisheries_io.
fetch_args
(args, create_outputs=True)¶ Fetches input arguments from the user, verifies for correctness and completeness, and returns a list of variables dictionaries
Parameters: args (dictionary) – arguments from the user Returns: - set of variable dictionaries for each
- model
Return type: model_list (list) Example Returns:
model_list = [ { 'workspace_dir': 'path/to/workspace_dir', 'results_suffix': 'scenario_name', 'output_dir': 'path/to/output_dir', 'aoi_vector_path': 'path/to/aoi_vector_path', 'total_timesteps': 100, 'population_type': 'Stage-Based', 'sexsp': 2, 'harvest_units': 'Individuals', 'do_batch': False, 'spawn_units': 'Weight', 'total_init_recruits': 100.0, 'recruitment_type': 'Ricker', 'alpha': 32.4, 'beta': 54.2, 'total_recur_recruits': 92.1, 'migr_cont': True, 'val_cont': True, 'frac_post_process': 0.5, 'unit_price': 5.0, # Pop Params 'population_csv_path': 'path/to/csv_path', 'Survnaturalfrac': numpy.array( [[[...], [...]], [[...], [...]], ...]), 'Classes': numpy.array([...]), 'Vulnfishing': numpy.array([...], [...]), 'Maturity': numpy.array([...], [...]), 'Duration': numpy.array([...], [...]), 'Weight': numpy.array([...], [...]), 'Fecundity': numpy.array([...], [...]), 'Regions': numpy.array([...]), 'Exploitationfraction': numpy.array([...]), 'Larvaldispersal': numpy.array([...]), # Mig Params 'migration_dir': 'path/to/mig_dir', 'Migration': [numpy.matrix, numpy.matrix, ...] }, { ... # additional dictionary doesn't exist when 'do_batch' # is false } ]
Note
This function receives an unmodified ‘args’ dictionary from the user
-
natcap.invest.fisheries.fisheries_io.
read_migration_tables
(args, class_list, region_list)¶ Parses, verifies and orders list of migration matrices necessary for program.
Parameters: - args (dictionary) – same args as model entry point
- class_list (list) – list of class names
- region_list (list) – list of region names
Returns: see example below
Return type: mig_dict (dictionary)
Example Returns:
mig_dict = { 'Migration': [numpy.matrix, numpy.matrix, ...] }
Note
If migration matrices are not provided for all classes, the function will generate identity matrices for missing classes
-
natcap.invest.fisheries.fisheries_io.
read_population_csv
(args, path)¶ Parses and verifies a single Population Parameters CSV file
Parses and verifies inputs from the Population Parameters CSV file. If not all necessary vectors are included, the function will raise a MissingParameter exception. Survival matrix will be arranged by class-elements, 2nd dim: sex, and 3rd dim: region. Class vectors will be arranged by class-elements, 2nd dim: sex (depending on whether model is sex-specific) Region vectors will be arraged by region-elements, sex-agnostic.
Parameters: - args (dictionary) – arguments provided by user
- path (string) – the particular Population Parameters CSV file to parse and verifiy
Returns: - dictionary containing verified population
arguments
Return type: pop_dict (dictionary)
Example Returns:
pop_dict = { 'population_csv_path': 'path/to/csv', 'Survnaturalfrac': numpy.array( [[...], [...]], [[...], [...]], ...), # Class Vectors 'Classes': numpy.array([...]), 'Vulnfishing': numpy.array([...], [...]), 'Maturity': numpy.array([...], [...]), 'Duration': numpy.array([...], [...]), 'Weight': numpy.array([...], [...]), 'Fecundity': numpy.array([...], [...]), # Region Vectors 'Regions': numpy.array([...]), 'Exploitationfraction': numpy.array([...]), 'Larvaldispersal': numpy.array([...]), }
-
natcap.invest.fisheries.fisheries_io.
read_population_csvs
(args)¶ Parses and verifies the Population Parameters CSV files
Parameters: args (dictionary) – arguments provided by user Returns: - list of dictionaries containing verified population
- arguments
Return type: pop_list (list) Example Returns:
pop_list = [ { 'Survnaturalfrac': numpy.array( [[...], [...]], [[...], [...]], ...), # Class Vectors 'Classes': numpy.array([...]), 'Vulnfishing': numpy.array([...], [...]), 'Maturity': numpy.array([...], [...]), 'Duration': numpy.array([...], [...]), 'Weight': numpy.array([...], [...]), 'Fecundity': numpy.array([...], [...]), # Region Vectors 'Regions': numpy.array([...]), 'Exploitationfraction': numpy.array([...]), 'Larvaldispersal': numpy.array([...]), }, { ... } ]
Fisheries Model Module¶
The Fisheries Model module contains functions for running the model
Variable Suffix Notation: t: time x: area/region a: age/class s: sex
-
natcap.invest.fisheries.fisheries_model.
initialize_vars
(vars_dict)¶ Initializes variables for model run
Parameters: vars_dict (dictionary) – verified arguments and variables Returns: modified vars_dict with additional variables Return type: vars_dict (dictionary) Example Returns:
vars_dict = { # (original vars) 'Survtotalfrac': np.array([...]), # a,s,x 'G_survtotalfrac': np.array([...]), # (same) 'P_survtotalfrac': np.array([...]), # (same) 'N_tasx': np.array([...]), # Index Order: t,a,s,x 'H_tx': np.array([...]), # t,x 'V_tx': np.array([...]), # t,x 'Spawners_t': np.array([...]), }
-
natcap.invest.fisheries.fisheries_model.
run_population_model
(vars_dict, init_cond_func, cycle_func, harvest_func)¶ Runs the model
Parameters: - vars_dict (dictionary) –
- init_cond_func (lambda function) – sets initial conditions
- cycle_func (lambda function) – computes numbers for the next time step
- harvest_func (lambda function) – computes harvest and valuation
Returns: vars_dict (dictionary)
Example Returned Dictionary:
{ # (other items) ... 'N_tasx': np.array([...]), # Index Order: time, class, sex, region 'H_tx': np.array([...]), # Index Order: time, region 'V_tx': np.array([...]), # Index Order: time, region 'Spawners_t': np,array([...]), 'equilibrate_timestep': <int>, }
-
natcap.invest.fisheries.fisheries_model.
set_cycle_func
(vars_dict, rec_func)¶ Creates a function to run a single cycle in the model
Parameters: - vars_dict (dictionary) –
- rec_func (lambda function) – recruitment function
Example Output of Returned Cycle Function:
N_asx = np.array([...]) spawners = <int> N_next, spawners = cycle_func(N_prev)
-
natcap.invest.fisheries.fisheries_model.
set_harvest_func
(vars_dict)¶ Creates harvest function that calculates the given harvest and valuation of the fisheries population over each time step for a given region. Returns None if harvest isn’t selected by user.
Example Outputs of Returned Harvest Function:
H_x, V_x = harv_func(N_tasx) H_x = np.array([3.0, 4.5, 2.5, ...]) V_x = np.array([6.0, 9.0, 5.0, ...])
-
natcap.invest.fisheries.fisheries_model.
set_init_cond_func
(vars_dict)¶ Creates a function to set the initial conditions of the model
Parameters: vars_dict (dictionary) – variables Returns: initial conditions function Return type: init_cond_func (lambda function) Example Return Array:
N_asx = np.ndarray([...])
-
natcap.invest.fisheries.fisheries_model.
set_recru_func
(vars_dict)¶ Creates recruitment function that calculates the number of recruits for class 0 at time t for each region (currently sex agnostic). Also returns number of spawners
Parameters: vars_dict (dictionary) – Returns: recruitment function Return type: rec_func (function) Example Output of Returned Recruitment Function:
N_next[0], spawners = rec_func(N_prev)
Fisheries Habitat Scenario Tool Module¶
The Fisheries Habitat Scenario Tool module contains the high-level code for generating a new Population Parameters CSV File based on habitat area change and the dependencies that particular classes of the given species have on particular habitats.
-
natcap.invest.fisheries.fisheries_hst.
convert_survival_matrix
(vars_dict)¶ Creates a new survival matrix based on the information provided by the user related to habitat area changes and class-level dependencies on those habitats.
Parameters: vars_dict (dictionary) – see fisheries_preprocessor_io.fetch_args for example Returns: - modified vars_dict with new Survival matrix
- accessible using the key ‘Surv_nat_xsa_mod’ with element values that exist between [0,1]
Return type: vars_dict (dictionary) Example Returns:
ret = { # Other Variables... 'Surv_nat_xsa_mod': np.ndarray([...]) }
-
natcap.invest.fisheries.fisheries_hst.
execute
(args)¶ Fisheries: Habitat Scenario Tool.
The Fisheries Habitat Scenario Tool generates a new Population Parameters CSV File with modified survival attributes across classes and regions based on habitat area changes and class-level dependencies on those habitats.
- args[‘workspace_dir’] (str): location into which the resultant
- modified Population Parameters CSV file should be placed.
- args[‘sexsp’] (str): specifies whether or not the age and stage
- classes are distinguished by sex. Options: ‘Yes’ or ‘No’
- args[‘population_csv_path’] (str): location of the population
- parameters csv file. This file contains all age and stage specific parameters.
- args[‘habitat_chg_csv_path’] (str): location of the habitat change
- parameters csv file. This file contains habitat area change information.
- args[‘habitat_dep_csv_path’] (str): location of the habitat dependency
- parameters csv file. This file contains habitat-class dependency information.
- args[‘gamma’] (float): describes the relationship between a change
- in habitat area and a change in survival of life stages dependent on that habitat
Returns: None Example Args:
args = { 'workspace_dir': 'path/to/workspace_dir/', 'sexsp': 'Yes', 'population_csv_path': 'path/to/csv', 'habitat_chg_csv_path': 'path/to/csv', 'habitat_dep_csv_path': 'path/to/csv', 'gamma': 0.5, }
Note
- Modified Population Parameters CSV File saved to ‘workspace_dir/output/’
Fisheries Habitat Scenario Tool IO Module¶
The Fisheries Habitat Scenarios Tool IO module contains functions for handling inputs and outputs
-
natcap.invest.fisheries.fisheries_hst_io.
fetch_args
(args)¶ Fetches input arguments from the user, verifies for correctness and completeness, and returns a list of variables dictionaries
Parameters: args (dictionary) – arguments from the user (same as Fisheries Preprocessor entry point) Returns: dictionary containing necessary variables Return type: vars_dict (dictionary) Raises: ValueError
– parameter mismatch between Population and Habitat CSV filesExample Returns:
vars_dict = { 'workspace_dir': 'path/to/workspace_dir/', 'output_dir': 'path/to/output_dir/', 'sexsp': 2, 'gamma': 0.5, # Pop Vars 'population_csv_path': 'path/to/csv_path', 'Surv_nat_xsa': np.array( [[[...], [...]], [[...], [...]], ...]), 'Classes': np.array([...]), 'Class_vectors': { 'Vulnfishing': np.array([...], [...]), 'Maturity': np.array([...], [...]), 'Duration': np.array([...], [...]), 'Weight': np.array([...], [...]), 'Fecundity': np.array([...], [...]), }, 'Regions': np.array([...]), 'Region_vectors': { 'Exploitationfraction': np.array([...]), 'Larvaldispersal': np.array([...]), }, # Habitat Vars 'habitat_chg_csv_path': 'path/to/csv', 'habitat_dep_csv_path': 'path/to/csv', 'Habitats': ['habitat1', 'habitat2', ...], 'Hab_classes': ['class1', 'class2', ...], 'Hab_regions': ['region1', 'region2', ...], 'Hab_chg_hx': np.array( [[[...], [...]], [[...], [...]], ...]), 'Hab_dep_ha': np.array( [[[...], [...]], [[...], [...]], ...]), 'Hab_class_mvmt_a': np.array([...]), 'Hab_dep_num_a': np.array([...]), }
-
natcap.invest.fisheries.fisheries_hst_io.
read_habitat_chg_csv
(args)¶ Parses and verifies a Habitat Change Parameters CSV file and returns a dictionary of information related to the interaction between a species and the given habitats.
Parses the Habitat Change Parameters CSV file for the following vectors:
- Names of Habitats and Regions
- Habitat Area Change
Parameters: args (dictionary) – arguments from the user (same as Fisheries HST entry point)
Returns: - dictionary containing necessary
variables
Return type: habitat_chg_dict (dictionary)
Raises: MissingParameter
– required parameter not includedValueError
– values are out of bounds or of wrong typeIndexError
– likely a file formatting issue
Example Returns:
habitat_chg_dict = { 'Habitats': ['habitat1', 'habitat2', ...], 'Hab_regions': ['region1', 'region2', ...], 'Hab_chg_hx': np.array( [[[...], [...]], [[...], [...]], ...]), }
-
natcap.invest.fisheries.fisheries_hst_io.
read_habitat_dep_csv
(args)¶ Parses and verifies a Habitat Dependency Parameters CSV file and returns a dictionary of information related to the interaction between a species and the given habitats.
Parses the Habitat Parameters CSV file for the following vectors:
- Names of Habitats and Classes
- Habitat-Class Dependency
The following vectors are derived from the information given in the file:
- Classes where movement between habitats occurs
- Number of habitats that a particular class depends upon
Parameters: args (dictionary) – arguments from the user (same as Fisheries HST entry point)
Returns: - dictionary containing necessary
variables
Return type: habitat_dep_dict (dictionary)
Raises: - MissingParameter - required parameter not included
- ValueError - values are out of bounds or of wrong type
- IndexError - likely a file formatting issue
Example Returns:
habitat_dep_dict = { 'Habitats': ['habitat1', 'habitat2', ...], 'Hab_classes': ['class1', 'class2', ...], 'Hab_dep_ha': np.array( [[[...], [...]], [[...], [...]], ...]), 'Hab_class_mvmt_a': np.array([...]), 'Hab_dep_num_a': np.array([...]), }
-
natcap.invest.fisheries.fisheries_hst_io.
read_population_csv
(args)¶ Parses and verifies a single Population Parameters CSV file
Parses and verifies inputs from the Population Parameters CSV file. If not all necessary vectors are included, the function will raise a MissingParameter exception. Survival matrix will be arranged by class-elements, 2nd dim: sex, and 3rd dim: region. Class vectors will be arranged by class-elements, 2nd dim: sex (depending on whether model is sex-specific) Region vectors will be arraged by region-elements, sex-agnostic.
Parameters: args (dictionary) – arguments provided by user
Returns: - dictionary containing verified population
arguments
Return type: pop_dict (dictionary)
Raises: MissingParameter
– required parameter not includedValueError
– values are out of bounds or of wrong type
Example Returns:
pop_dict = { 'population_csv_path': 'path/to/csv', 'Surv_nat_xsa': np.array( [[...], [...]], [[...], [...]], ...), # Class Vectors 'Classes': np.array([...]), 'Class_vector_names': [...], 'Class_vectors': { 'Vulnfishing': np.array([...], [...]), 'Maturity': np.array([...], [...]), 'Duration': np.array([...], [...]), 'Weight': np.array([...], [...]), 'Fecundity': np.array([...], [...]), }, # Region Vectors 'Regions': np.array([...]), 'Region_vector_names': [...], 'Region_vectors': { 'Exploitationfraction': np.array([...]), 'Larvaldispersal': np.array([...]), }, }
-
natcap.invest.fisheries.fisheries_hst_io.
save_population_csv
(vars_dict)¶ Creates a new Population Parameters CSV file based the provided inputs.
Parameters: vars_dict (dictionary) – variables generated by preprocessor arguments and run. Example Args:
args = { 'workspace_dir': 'path/to/workspace_dir/', 'output_dir': 'path/to/output_dir/', 'sexsp': 2, 'population_csv_path': 'path/to/csv', # original csv file 'Surv_nat_xsa': np.ndarray([...]), 'Surv_nat_xsa_mod': np.ndarray([...]), # Class Vectors 'Classes': np.array([...]), 'Class_vector_names': [...], 'Class_vectors': { 'Vulnfishing': np.array([...], [...]), 'Maturity': np.array([...], [...]), 'Duration': np.array([...], [...]), 'Weight': np.array([...], [...]), 'Fecundity': np.array([...], [...]), }, # Region Vectors 'Regions': np.array([...]), 'Region_vector_names': [...], 'Region_vectors': { 'Exploitationfraction': np.array([...]), 'Larvaldispersal': np.array([...]), }, # other arguments are ignored ... }
Note
- Creates a modified Population Parameters CSV file located in the ‘workspace/output/’ folder
- Currently appends ‘_modified’ to original filename for new filename
Module contents¶
Hydropower Package¶
Model Entry Point¶
-
natcap.invest.hydropower.hydropower_water_yield.
execute
(args)¶ Annual Water Yield: Reservoir Hydropower Production.
Executes the hydropower/water_yield model
Parameters: - args['workspace_dir'] (string) – a path to the directory that will write output and other temporary files during calculation. (required)
- args['lulc_path'] (string) – a path to a land use/land cover raster whose LULC indexes correspond to indexes in the biophysical table input. Used for determining soil retention and other biophysical properties of the landscape. (required)
- args['depth_to_root_rest_layer_path'] (string) – a path to an input raster describing the depth of “good” soil before reaching this restrictive layer (required)
- args['precipitation_path'] (string) – a path to an input raster describing the average annual precipitation value for each cell (mm) (required)
- args['pawc_path'] (string) – a path to an input raster describing the plant available water content value for each cell. Plant Available Water Content fraction (PAWC) is the fraction of water that can be stored in the soil profile that is available for plants’ use. PAWC is a fraction from 0 to 1 (required)
- args['eto_path'] (string) – a path to an input raster describing the annual average evapotranspiration value for each cell. Potential evapotranspiration is the potential loss of water from soil by both evaporation from the soil and transpiration by healthy Alfalfa (or grass) if sufficient water is available (mm) (required)
- args['watersheds_path'] (string) – a path to an input shapefile of the watersheds of interest as polygons. (required)
- args['sub_watersheds_path'] (string) – a path to an input shapefile of
the subwatersheds of interest that are contained in the
args['watersheds_path']
shape provided as input. (optional) - args['biophysical_table_path'] (string) – a path to an input CSV table of land use/land cover classes, containing data on biophysical coefficients such as root_depth (mm) and Kc, which are required. A column with header LULC_veg is also required which should have values of 1 or 0, 1 indicating a land cover type of vegetation, a 0 indicating non vegetation or wetland, water. NOTE: these data are attributes of each LULC class rather than attributes of individual cells in the raster map (required)
- args['seasonality_constant'] (float) – floating point value between 1 and 30 corresponding to the seasonal distribution of precipitation (required)
- args['results_suffix'] (string) – a string that will be concatenated onto the end of file names (optional)
- args['demand_table_path'] (string) – (optional) if a non-empty string, a path to an input CSV table of LULC classes, showing consumptive water use for each landuse / land-cover type (cubic meters per year) to calculate water scarcity.
- args['valuation_table_path'] (string) –
(optional) if a non-empty string, a path to an input CSV table of hydropower stations with the following fields to calculate valuation:
(‘ws_id’, ‘time_span’, ‘discount’, ‘efficiency’, ‘fraction’, ‘cost’, ‘height’, ‘kw_price’)Required if
calculate_valuation
is True. - args['n_workers'] (int) – (optional) The number of worker processes to use for processing this model. If omitted, computation will take place in the current process.
Returns: None
Hydropower Water Yield¶
InVEST Hydropower Water Yield model.
-
natcap.invest.hydropower.hydropower_water_yield.
aet_op
(fractp, precip, precip_nodata, output_nodata)¶ Compute actual evapotranspiration values.
Parameters: - fractp (numpy.ndarray float) – fractp raster values.
- precip (numpy.ndarray) – precipitation raster values (mm).
- precip_nodata (float) – nodata value from the precip raster.
- output_nodata (float) – nodata value assigned to output of raster_calculator.
Returns: numpy.ndarray of actual evapotranspiration values (mm).
-
natcap.invest.hydropower.hydropower_water_yield.
compute_rsupply_volume
(watershed_results_vector_path)¶ Calculate the total realized water supply volume.
And the mean realized water supply volume per hectare for the given sheds. Output units in cubic meters and cubic meters per hectare respectively.
Parameters: watershed_results_vector_path (string) – a path to a vector that contains fields ‘wyield_vol’ and ‘wyield_mn’. Returns: None.
-
natcap.invest.hydropower.hydropower_water_yield.
compute_water_yield_volume
(watershed_results_vector_path)¶ Calculate the water yield volume per sub-watershed or watershed.
Results are added to a ‘wyield_vol’ field in watershed_results_vector_path. Units are cubic meters.
Parameters: watershed_results_vector_path (str) – Path to a sub-watershed or watershed vector. This vector’s features should have a ‘wyield_mn’ attribute. Returns: None.
-
natcap.invest.hydropower.hydropower_water_yield.
compute_watershed_valuation
(watershed_results_vector_path, val_dict)¶ Compute net present value and energy for the watersheds.
Parameters: - watershed_results_vector_path (string) – Path to an OGR shapefile for the watershed results. Where the results will be added.
- val_dict (dict) – a python dictionary that has all the valuation parameters for each watershed.
Returns: None.
-
natcap.invest.hydropower.hydropower_water_yield.
convert_vector_to_csv
(base_vector_path, target_csv_path)¶ Create a CSV with all the fields present in vector attribute table.
Parameters: - base_vector_path (string) – Path to the watershed shapefile in the output workspace.
- target_csv_path (string) – Path to a CSV to create in the output workspace.
Returns: None
-
natcap.invest.hydropower.hydropower_water_yield.
create_vector_output
(base_vector_path, target_vector_path, ws_id_name, stats_path_list, valuation_params)¶ Create the main vector outputs of this model.
Join results of zonal stats to copies of the watershed shapefiles. Also do optional scarcity and valuation calculations.
Parameters: - base_vector_path (string) – Path to a watershed shapefile provided in the args dictionary.
- target_vector_path (string) – Path where base_vector_path will be copied to in the output workspace.
- ws_id_name (string) – Either ‘ws_id’ or ‘subws_id’, which are required names of a unique ID field in the watershed and subwatershed shapefiles, respectively. Used to determine if the polygons represent watersheds or subwatersheds.
- stats_path_list (list) – List of file paths to pickles storing the zonal stats results.
- valuation_params (dict) – The dictionary built from args[‘valuation_table_path’]. Or None if valuation table was not provided.
Returns: None
-
natcap.invest.hydropower.hydropower_water_yield.
execute
(args) Annual Water Yield: Reservoir Hydropower Production.
Executes the hydropower/water_yield model
Parameters: - args['workspace_dir'] (string) – a path to the directory that will write output and other temporary files during calculation. (required)
- args['lulc_path'] (string) – a path to a land use/land cover raster whose LULC indexes correspond to indexes in the biophysical table input. Used for determining soil retention and other biophysical properties of the landscape. (required)
- args['depth_to_root_rest_layer_path'] (string) – a path to an input raster describing the depth of “good” soil before reaching this restrictive layer (required)
- args['precipitation_path'] (string) – a path to an input raster describing the average annual precipitation value for each cell (mm) (required)
- args['pawc_path'] (string) – a path to an input raster describing the plant available water content value for each cell. Plant Available Water Content fraction (PAWC) is the fraction of water that can be stored in the soil profile that is available for plants’ use. PAWC is a fraction from 0 to 1 (required)
- args['eto_path'] (string) – a path to an input raster describing the annual average evapotranspiration value for each cell. Potential evapotranspiration is the potential loss of water from soil by both evaporation from the soil and transpiration by healthy Alfalfa (or grass) if sufficient water is available (mm) (required)
- args['watersheds_path'] (string) – a path to an input shapefile of the watersheds of interest as polygons. (required)
- args['sub_watersheds_path'] (string) – a path to an input shapefile of
the subwatersheds of interest that are contained in the
args['watersheds_path']
shape provided as input. (optional) - args['biophysical_table_path'] (string) – a path to an input CSV table of land use/land cover classes, containing data on biophysical coefficients such as root_depth (mm) and Kc, which are required. A column with header LULC_veg is also required which should have values of 1 or 0, 1 indicating a land cover type of vegetation, a 0 indicating non vegetation or wetland, water. NOTE: these data are attributes of each LULC class rather than attributes of individual cells in the raster map (required)
- args['seasonality_constant'] (float) – floating point value between 1 and 30 corresponding to the seasonal distribution of precipitation (required)
- args['results_suffix'] (string) – a string that will be concatenated onto the end of file names (optional)
- args['demand_table_path'] (string) – (optional) if a non-empty string, a path to an input CSV table of LULC classes, showing consumptive water use for each landuse / land-cover type (cubic meters per year) to calculate water scarcity.
- args['valuation_table_path'] (string) –
(optional) if a non-empty string, a path to an input CSV table of hydropower stations with the following fields to calculate valuation:
(‘ws_id’, ‘time_span’, ‘discount’, ‘efficiency’, ‘fraction’, ‘cost’, ‘height’, ‘kw_price’)Required if
calculate_valuation
is True. - args['n_workers'] (int) – (optional) The number of worker processes to use for processing this model. If omitted, computation will take place in the current process.
Returns: None
-
natcap.invest.hydropower.hydropower_water_yield.
fractp_op
(Kc, eto, precip, root, soil, pawc, veg, nodata_dict, seasonality_constant)¶ Calculate actual evapotranspiration fraction of precipitation.
Parameters: - Kc (numpy.ndarray) – Kc (plant evapotranspiration coefficient) raster values
- eto (numpy.ndarray) – potential evapotranspiration raster values (mm)
- precip (numpy.ndarray) – precipitation raster values (mm)
- root (numpy.ndarray) – root depth (maximum root depth for vegetated land use classes) raster values (mm)
- soil (numpy.ndarray) – depth to root restricted layer raster values (mm)
- pawc (numpy.ndarray) – plant available water content raster values
- veg (numpy.ndarray) – 1 or 0 where 1 depicts the land type as vegetation and 0 depicts the land type as non vegetation (wetlands, urban, water, etc…). If 1 use regular AET equation if 0 use: AET = Kc * ETo
- nodata_dict (dict) – stores nodata values keyed by raster names
- seasonality_constant (float) – floating point value between 1 and 30 corresponding to the seasonal distribution of precipitation.
Returns: - numpy.ndarray (float) of actual evapotranspiration as fraction
of precipitation.
-
natcap.invest.hydropower.hydropower_water_yield.
pet_op
(eto_pix, Kc_pix, eto_nodata, output_nodata)¶ Calculate the plant potential evapotranspiration.
Parameters: - eto_pix (numpy.ndarray) – a numpy array of ETo
- Kc_pix (numpy.ndarray) – a numpy array of Kc coefficient
- precip_nodata (float) – nodata value from the precip raster
- output_nodata (float) – nodata value assigned to output of raster_calculator
Returns: numpy.ndarray of potential evapotranspiration (mm)
-
natcap.invest.hydropower.hydropower_water_yield.
wyield_op
(fractp, precip, precip_nodata, output_nodata)¶ Calculate water yield.
Parameters: - fractp (numpy.ndarray float) – fractp raster values.
- precip (numpy.ndarray) – precipitation raster values (mm).
- precip_nodata (float) – nodata value from the precip raster.
- output_nodata (float) – nodata value assigned to output of raster_calculator.
Returns: numpy.ndarray of water yield value (mm).
-
natcap.invest.hydropower.hydropower_water_yield.
zonal_stats_tofile
(base_vector_path, raster_path, target_stats_pickle)¶ Calculate zonal statistics for watersheds and write results to a file.
Parameters: - base_vector_path (string) – Path to the watershed shapefile in the output workspace.
- raster_path (string) – Path to raster to aggregate.
- target_stats_pickle (string) – Path to pickle file to store dictionary returned by zonal stats.
Returns: None
Module contents¶
Pollination Package¶
Model Entry Point¶
Overlap Analysis¶
Overlap Analysis Core¶
Overlap Analysis Management Zone¶
Overlap Analysis Management Zone Core¶
Module contents¶
Recreation Package¶
Model Entry Point¶
-
natcap.invest.recreation.recmodel_client.
execute
(args)¶ Recreation.
Execute recreation client model on remote server.
Parameters: - args['workspace_dir'] (string) – path to workspace directory
- args['aoi_path'] (string) – path to AOI vector
- args['hostname'] (string) – FQDN to recreation server
- args['port'] (string or int) – port on hostname for recreation server
- args['start_year'] (string) – start year in form YYYY. This year is the inclusive lower bound to consider points in the PUD and regression.
- args['end_year'] (string) – end year in form YYYY. This year is the inclusive upper bound to consider points in the PUD and regression.
- args['grid_aoi'] (boolean) – if true the polygon vector in args[‘aoi_path’] should be gridded into a new vector and the recreation model should be executed on that
- args['grid_type'] (string) – optional, but must exist if args[‘grid_aoi’] is True. Is one of ‘hexagon’ or ‘square’ and indicates the style of gridding.
- args['cell_size'] (string/float) – optional, but must exist if args[‘grid_aoi’] is True. Indicates the cell size of square pixels and the width of the horizontal axis for the hexagonal cells.
- args['compute_regression'] (boolean) – if True, then process the predictor table and scenario table (if present).
- args['predictor_table_path'] (string) –
required if args[‘compute_regression’] is True. Path to a table that describes the regression predictors, their IDs and types. Must contain the fields ‘id’, ‘path’, and ‘type’ where:
- ’id’: is a <=10 character length ID that is used to uniquely describe the predictor. It will be added to the output result shapefile attribute table which is an ESRI Shapefile, thus limited to 10 characters.
- ’path’: an absolute or relative (to this table) path to the predictor dataset, either a vector or raster type.
- ’type’: one of the following,
- ’raster_mean’: mean of values in the raster under the response polygon
- ’raster_sum’: sum of values in the raster under the response polygon
- ’point_count’: count of the points contained in the response polygon
- ’point_nearest_distance’: distance to the nearest point from the response polygon
- ’line_intersect_length’: length of lines that intersect with the response polygon in projected units of AOI
- ’polygon_area’: area of the polygon contained within response polygon in projected units of AOI
- args['scenario_predictor_table_path'] (string) – (optional) if present runs the scenario mode of the recreation model with the datasets described in the table on this path. Field headers are identical to args[‘predictor_table_path’] and ids in the table are required to be identical to the predictor list.
- args['results_suffix'] (string) – optional, if exists is appended to any output file paths.
Returns: None
Recreation Server¶
Recreation Client¶
InVEST Recreation Client.
-
natcap.invest.recreation.recmodel_client.
delay_op
(last_time, time_delay, func)¶ Execute func if last_time + time_delay >= current time.
Parameters: - last_time (float) – last time in seconds that func was triggered
- time_delay (float) – time to wait in seconds since last_time before triggering func
- func (function) – parameterless function to invoke if current_time >= last_time + time_delay
Returns: If func was triggered, return the time which it was triggered in seconds, otherwise return last_time.
-
natcap.invest.recreation.recmodel_client.
execute
(args) Recreation.
Execute recreation client model on remote server.
Parameters: - args['workspace_dir'] (string) – path to workspace directory
- args['aoi_path'] (string) – path to AOI vector
- args['hostname'] (string) – FQDN to recreation server
- args['port'] (string or int) – port on hostname for recreation server
- args['start_year'] (string) – start year in form YYYY. This year is the inclusive lower bound to consider points in the PUD and regression.
- args['end_year'] (string) – end year in form YYYY. This year is the inclusive upper bound to consider points in the PUD and regression.
- args['grid_aoi'] (boolean) – if true the polygon vector in args[‘aoi_path’] should be gridded into a new vector and the recreation model should be executed on that
- args['grid_type'] (string) – optional, but must exist if args[‘grid_aoi’] is True. Is one of ‘hexagon’ or ‘square’ and indicates the style of gridding.
- args['cell_size'] (string/float) – optional, but must exist if args[‘grid_aoi’] is True. Indicates the cell size of square pixels and the width of the horizontal axis for the hexagonal cells.
- args['compute_regression'] (boolean) – if True, then process the predictor table and scenario table (if present).
- args['predictor_table_path'] (string) –
required if args[‘compute_regression’] is True. Path to a table that describes the regression predictors, their IDs and types. Must contain the fields ‘id’, ‘path’, and ‘type’ where:
- ’id’: is a <=10 character length ID that is used to uniquely describe the predictor. It will be added to the output result shapefile attribute table which is an ESRI Shapefile, thus limited to 10 characters.
- ’path’: an absolute or relative (to this table) path to the predictor dataset, either a vector or raster type.
- ’type’: one of the following,
- ’raster_mean’: mean of values in the raster under the response polygon
- ’raster_sum’: sum of values in the raster under the response polygon
- ’point_count’: count of the points contained in the response polygon
- ’point_nearest_distance’: distance to the nearest point from the response polygon
- ’line_intersect_length’: length of lines that intersect with the response polygon in projected units of AOI
- ’polygon_area’: area of the polygon contained within response polygon in projected units of AOI
- args['scenario_predictor_table_path'] (string) – (optional) if present runs the scenario mode of the recreation model with the datasets described in the table on this path. Field headers are identical to args[‘predictor_table_path’] and ids in the table are required to be identical to the predictor list.
- args['results_suffix'] (string) – optional, if exists is appended to any output file paths.
Returns: None
Recreation Workspace Fetcher¶
InVEST recreation workspace fetcher.
-
natcap.invest.recreation.recmodel_workspace_fetcher.
execute
(args)¶ Fetch workspace from remote server.
After the call a .zip file exists at args[‘workspace_dir’] named args[‘workspace_id’] + ‘.zip’ and contains the zipped workspace of that model run.
Parameters: - args['workspace_dir'] (string) – path to workspace directory
- args['hostname'] (string) – FQDN to recreation server
- args['port'] (string or int) – port on hostname for recreation server
- args['workspace_id'] (string) – workspace identifier
Returns: None
Scenic Quality Package¶
Model Entry Point¶
Scenic Quality¶
Scenic Quality Core¶
Scenic Quality Cython Core¶
Grass Examples¶
Los Sextante¶
Viewshed Grass¶
Viewshed Sextante¶
Module contents¶
Sediment Delivery Ratio Package¶
Model Entry Point¶
Sediment Delivery Ratio¶
Module contents¶
InVEST Sediment Delivery Ratio (SDR) module.
- The SDR method in this model is based on:
- Winchell, M. F., et al. “Extension and validation of a geographic information system-based method for calculating the Revised Universal Soil Loss Equation length-slope factor for erosion risk assessments in large watersheds.” Journal of Soil and Water Conservation 63.3 (2008): 105-111.
-
natcap.invest.sdr.
execute
(args)¶ Sediment Delivery Ratio.
This function calculates the sediment export and retention of a landscape using the sediment delivery ratio model described in the InVEST user’s guide.
Parameters: - args['workspace_dir'] (string) – output directory for intermediate, temporary, and final files
- args['results_suffix'] (string) – (optional) string to append to any output file names
- args['dem_path'] (string) – path to a digital elevation raster
- args['erosivity_path'] (string) – path to rainfall erosivity index raster
- args['erodibility_path'] (string) – a path to soil erodibility raster
- args['lulc_path'] (string) – path to land use/land cover raster
- args['watersheds_path'] (string) – path to vector of the watersheds
- args['biophysical_table_path'] (string) – path to CSV file with biophysical information of each land use classes. contain the fields ‘usle_c’ and ‘usle_p’
- args['threshold_flow_accumulation'] (number) – number of upstream pixels on the dem to threshold to a stream.
- args['k_param'] (number) – k calibration parameter
- args['sdr_max'] (number) – max value the SDR
- args['ic_0_param'] (number) – ic_0 calibration parameter
- args['drainage_path'] (string) – (optional) path to drainage raster that is used to add additional drainage areas to the internally calculated stream layer
- args['n_workers'] (int) – if present, indicates how many worker processes should be used in parallel processing. -1 indicates single process mode, 0 is single process but non-blocking mode, and >= 1 is number of processes.
Returns: None.
Wave Energy Package¶
Model Entry Point¶
Wave Energy¶
Module contents¶
InVEST Wave Energy Model Core Code
-
exception
natcap.invest.wave_energy.
IntersectionError
¶ Bases:
exceptions.Exception
A custom error message for when the AOI does not intersect any wave data points.
-
natcap.invest.wave_energy.
execute
(args)¶ Wave Energy.
Executes both the biophysical and valuation parts of the wave energy model (WEM). Files will be written on disk to the intermediate and output directories. The outputs computed for biophysical and valuation include: wave energy capacity raster, wave power raster, net present value raster, percentile rasters for the previous three, and a point shapefile of the wave points with attributes.
Parameters: - workspace_dir (str) – Where the intermediate and output folder/files will be saved. (required)
- wave_base_data_path (str) – Directory location of wave base data including WAVEWATCH III (WW3) data and analysis area shapefile. (required)
- analysis_area_path (str) – A string identifying the analysis area of interest. Used to determine wave data shapefile, wave data text file, and analysis area boundary shape. (required)
- aoi_path (str) – A polygon OGR vector outlining a more detailed area within the analysis area. This vector should be projected with linear units being in meters. (required to run Valuation model)
- machine_perf_path (str) – The path of a CSV file that holds the machine performance table. (required)
- machine_param_path (str) – The path of a CSV file that holds the machine parameter table. (required)
- dem_path (str) – The path of the Global Digital Elevation Model (DEM). (required)
- suffix (str) – A python string of characters to append to each output filename (optional)
- valuation_container (boolean) – Indicates whether the model includes valuation
- land_gridPts_path (str) – A CSV file path containing the Landing and Power Grid Connection Points table. (required for Valuation)
- machine_econ_path (str) – A CSV file path for the machine economic parameters table. (required for Valuation)
- number_of_machines (int) – An integer specifying the number of machines for a wave farm site. (required for Valuation)
- n_workers (int) – The number of worker processes to use for processing this model. If omitted, computation will take place in the current process. (optional)
Example Args Dictionary:
{ 'workspace_dir': 'path/to/workspace_dir', 'wave_base_data_path': 'path/to/base_data_dir', 'analysis_area_path': 'West Coast of North America and Hawaii', 'aoi_path': 'path/to/vector', 'machine_perf_path': 'path/to/csv', 'machine_param_path': 'path/to/csv', 'dem_path': 'path/to/raster', 'suffix': '_results', 'valuation_container': True, 'land_gridPts_path': 'path/to/csv', 'machine_econ_path': 'path/to/csv', 'number_of_machines': 28, }
Wind Energy Package¶
Model Entry Point¶
Wave Energy¶
Module contents¶
InVEST Wind Energy model.
-
natcap.invest.wind_energy.
execute
(args)¶ Wind Energy.
This module handles the execution of the wind energy model given the following dictionary:
Parameters: - workspace_dir (str) – a path to the output workspace folder (required)
- wind_data_path (str) – path to a CSV file with the following header: [‘LONG’,’LATI’,’LAM’, ‘K’, ‘REF’]. Each following row is a location with at least the Longitude, Latitude, Scale (‘LAM’), Shape (‘K’), and reference height (‘REF’) at which the data was collected (required)
- aoi_vector_path (str) – a path to an OGR polygon vector that is projected in linear units of meters. The polygon specifies the area of interest for the wind data points. If limiting the wind farm bins by distance, then the aoi should also cover a portion of the land polygon that is of interest (optional for biophysical and no distance masking, required for biophysical and distance masking, required for valuation)
- bathymetry_path (str) – a path to a GDAL raster that has the depth values of the area of interest (required)
- land_polygon_vector_path (str) – a path to an OGR polygon vector that provides a coastline for determining distances from wind farm bins. Enabled by AOI and required if wanting to mask by distances or run valuation
- global_wind_parameters_path (str) – a float for the average distance in kilometers from a grid connection point to a land connection point (required for valuation if grid connection points are not provided)
- suffix (str) – a str to append to the end of the output files (optional)
- turbine_parameters_path (str) – a path to a CSV file that holds the turbines biophysical parameters as well as valuation parameters (required)
- number_of_turbines (int) – an integer value for the number of machines for the wind farm (required for valuation)
- min_depth (float) – a float value for the minimum depth for offshore wind farm installation (meters) (required)
- max_depth (float) – a float value for the maximum depth for offshore wind farm installation (meters) (required)
- min_distance (float) – a float value for the minimum distance from shore for offshore wind farm installation (meters) The land polygon must be selected for this input to be active (optional, required for valuation)
- max_distance (float) – a float value for the maximum distance from shore for offshore wind farm installation (meters) The land polygon must be selected for this input to be active (optional, required for valuation)
- valuation_container (boolean) – Indicates whether model includes valuation
- foundation_cost (float) – a float representing how much the foundation will cost for the specific type of turbine (required for valuation)
- discount_rate (float) – a float value for the discount rate (required for valuation)
- grid_points_path (str) – a path to a CSV file that specifies the landing and grid point locations (optional)
- avg_grid_distance (float) – a float for the average distance in kilometers from a grid connection point to a land connection point (required for valuation if grid connection points are not provided)
- price_table (boolean) – a bool indicating whether to use the wind energy price table or not (required)
- wind_schedule (str) – a path to a CSV file for the yearly prices of wind energy for the lifespan of the farm (required if ‘price_table’ is true)
- wind_price (float) – a float for the wind energy price at year 0 (required if price_table is false)
- rate_change (float) – a float as a percent for the annual rate of change in the price of wind energy. (required if price_table is false)
- n_workers (int) – The number of worker processes to use for processing this model. If omitted, computation will take place in the current process. (optional)
Example Args Dictionary:
{ 'workspace_dir': 'path/to/workspace_dir', 'wind_data_path': 'path/to/file', 'aoi_vector_path': 'path/to/shapefile', 'bathymetry_path': 'path/to/raster', 'land_polygon_vector_path': 'path/to/shapefile', 'global_wind_parameters_path': 'path/to/csv', 'suffix': '_results', 'turbine_parameters_path': 'path/to/csv', 'number_of_turbines': 10, 'min_depth': 3, 'max_depth': 60, 'min_distance': 0, 'max_distance': 200000, 'valuation_container': True, 'foundation_cost': 3.4, 'discount_rate': 7.0, 'grid_points_path': 'path/to/csv', 'avg_grid_distance': 4, 'price_table': True, 'wind_schedule': 'path/to/csv', 'wind_price': 0.4, 'rate_change': 0.0, }
Returns: None
Supporting Ecosystem Services¶
Habitat Quality Package¶
Model Entry Point¶
Habitat Quality¶
Module contents¶
InVEST Habitat Quality model.
-
natcap.invest.habitat_quality.
execute
(args)¶ Habitat Quality.
Open files necessary for the portion of the habitat_quality model.
Parameters: - workspace_dir (string) – a path to the directory that will write output and other temporary files (required)
- lulc_cur_path (string) – a path to an input land use/land cover raster (required)
- lulc_fut_path (string) – a path to an input land use/land cover raster (optional)
- lulc_bas_path (string) – a path to an input land use/land cover raster (optional, but required for rarity calculations)
- threat_folder (string) – a path to the directory that will contain all threat rasters (required)
- threats_table_path (string) – a path to an input CSV containing data of all the considered threats. Each row is a degradation source and each column a different attribute of the source with the following names: ‘THREAT’,’MAX_DIST’,’WEIGHT’ (required).
- access_vector_path (string) – a path to an input polygon shapefile containing data on the relative protection against threats (optional)
- sensitivity_table_path (string) – a path to an input CSV file of LULC types, whether they are considered habitat, and their sensitivity to each threat (required)
- half_saturation_constant (float) – a python float that determines the spread and central tendency of habitat quality scores (required)
- suffix (string) – a python string that will be inserted into all raster path paths just before the file extension.
Example Args Dictionary:
{ 'workspace_dir': 'path/to/workspace_dir', 'lulc_cur_path': 'path/to/lulc_cur_raster', 'lulc_fut_path': 'path/to/lulc_fut_raster', 'lulc_bas_path': 'path/to/lulc_bas_raster', 'threat_raster_folder': 'path/to/threat_rasters/', 'threats_table_path': 'path/to/threats_csv', 'access_vector_path': 'path/to/access_shapefile', 'sensitivity_table_path': 'path/to/sensitivity_csv', 'half_saturation_constant': 0.5, 'suffix': '_results', }
Returns: None
-
natcap.invest.habitat_quality.
make_linear_decay_kernel_path
(max_distance, kernel_path)¶ Create a linear decay kernel as a raster.
Pixels in raster are equal to d / max_distance where d is the distance to the center of the raster in number of pixels.
Parameters: - max_distance (int) – number of pixels out until the decay is 0.
- kernel_path (string) – path to output raster whose values are in (0,1) representing distance to edge. Size is (max_distance * 2 + 1)^2
Returns: None
-
natcap.invest.habitat_quality.
map_raster_to_dict_values
(key_raster_path, out_path, attr_dict, field, out_nodata, values_required)¶ Creates a new raster from ‘key_raster’ where the pixel values from ‘key_raster’ are the keys to a dictionary ‘attr_dict’. The values corresponding to those keys is what is written to the new raster. If a value from ‘key_raster’ does not appear as a key in ‘attr_dict’ then raise an Exception if ‘raise_error’ is True, otherwise return a ‘out_nodata’
- key_raster_path - a GDAL raster path dataset whose pixel values relate to
- the keys in ‘attr_dict’
out_path - a string for the output path of the created raster attr_dict - a dictionary representing a table of values we are interested
in making into a raster- field - a string of which field in the table or key in the dictionary
- to use as the new raster pixel values
out_nodata - a floating point value that is the nodata value. raise_error - a string that decides how to handle the case where the
value from ‘key_raster’ is not found in ‘attr_dict’. If ‘raise_error’ is ‘values_required’, raise Exception, if ‘none’, return ‘out_nodata’- returns - a GDAL raster, or raises an Exception and fail if:
- raise_error is True and
- the value from ‘key_raster’ is not a key in ‘attr_dict’
-
natcap.invest.habitat_quality.
raster_pixel_count
(raster_path)¶ Count unique pixel values in raster.
Parameters: raster_path (string) – path to a raster Returns: dict of pixel values to frequency.
-
natcap.invest.habitat_quality.
resolve_ambiguous_raster_path
(path, raise_error=True)¶ Determine real path when we don’t know true path extension.
Parameters: - path (string) – file path that includes the name of the file but not its extension
- raise_error (boolean) – if True then function will raise an ValueError if a valid raster file could not be found.
Returns: the full path, plus extension, to the valid raster.
Utilities¶
Reporting Package¶
Table Generator¶
A helper module for generating html tables that are represented as Strings
-
natcap.invest.reporting.table_generator.
add_checkbox_column
(col_list, row_list, checkbox_pos=1)¶ Insert a new column into the list of column dictionaries so that it is the second column dictionary found in the list. Also add the checkbox column header to the list of row dictionaries and subsequent checkbox value
- ‘col_list’- a list of dictionaries that defines the column
structure for the table (required). The order of the columns from left to right is depicted by the index of the column dictionary in the list. Each dictionary in the list has the following keys and values:
‘name’ - a string for the column name (required) ‘total’ - a boolean for whether the column should be
totaled (required)- ‘row_list’ - a list of dictionaries that represent the rows. Each
dictionaries keys should match the column names found in ‘col_list’ (required) Example: [{col_name_1: value, col_name_2: value, …},
{col_name_1: value, col_name_2: value, …}, …]- checkbox_pos - an integer for the position of the checkbox
- column. Defaulted at 1 (optional)
- returns - a tuple of the updated column and rows list of dictionaries
- in that order
-
natcap.invest.reporting.table_generator.
add_totals_row
(col_headers, total_list, total_name, checkbox_total, tdata_tuples)¶ Construct a totals row as an html string. Creates one row element with data where the row gets a class name and the data get a class name if the corresponding column is a totalable column
col_headers - a list of the column headers in order (required)
- total_list - a list of booleans that corresponds to ‘col_headers’ and
- indicates whether a column should be totaled (required)
- total_name - a string for the name of the total row, ex: ‘Total’, ‘Sum’
- (required)
- checkbox_total - a boolean value that distinguishes whether a checkbox
- total row is being added or a regular total row. Checkbox total row is True. This will determine the row class name and row data class name (required)
- tdata_tuples - a list of tuples where the first index in the tuple is a
- boolean which indicates if a table data element has a attribute class. The second index is the String value of that class or None (required)
- return - a string representing the html contents of a row which should
- later be used in a ‘tfoot’ element
-
natcap.invest.reporting.table_generator.
generate_table
(table_dict, attributes=None)¶ Takes in a dictionary representation of a table and generates a String of the the table in the form of hmtl
- table_dict - a dictionary with the following arguments:
- ‘cols’- a list of dictionaries that defines the column
structure for the table (required). The order of the columns from left to right is depicted by the index of the column dictionary in the list. Each dictionary in the list has the following keys and values:
‘name’ - a string for the column name (required) ‘total’ - a boolean for whether the column should be
totaled (required)- ‘attr’ - a dictionary that has key value pairs for
- optional tag attributes (optional). Ex: ‘attr’: {‘class’: ‘offsets’}
- ‘td_class’ - a String to assign as a class name to
- the table data tags under the column. Each table data tag under the column will have a class attribute assigned to ‘td_class’ value (optional)
- ‘rows’ - a list of dictionaries that represent the rows. Each
dictionaries keys should match the column names found in ‘cols’ (possibly empty list) (required) Example: [{col_name_1: value, col_name_2: value, …},
{col_name_1: value, col_name_2: value, …}, …]- ‘checkbox’ - a boolean value for whether there should be a
- checkbox column. If True a ‘selected total’ row will be added to the bottom of the table that will show the total of the columns selected (optional)
- ‘checkbox_pos’ - an integer value for in which column
- position the the checkbox column should appear (optional)
- ‘total’- a boolean value for whether there should be a constant
- total row at the bottom of the table that sums the column values (optional)
- ‘attributes’ - a dictionary of html table attributes. The attribute
- name is the key which gets set to the value of the key. (optional) Example: {‘class’: ‘sorttable’, ‘id’: ‘parcel_table’}
returns - a string representing an html table
-
natcap.invest.reporting.table_generator.
get_dictionary_values_ordered
(dict_list, key_name)¶ Generate a list, with values from ‘key_name’ found in each dictionary in the list of dictionaries ‘dict_list’. The order of the values in the returned list match the order they are retrieved from ‘dict_list’
- dict_list - a list of dictionaries where each dictionary has the same
- keys. Each dictionary should have at least one key:value pair with the key being ‘key_name’ (required)
- key_name - a String or Int for the key name of interest in the
- dictionaries (required)
- return - a list of values from ‘key_name’ in ascending order based
- on ‘dict_list’ keys
-
natcap.invest.reporting.table_generator.
get_row_data
(row_list, col_headers)¶ Construct the rows in a 2D List from the list of dictionaries, using col_headers to properly order the row data.
- ‘row_list’ - a list of dictionaries that represent the rows. Each
dictionaries keys should match the column names found in ‘col_headers’. The rows will be ordered the same as they are found in the dictionary list (required) Example: [{‘col_name_1’:‘9/13’, ‘col_name_3’:’expensive’,
‘col_name_2’:’chips’},- {‘col_name_1’:‘3/13’, ‘col_name_2’:’cheap’,
- ‘col_name_3’:’peanuts’},
- {‘col_name_1’:‘5/12’, ‘col_name_2’:’moderate’,
- ‘col_name_3’:’mints’}]
- col_headers - a List of the names of the column headers in order
- example : [col_name_1, col_name_2, col_name_3…]
return - a 2D list with each inner list representing a row
HTML¶
Module contents¶
natcap.invest.reporting package.
-
natcap.invest.reporting.
add_head_element
(param_args)¶ Generates a string that represents a valid element in the head section of an html file. Currently handles ‘style’ and ‘script’ elements, where both the script and style are locally embedded
param_args - a dictionary that holds the following arguments:
- param_args[‘format’] - a string representing the type of element to
- be added. Currently : ‘script’, ‘style’ (required)
- param_args[‘data_src’] - a string URI path for the external source
- of the element OR a String representing the html (DO NOT include html tags, tags are automatically generated). If a URI the file is read in as a String. (required)
- param_args[‘input_type’] - ‘Text’ or ‘File’. Determines how the
- input from ‘data_src’ is handled (required)
- ‘attributes’ - a dictionary that has key value pairs for
- optional tag attributes (optional). Ex: ‘attributes’: {‘class’: ‘offsets’}
returns - a string representation of the html head element
-
natcap.invest.reporting.
add_text_element
(param_args)¶ Generates a string that represents a html text block. The input string should be wrapped in proper html tags
param_args - a dictionary with the following arguments:
param_args[‘text’] - a stringreturns - a string
-
natcap.invest.reporting.
build_table
(param_args)¶ Generates a string representing a table in html format.
- param_args - a dictionary that has the parameters for building up the
html table. The dictionary includes the following:
- ‘attributes’ - a dictionary of html table attributes. The attribute
- name is the key which gets set to the value of the key. (optional) Example: {‘class’: ‘sorttable’, ‘id’: ‘parcel_table’}
- param_args[‘sortable’] - a boolean value that determines whether the
- table should be sortable (required)
- param_args[‘data_type’] - a string depicting the type of input to
- build the table from. Either ‘shapefile’, ‘csv’, or ‘dictionary’ (required)
- param_args[‘data’] - a URI to a csv or shapefile OR a list of
dictionaries. If a list of dictionaries the data should be represented in the following format: (required)
- [{col_name_1: value, col_name_2: value, …},
- {col_name_1: value, col_name_2: value, …}, …]
- param_args[‘key’] - a string that depicts which column (csv) or
- field (shapefile) will be the unique key to use in extracting the data into a dictionary. (required for ‘data_type’ ‘shapefile’ and ‘csv’)
- param_args[‘columns’] - a list of dictionaries that defines the
column structure for the table (required). The order of the columns from left to right is depicted by the index of the column dictionary in the list. Each dictionary in the list has the following keys and values:
‘name’ - a string for the column name (required) ‘total’ - a boolean for whether the column should be
totaled (required)- ‘attr’ - a dictionary that has key value pairs for
- optional tag attributes (optional). Ex: ‘attr’: {‘class’: ‘offsets’}
- ‘td_class’ - a String to assign as a class name to
- the table data tags under the column. Each table data tag under the column will have a class attribute assigned to ‘td_class’ value (optional)
- param_args[‘total’] - a boolean value where if True a constant
- total row will be placed at the bottom of the table that sums the columns (required)
returns - a string that represents an html table
-
natcap.invest.reporting.
data_dict_to_list
(data_dict)¶ Abstract out inner dictionaries from data_dict into a list, where the inner dictionaries are added to the list in the order of their sorted keys
- data_dict - a dictionary with unique keys pointing to dictionaries.
- Could be empty (required)
returns - a list of dictionaries, or empty list if data_dict is empty
-
natcap.invest.reporting.
generate_report
(args)¶ Generate an html page from the arguments given in ‘reporting_args’
- reporting_args[title] - a string for the title of the html page
- (required)
- reporting_args[sortable] - a boolean value indicating whether
- the sorttable.js library should be added for table sorting functionality (optional)
- reporting_args[totals] - a boolean value indicating whether
- the totals_function.js script should be added for table totals functionality (optional)
- reporting_args[out_uri] - a URI to the output destination for the html
- page (required)
- reporting_args[elements] - a list of dictionaries that represent html
elements to be added to the html page. (required) If no elements are provided (list is empty) a blank html page will be generated. The 3 main element types are ‘table’, ‘head’, and ‘text’. All elements share the following arguments:
- ‘type’ - a string that depicts the type of element being add.
- Currently ‘table’, ‘head’, and ‘text’ are defined (required)
- ‘section’ - a string that depicts whether the element belongs
- in the body or head of the html page. Values: ‘body’ | ‘head’ (required)
Table element dictionary has at least the following additional arguments:
- ‘attributes’ - a dictionary of html table attributes. The
- attribute name is the key which gets set to the value of the key. (optional) Example: {‘class’: ‘sorttable’, ‘id’: ‘parcel_table’}
- ‘sortable’ - a boolean value for whether the tables columns
- should be sortable (required)
- ‘checkbox’ - a boolean value for whether there should be a
- checkbox column. If True a ‘selected total’ row will be added to the bottom of the table that will show the total of the columns selected (optional)
- ‘checkbox_pos’ - an integer value for in which column
- position the the checkbox column should appear (optional)
- ‘data_type’ - one of the following string values:
- ‘shapefile’|’hg csv’|’dictionary’. Depicts the type of data structure to build the table from (required)
- ‘data’ - either a list of dictionaries if ‘data_type’ is
‘dictionary’ or a URI to a CSV table or shapefile if ‘data_type’ is ‘shapefile’ or ‘csv’ (required). If a list of dictionaries, each dictionary should have keys that represent the columns, where each dictionary is a row (list could be empty) How the rows are ordered are defined by their index in the list. Formatted example: [{col_name_1: value, col_name_2: value, …},
{col_name_1: value, col_name_2: value, …}, …]- ‘key’ - a string that defines which column or field should be
- used as the keys for extracting data from a shapefile or csv table ‘key_field’. (required for ‘data_type’ = ‘shapefile’ | ‘csv’)
- ‘columns’- a list of dictionaries that defines the column
structure for the table (required). The order of the columns from left to right is depicted by the index of the column dictionary in the list. Each dictionary in the list has the following keys and values:
‘name’ - a string for the column name (required) ‘total’ - a boolean for whether the column should be
totaled (required)- ‘attr’ - a dictionary that has key value pairs for
- optional tag attributes (optional). Ex: ‘attr’: {‘class’: ‘offsets’}
- ‘td_class’ - a String to assign as a class name to
- the table data tags under the column. Each table data tag under the column will have a class attribute assigned to ‘td_class’ value (optional)
- ‘total’- a boolean value for whether there should be a constant
- total row at the bottom of the table that sums the column values (optional)
Head element dictionary has at least the following additional arguments:
- ‘format’ - a string representing the type of head element being
- added. Currently ‘script’ (javascript) and ‘style’ (css style) accepted (required)
- ‘data_src’- a URI to the location of the external file for
- either the ‘script’ or the ‘style’ OR a String representing the html script or style (DO NOT include the tags) (required)
- ‘input_type’ - a String, ‘File’ or ‘Text’ that refers to how
- ‘data_src’ is being passed in (URI vs String) (required).
- ‘attributes’ - a dictionary that has key value pairs for
- optional tag attributes (optional). Ex: ‘attributes’: {‘id’: ‘muni_data’}
Text element dictionary has at least the following additional arguments:
- ‘text’- a string to add as a paragraph element in the html page
- (required)
returns - nothing
-
natcap.invest.reporting.
u
(string)¶
-
natcap.invest.reporting.
write_html
(html_obj, out_uri)¶ Write an html file to ‘out_uri’ from html element represented as strings in ‘html_obj’
- html_obj - a dictionary with two keys, ‘head’ and ‘body’, that point to
lists. The list for each key is a list of the htmls elements as strings (required) example: {‘head’:[‘elem_1’, ‘elem_2’,…],
‘body’:[‘elem_1’, ‘elem_2’,…]}
out_uri - a URI for the output html file
returns - nothing