natcap.invest.recreation.recmodel_server

InVEST Recreation Server.

class natcap.invest.recreation.recmodel_server.RecManager(servers_dict, max_allowable_query)

Bases: object

A class to manage incoming Pyro requests.

This class’s methods will typically be called by a remote client.

calculate_userdays(**kwargs): General purpose try/except wrapper.

estimate_aoi_query_size(bounding_box, dataset)

Count points in quadtree nodes that intersect a bounding box.

This allows for a quick upper-limit estimate of the number of points found within an AOI extent.

Parameters:

bounding_box (list) – of the form [xmin, ymin, xmax, ymax] where coordinates are WGS84 decimal degrees.
dataset (str) – one of ‘flickr’ or ‘twitter’

Returns:

(n points, max number of points allowed by this manager)

Return type:

(int, int)

fetch_aoi_workspaces(**kwargs): General purpose try/except wrapper.

get_valid_year_range(dataset)

Return the min and max year supported for dataset queries.

Parameters:: dataset (str) – one of ‘flickr’ or ‘twitter’.
Returns:: (min_year, max_year)

log_to_client(client_id)

Get queued log messages and return them to client.

Parameters:

client_id (string) – a unique id sent by the Pyro client.

Returns:

LogRecords cannot be returned with the Pyro5 serializer,: so they are returned as dicts to be reconstructed.

Return type:

(dict)

class natcap.invest.recreation.recmodel_server.RecModel(min_year, max_year, cache_workspace, raw_csv_filename=None, quadtree_pickle_filename=None, max_points_per_node=10000, max_depth=10, dataset_name='flickr')

Bases: object

Class that manages quadtree construction and queries.

calc_user_days_in_aoi(zip_file_binary, aoi_filename, start_year, end_year, out_vector_filename, client_id=None)

Calculate annual average and per monthly average user days.

Parameters:

zip_file_binary (string) – a bytestring that is a zip file of a GDAL vector.
aoi_filename (string) – the filename for the AOI expected to be extracted from zip_file_binary.
start_year (string | int) – formatted as ‘YYYY’ or YYYY
end_year (string | int) – formatted as ‘YYYY’ or YYYY
out_vector_filename (string) – base filename of output vector
client_id (string) – a unique id sent by the Pyro client.

Returns:

bytestring of a zipped copy of zip_file_binary
with a “PUD_YR_AVG”, and a “PUD_{MON}_AVG” for {MON} in the calendar months.
string that can be used to uniquely identify this workspace
on the server
string representing the server version

Return type:

(tuple)

find_workspace(workspace_id)

Find the AOI of the workspace specified by workspace_id.

Parameters:: workspace_id (string) – unique workspace ID on server to query.
Returns:: path to a zip file
Return type:: string

get_valid_year_range()

Return the min and max year queriable.

Returns:: (min_year, max_year)

get_version()

Return the rec model server version.

This string can be used to uniquely identify the userday database and algorithm for publication in terms of reproducibility.

n_points_in_intersecting_nodes(bounding_box)

Count points in quadtree nodes that intersect a bounding box.

This allows for a quick upper-limit estimate of the number of points found within an AOI extent.

Parameters:: bounding_box (list) – of the form [xmin, ymin, xmax, ymax] where coordinates are WGS84 decimal degrees.
Returns:: the number of points in the intersecting nodes.
Return type:: int

natcap.invest.recreation.recmodel_server.build_quadtree_shape(quad_tree_shapefile_path, quadtree, spatial_reference)

Generate a vector of the quadtree geometry.

Parameters:

quad_tree_shapefile_path (string) – path to save the vector
quadtree (out_of_core_quadtree.OutOfCoreQuadTree) – quadtree data structure
spatial_reference (osr.SpatialReference) – spatial reference for the output vector

Returns:

None

natcap.invest.recreation.recmodel_server.construct_userday_quadtree(initial_bounding_box, raw_csv_file_list, dataset_name, cache_dir, ooc_qt_picklefilename, max_points_per_node, max_depth, n_workers=None, build_shapefile=True, fast_point_count=False)

Construct a spatial quadtree for fast querying of userday points.

Parameters:

initial_bounding_box (list of int)
raw_csv_file_list (list) – list of filepaths of point CSVs
dataset_name (string) – one of ‘flickr’, ‘twitter’, indicating the expected structure of the csv.
cache_dir (string) – path to a directory that can be used to cache the quadtree files on disk
ooc_qt_picklefilename (string) – name for the pickle file quadtree index created in the cache_dir.
max_points_per_node (int) – maximum number of points to allow per node of the quadtree. A larger amount will cause the quadtree to subdivide.
max_depth (int) – maximum depth of nodes in the quadtree. Once reached, the leaf nodes will not subdivide, even if max_points_per_node is exceeded.
n_workers (int) – number of cores for multiprocessing.
build_shapefile (boolean) – whether or not to create vector geometries representing nodes of the quadtree.
fast_point_count (boolean) – If False, count the number of lines in all the csv files. If True, estimate total number of points by counting a random sample of files.

Returns:

None

natcap.invest.recreation.recmodel_server.execute(args)

Launch recreation manager, initializing RecModel servers.

A call to this function registers a Pyro RPC RecManager entry point given the configuration input parameters described below.

The RecManager instantiates RecModel servers, which parse input data and construct quadtrees if necessary.

For a usage example, see invest/scripts/recreation_server/launch_recserver.sh and invest/scripts/recreation_server/execute_recmodel_server.py

Example:

args = {
    'hostname': '',
    'port': 54322,
    'max_allowable_query': 40_000_000,
    'datasets': {
        'flickr': {
            'raw_csv_point_data_path': 'photos_2005-2017_odlla.csv',
            'min_year': 2005,
            'max_year': 2017
        },
        'twitter': {
            'quadtree_pickle_filename': 'global_twitter_qt.pickle',
            'min_year': 2012,
            'max_year': 2022
        }
    }
}

Parameters:

args['hostname'] (string) – hostname to host Pyro server.
args['port'] (int/or string representation of int) – port number to host Pyro entry point.
args['cache_workspace'] (string) – Path to a local, writeable, directory. Avoid network-mounted volumes.
args['max_allowable_query'] (int) – the maximum number of points allowed within the bounding box of a query.
args['datasets'] (dict) – args for instantiating each RecModel server. Keys should include ‘flickr’, ‘twitter’, or both.

Returns:

Never returns

natcap.invest.recreation.recmodel_server.transplant_quadtree(qt_pickle_filepath, workspace)

Move quadtree filepath references to a local filesystem.

The quadtree index contains paths that are remnants of the filesystem where it was created. Since we’re serving it from a different filesystem, we overwrite those paths and write a new quadtree index file.

Parameters:

qt_pickle_filepath (string) – path to a quadtree pickle file
workspace (string) – path to a local directory to write the modified quadtree index.

Returns:

None