rook.operations package

Operation execution adapters and data operations.

class rook.operations.Average(collection, file_namer='standard', split_method='time:auto', output_dir=None, output_type='netcdf', **params)

Bases: Operation

get_operation_callable()

rook.operations.AverageByDimension(output_dir)

rook.operations.AverageByShape(output_dir)

rook.operations.AverageByTime(output_dir)

class rook.operations.AverageShape(collection, file_namer='standard', split_method='time:auto', output_dir=None, output_type='netcdf', **params)

Bases: Operation

get_operation_callable()

class rook.operations.AverageTime(collection, file_namer='standard', split_method='time:auto', output_dir=None, output_type='netcdf', **params)

Bases: Operation

get_operation_callable()

rook.operations.Concat(output_dir)

rook.operations.ConcatOperation

alias of Concat

class rook.operations.Operation(collection, file_namer='standard', split_method='time:auto', output_dir=None, output_type='netcdf', **params)

Bases: object

Base class for all operations.

calculate()

get_operation_callable()

class rook.operations.Operator(output_dir, prefix, runner)

Bases: object

Workflow operation adapter.

call(args)

rook.operations.Regrid(output_dir)

rook.operations.RegridOperation

alias of Regrid

rook.operations.Subset(output_dir)

rook.operations.SubsetOperation

alias of Subset

rook.operations.WeightedAverage(output_dir)

rook.operations.make_workflow_operator(name, output_dir)

Return the configured workflow operation adapter.

rook.operations.run_average_by_dim(args)

rook.operations.run_average_by_shape(args)

rook.operations.run_average_by_time(args)

rook.operations.run_concat(args)

rook.operations.run_regrid(args)

rook.operations.run_subset(args)

rook.operations.run_weighted_average(args)

Submodules

rook.operations.average module

Operations for averaging data over dimensions, shape or time.

class rook.operations.average.Average(collection, file_namer='standard', split_method='time:auto', output_dir=None, output_type='netcdf', **params)

Bases: Operation

get_operation_callable()

rook.operations.average.average_over_dims(collection, dims=None, ignore_undetected_dims=False, output_dir=None, output_type='netcdf', split_method='time:auto', file_namer='standard')

rook.operations.average.average_shape(collection, shape, variable=None, output_dir=None, output_type='netcdf', split_method='time:auto', file_namer='standard')

rook.operations.average.average_time(collection, freq='year', output_dir=None, output_type='netcdf', split_method='time:auto', file_namer='standard')

rook.operations.base module

Base class for operation execution.

class rook.operations.base.Operation(collection, file_namer='standard', split_method='time:auto', output_dir=None, output_type='netcdf', **params)

Bases: object

Base class for all operations.

calculate()

get_operation_callable()

class rook.operations.base.PreparedCollection(value: tuple[DatasetSource, ...])

Bases: object

Collection wrapper for already-resolved dataset sources.

value: tuple[DatasetSource, ...]

rook.operations.base.is_prepared_dataset_collection(collection)

Return whether a collection contains normalized dataset sources.

rook.operations.base.resolve_collection(collection)

Return a collection value suitable for operation consolidation.

rook.operations.concat module

class rook.operations.concat.Concat(collection, file_namer='standard', split_method='time:auto', output_dir=None, output_type='netcdf', **params)

Bases: Operation

calculate()

rook.operations.concat.apply_concat_calendar_fix(ds, fix_provider=None)

Apply concat-specific preparation before grouped files are combined.

rook.operations.concat.apply_concat_dataset_fixes(collection, output_dir, fix_provider=None)

Apply concat-specific decadal fixes to each opened dataset.

rook.operations.concat.combine_concat_datasets(datasets, dim, standard_name)

Concatenate datasets and restore concat coordinate metadata.

rook.operations.concat.concat(collection, time=None, time_components=None, dims=None, ignore_undetected_dims=False, output_dir=None, output_type='netcdf', split_method='time:auto', file_namer='standard', apply_average=False, fix_provider=None)

rook.operations.concat.concat_dimension(dims)

Return the dimension name and standard name used for concat.

rook.operations.concat.dataset_paths_by_id(sources)

Return concat input paths keyed by dataset id.

rook.operations.concat.drop_time_bnds(ds: Dataset) → Dataset

rook.operations.concat.finalise_concat_output(ds, params, dim)

Apply optional average and time selection to concat output.

rook.operations.consolidate module

Consolidate file paths for each dataset in a collection.

rook.operations.consolidate.consolidate(collection, **kwargs)

Find file paths relating to each input dataset.

rook.operations.consolidate.get_files_matching_time_range(time_param, file_paths)

Filter files whose years intersect requested time range.

rook.operations.consolidate.get_year(value, default)

Get a year from a datetime string.

rook.operations.consolidate.get_years_from_file(fpath)

Attempt to extract years from file name or file time axis.

rook.operations.consolidate.to_year(time_string)

Return the year in a time string as an integer.

rook.operations.execution module

Execution adapters used by WPS processes and workflows.

rook.operations.execution.AverageByDimension(output_dir)

rook.operations.execution.AverageByShape(output_dir)

rook.operations.execution.AverageByTime(output_dir)

rook.operations.execution.Concat(output_dir)

class rook.operations.execution.Operator(output_dir, prefix, runner)

Bases: object

Workflow operation adapter.

call(args)

rook.operations.execution.Regrid(output_dir)

rook.operations.execution.Subset(output_dir)

rook.operations.execution.WeightedAverage(output_dir)

class rook.operations.execution.WorkflowOperation(prefix: str, runner: object)

Bases: object

Configuration for a workflow operation adapter.

prefix: str

runner: object

rook.operations.execution.average_dimension_operator(output_dir)

rook.operations.execution.average_shape_operator(output_dir)

rook.operations.execution.average_time_operator(output_dir)

rook.operations.execution.collect_file_uris(operation, args)

Run an operation function and return its file URIs.

rook.operations.execution.concat_operator(output_dir)

rook.operations.execution.make_workflow_operator(name, output_dir)

Return the configured workflow operation adapter.

rook.operations.execution.prepare_workflow_file_inputs(args, source)

Return operation inputs for files produced by a previous workflow step.

rook.operations.execution.regrid_operator(output_dir)

rook.operations.execution.run_average_by_dim(args)

rook.operations.execution.run_average_by_shape(args)

rook.operations.execution.run_average_by_time(args)

rook.operations.execution.run_concat(args)

rook.operations.execution.run_regrid(args)

rook.operations.execution.run_subset(args)

rook.operations.execution.run_workflow_files(args, runner)

Run an operation using previous workflow step output files.

rook.operations.execution.subset_operator(output_dir)

rook.operations.execution.weighted_average_operator(output_dir)

rook.operations.normalise module

Normalise datasets and hold operation results.

class rook.operations.normalise.ResultSet(inputs=None)

Bases: object

A class to hold the results from an operation.

add(dset, result)

Add outputs with ds id key and collect file URIs.

rook.operations.normalise.keep_dataset(ds)

Return a dataset unchanged.

rook.operations.normalise.normalise(collection, *, fix_provider=None)

Open input collections.

rook.operations.normalise.normalise_file_groups(collection, *, prepare_dataset=None, concat_dim='time', opener=<function open_xr_dataset>)

Open grouped file paths and concatenate each group.

rook.operations.processor module

Dispatch processing operations in serial or parallel mode.

rook.operations.processor.process(operation, dset, **kwargs)

Run processing operation on a dataset.

rook.operations.regrid module

Regrid operation.

class rook.operations.regrid.Regrid(collection, file_namer='standard', split_method='time:auto', output_dir=None, output_type='netcdf', **params)

Bases: Operation

get_operation_callable()

rook.operations.regrid.regrid(collection, method='nn', adaptive_masking_threshold=0.5, grid='1deg', output_dir=None, output_type='netcdf', split_method='time:auto', file_namer='standard')

rook.operations.subset module

Subset operation.

class rook.operations.subset.Subset(collection, file_namer='standard', split_method='time:auto', output_dir=None, output_type='netcdf', **params)

Bases: Operation

get_operation_callable()

rook.operations.subset.subset(collection, time=None, area=None, level=None, time_components=None, fix_provider=None, output_dir=None, output_type='netcdf', split_method='time:auto', file_namer='standard')