rook.pflow package

Processing-flow layer for request decisions and execution.

class rook.pflow.CatalogCollection(collection: list[str], project: str)[source]

Bases: RequestSource

A request source that should be resolved through a project catalog.

collection: list[str]
project: str
class rook.pflow.DirectDataset(collection: list[str], project: str)[source]

Bases: RequestSource

A request source that should be processed directly without catalog lookup.

collection: list[str]
project: str
class rook.pflow.InvalidRequest(message: str)[source]

Bases: RequestDecision

Decision value reserved for explicit invalid-request outcomes.

message: str
property returns_original_files

Return whether this request should bypass processing.

class rook.pflow.RequestResult(decision: object, output_uris: list[str])[source]

Bases: object

Result values produced by resolving and executing a request.

property dataset_sources

Return the dataset sources prepared for operation execution.

decision: object
property original_file_urls

Return original files selected by the request decision.

output_uris: list[str]
property project

Return the project resolved for the request.

property search_result

Return the catalog search result, when catalog lookup was used.

property use_original_files

Return whether processing was skipped.

class rook.pflow.ReturnOriginalFiles(project: str, search_result: object | None = None, original_file_urls: OrderedDict | None = None)[source]

Bases: RequestDecision

Decision to return catalog file URLs without running an operation.

property dataset_sources

Return no prepared operation sources for original-file responses.

original_file_urls: OrderedDict | None = None
project: str
property returns_original_files

Return whether this request should bypass processing.

search_result: object | None = None
class rook.pflow.RunOperation(project: str, search_result: object | None = None, dataset_sources: tuple[DatasetSource, ...] = ())[source]

Bases: RequestDecision

Decision to run an operation on the requested or prepared sources.

dataset_sources: tuple[DatasetSource, ...] = ()
property original_file_urls

Return no original file URLs for operation responses.

project: str
property returns_original_files

Return whether this request should bypass processing.

search_result: object | None = None
class rook.pflow.WorkflowFiles(files: list[str])[source]

Bases: RequestSource

Files produced by an earlier workflow step.

files: list[str]
rook.pflow.execute_resolved_request(collection, inputs, runner)[source]

Resolve a request, execute it when needed, and translate WPS errors.

rook.pflow.resolve_request_decision(collection, inputs)[source]

Return the decision for a request.

Submodules

rook.pflow.alignment module

Subset-to-file alignment checks.

class rook.pflow.alignment.SubsetAlignmentChecker(input_files, inputs)[source]

Bases: object

rook.pflow.base module

Abstract processing-flow vocabulary.

class rook.pflow.base.DecisionExecutor[source]

Bases: ABC

Executes one request decision.

abstractmethod execute(decision, inputs, runner)[source]

Return output URIs for a request decision.

class rook.pflow.base.RequestDecision[source]

Bases: ABC

One decision about how a request should be handled.

abstract property returns_original_files

Return whether the decision bypasses operation execution.

class rook.pflow.base.RequestPolicy[source]

Bases: object

Optional base for small context-specific request policies.

class rook.pflow.base.RequestResolver[source]

Bases: ABC

Turns raw request inputs into one request decision.

abstractmethod resolve(collection, inputs)[source]

Return one request decision.

class rook.pflow.base.RequestSource[source]

Bases: object

Normalized source for a request.

rook.pflow.catalog module

Catalog lookup, validation, and dataset-source preparation.

rook.pflow.catalog.catalog_for(project)[source]

Return the project catalog or raise an invalid-collection error.

Return normalized sources for catalog-resolved operation inputs.

Resolve a catalog search result and validate all requested collections.

rook.pflow.catalog.validate_search_result(search_result, collection)[source]

Raise when the catalog did not match every requested collection.

rook.pflow.decisions module

Request decision values.

class rook.pflow.decisions.InvalidRequest(message: str)[source]

Bases: RequestDecision

Decision value reserved for explicit invalid-request outcomes.

message: str
property returns_original_files

Return whether this request should bypass processing.

class rook.pflow.decisions.ReturnOriginalFiles(project: str, search_result: object | None = None, original_file_urls: OrderedDict | None = None)[source]

Bases: RequestDecision

Decision to return catalog file URLs without running an operation.

property dataset_sources

Return no prepared operation sources for original-file responses.

original_file_urls: OrderedDict | None = None
project: str
property returns_original_files

Return whether this request should bypass processing.

search_result: object | None = None
class rook.pflow.decisions.RunOperation(project: str, search_result: object | None = None, dataset_sources: tuple[DatasetSource, ...] = ())[source]

Bases: RequestDecision

Decision to run an operation on the requested or prepared sources.

dataset_sources: tuple[DatasetSource, ...] = ()
property original_file_urls

Return no original file URLs for operation responses.

project: str
property returns_original_files

Return whether this request should bypass processing.

search_result: object | None = None

rook.pflow.execution module

Execute request decisions and adapt their outputs.

rook.pflow.execution.collect_original_file_uris(original_file_urls)[source]

Return original file URIs in the same shape as operation outputs.

rook.pflow.execution.execute_decision(decision, inputs, runner)[source]

Return output URIs for a request decision.

rook.pflow.execution.execute_request(collection, inputs, runner)[source]

Resolve and execute a request.

rook.pflow.execution.prepare_operation_inputs(decision, inputs)[source]

Return cleaned operation inputs without mutating caller inputs.

rook.pflow.execution.run_operation(decision, inputs, runner)[source]

Run an operation with inputs prepared from a request decision.

rook.pflow.policies module

Request decision policies.

rook.pflow.policies.has_processing_required_input(inputs)[source]

Return whether current request parameters imply changed output data.

This preserves the existing operation detection rule while keeping the hard-coded parameter names out of the main decision flow.

rook.pflow.policies.may_return_original_files(project, inputs)[source]

Return whether catalog data may be returned without processing.

rook.pflow.policies.project_returns_original_files(project)[source]

Return whether a project currently bypasses processing by policy.

rook.pflow.policies.requests_original_files(inputs)[source]

Return whether the caller explicitly requested original files.

rook.pflow.policies.requires_processing(inputs)[source]

Return whether the requested operation changes data and must run.

rook.pflow.resolver module

Resolve requests to processing-flow decisions.

rook.pflow.resolver.aligned_original_file_urls(search_result, inputs)[source]

Return aligned original file URLs, or None when processing is required.

rook.pflow.resolver.aligned_subset_decision(project, search_result, inputs)[source]

Return an original-file decision when subset bounds align with files.

rook.pflow.resolver.classify_request_source(collection)[source]

Return the request source represented by a collection argument.

rook.pflow.resolver.operation_decision(project, search_result)[source]

Return a decision that runs operation execution.

rook.pflow.resolver.original_files_decision(project, search_result, original_file_urls=None)[source]

Return a decision that bypasses operation execution.

rook.pflow.resolver.resolve_catalog_collection(source, inputs)[source]

Return the decision for a catalog collection request.

rook.pflow.resolver.resolve_project(collection)[source]

Return the project for the first collection entry.

rook.pflow.resolver.resolve_request_decision(collection, inputs)[source]

Return the decision for a request.

rook.pflow.resolver.uses_catalog(project)[source]

Return whether requests for a project should use catalog lookup.

rook.pflow.results module

Result values produced by request decisions and execution.

class rook.pflow.results.RequestResult(decision: object, output_uris: list[str])[source]

Bases: object

Result values produced by resolving and executing a request.

property dataset_sources

Return the dataset sources prepared for operation execution.

decision: object
property original_file_urls

Return original files selected by the request decision.

output_uris: list[str]
property project

Return the project resolved for the request.

property search_result

Return the catalog search result, when catalog lookup was used.

property use_original_files

Return whether processing was skipped.

rook.pflow.sources module

Request source values.

class rook.pflow.sources.CatalogCollection(collection: list[str], project: str)[source]

Bases: RequestSource

A request source that should be resolved through a project catalog.

collection: list[str]
project: str
class rook.pflow.sources.DirectDataset(collection: list[str], project: str)[source]

Bases: RequestSource

A request source that should be processed directly without catalog lookup.

collection: list[str]
project: str
class rook.pflow.sources.WorkflowFiles(files: list[str])[source]

Bases: RequestSource

Files produced by an earlier workflow step.

files: list[str]