Pattern 12 - one input, three outputs of different kind¶
The CWL includes:
- one input parameter type
Directory
; - output parameters:
- two of type
Directory
; - one of type
Directory[]
.
- two of type
This scenario takes as input an array of acquisition, applies an algorithm to each of them.
This scenario shows the need to flatten the output of the subworkflow, which is an array of arrays, into a single array of directories.
Building the Worflow¶
In [1]:
Copied!
from eoap_cwlwrap import wrap
from eoap_cwlwrap.loader import ( load_workflow, dump_workflow )
import sys
base_url = 'https://raw.githubusercontent.com/eoap/application-package-patterns/refs/heads/main'
workflow_id = 'pattern-12'
directory_stage_in = load_workflow(path=f"{base_url}/templates/stage-in.cwl")
file_stage_in = load_workflow(path=f"{base_url}/templates/stage-in-file.cwl")
workflows_cwl = load_workflow(path=f"{base_url}/cwl-workflow/{workflow_id}.cwl")
stage_out_cwl = load_workflow(path=f"{base_url}/templates/stage-out.cwl")
main_workflow = wrap(
directory_stage_in=directory_stage_in,
file_stage_in=file_stage_in,
workflows=workflows_cwl,
workflow_id=workflow_id,
stage_out=stage_out_cwl
)
print('------------------------------------------------------------------------')
print('BUILD SUCCESS')
print('------------------------------------------------------------------------')
dump_workflow(main_workflow, sys.stdout)
from eoap_cwlwrap import wrap
from eoap_cwlwrap.loader import ( load_workflow, dump_workflow )
import sys
base_url = 'https://raw.githubusercontent.com/eoap/application-package-patterns/refs/heads/main'
workflow_id = 'pattern-12'
directory_stage_in = load_workflow(path=f"{base_url}/templates/stage-in.cwl")
file_stage_in = load_workflow(path=f"{base_url}/templates/stage-in-file.cwl")
workflows_cwl = load_workflow(path=f"{base_url}/cwl-workflow/{workflow_id}.cwl")
stage_out_cwl = load_workflow(path=f"{base_url}/templates/stage-out.cwl")
main_workflow = wrap(
directory_stage_in=directory_stage_in,
file_stage_in=file_stage_in,
workflows=workflows_cwl,
workflow_id=workflow_id,
stage_out=stage_out_cwl
)
print('------------------------------------------------------------------------')
print('BUILD SUCCESS')
print('------------------------------------------------------------------------')
dump_workflow(main_workflow, sys.stdout)
Loading CWL document from https://raw.githubusercontent.com/eoap/application-package-patterns/refs/heads/main/templates/stage-in.cwl... Raw CWL document successfully loaded from https://raw.githubusercontent.com/eoap/application-package-patterns/refs/heads/main/templates/stage-in.cwl! Now updating the model to v1.2... Raw CWL document successfully updated! Now converting to the CWL model...
Raw CWL document successfully updated! Now dereferencing the FQNs... Cleaning CommandLineTool my-asthonishing-stage-in-directory... CWL document successfully dereferenced! Loading CWL document from https://raw.githubusercontent.com/eoap/application-package-patterns/refs/heads/main/templates/stage-in-file.cwl... Raw CWL document successfully loaded from https://raw.githubusercontent.com/eoap/application-package-patterns/refs/heads/main/templates/stage-in-file.cwl! Now updating the model to v1.2... Raw CWL document successfully updated! Now converting to the CWL model...
Raw CWL document successfully updated! Now dereferencing the FQNs... Cleaning CommandLineTool my-asthonishing-stage-in-file... CWL document successfully dereferenced! Loading CWL document from https://raw.githubusercontent.com/eoap/application-package-patterns/refs/heads/main/cwl-workflow/pattern-12.cwl... Raw CWL document successfully loaded from https://raw.githubusercontent.com/eoap/application-package-patterns/refs/heads/main/cwl-workflow/pattern-12.cwl! Now updating the model to v1.2... Raw CWL document successfully updated! Now converting to the CWL model...
Raw CWL document successfully updated! Now dereferencing the FQNs... Cleaning Workflow pattern-12... Cleaning CommandLineTool crop... Cleaning CommandLineTool norm_diff... Cleaning CommandLineTool otsu... CWL document successfully dereferenced! Loading CWL document from https://raw.githubusercontent.com/eoap/application-package-patterns/refs/heads/main/templates/stage-out.cwl... Raw CWL document successfully loaded from https://raw.githubusercontent.com/eoap/application-package-patterns/refs/heads/main/templates/stage-out.cwl! Now updating the model to v1.2... Raw CWL document successfully updated! Now converting to the CWL model...
------------------------------------------------------------------------ BUILD SUCCESS ------------------------------------------------------------------------ cwlVersion: v1.2 $graph: - id: main class: Workflow label: Workflow pattern-12 orchestrator doc: This Workflow is used to orchestrate the Workflow pattern-12 inputs: - id: aoi label: area of interest - pattern-12/aoi doc: area of interest as a bounding box - This parameter is derived from pattern-12/aoi type: https://raw.githubusercontent.com/eoap/schemas/main/ogc.yaml#BBox - id: bands label: bands used for the NDWI - pattern-12/bands doc: bands used for the NDWI - This parameter is derived from pattern-12/bands default: - green - nir type: name: _:b7fa8882-6bd2-464b-9990-e97ec8a2f7cf items: string type: array - id: another_input label: Another Input - my-asthonishing-stage-in-directory/another_input doc: An additional input for demonstration purposes - This parameter is derived from my-asthonishing-stage-in-directory/another_input type: string - id: item label: Landsat-8/9 acquisition reference - pattern-12/item doc: Landsat-8/9 acquisition reference - This parameter is derived from pattern-12/item type: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#URI - id: cropped-collection label: cropped reflectances STAC Collection - pattern-12/cropped-collection doc: STAC Collection URL for the cropped reflectances - This parameter is derived from pattern-12/cropped-collection type: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#URI - id: ndwi-collection label: NDWI STAC Collection - pattern-12/ndwi-collection doc: STAC Collection URL for the NDWI - This parameter is derived from pattern-12/ndwi-collection type: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#URI - id: water-bodies-collection label: Water bodies STAC Collection - pattern-12/water-bodies-collection doc: STAC Collection URL for the water bodies - This parameter is derived from pattern-12/water-bodies-collection type: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#URI - id: s3_bucket label: my-super-stage-out/s3_bucket doc: 'This parameter is derived from: my-super-stage-out/s3_bucket' type: string - id: sub_path label: my-super-stage-out/sub_path doc: 'This parameter is derived from: my-super-stage-out/sub_path' type: string - id: aws_access_key_id label: my-super-stage-out/aws_access_key_id doc: 'This parameter is derived from: my-super-stage-out/aws_access_key_id' type: string - id: aws_secret_access_key label: my-super-stage-out/aws_secret_access_key doc: 'This parameter is derived from: my-super-stage-out/aws_secret_access_key' type: string - id: region_name label: my-super-stage-out/region_name doc: 'This parameter is derived from: my-super-stage-out/region_name' type: string - id: endpoint_url label: my-super-stage-out/endpoint_url doc: 'This parameter is derived from: my-super-stage-out/endpoint_url' type: string outputs: - id: cropped label: Cropped reflectances doc: Cropped reflectances outputSource: - stage_out_0/s3_catalog_output type: name: _:9dc94164-5b17-4c19-8fa6-aeae5cd507a8 items: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#URI type: array - id: ndwi label: Normalized Difference Water Index doc: Normalized Difference Water Index calculated from the input bands outputSource: - stage_out_1/s3_catalog_output type: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#URI - id: water_bodies label: Water bodies detected doc: Water bodies detected based on the NDWI and otsu threshold outputSource: - stage_out_2/s3_catalog_output type: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#URI requirements: - class: SubworkflowFeatureRequirement - class: ScatterFeatureRequirement - class: SchemaDefRequirement types: - $import: https://raw.githubusercontent.com/eoap/schemas/main/ogc.yaml - $import: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml steps: - id: directory_stage_in_0 in: - id: reference source: item - id: another_input source: another_input out: - staged run: '#my-asthonishing-stage-in-directory' - id: app in: - id: aoi source: aoi - id: bands source: bands - id: item source: directory_stage_in_0/staged - id: cropped-collection source: cropped-collection - id: ndwi-collection source: ndwi-collection - id: water-bodies-collection source: water-bodies-collection out: - cropped - ndwi - water_bodies run: '#pattern-12' - id: stage_out_0 in: - id: s3_bucket source: s3_bucket - id: sub_path source: sub_path - id: aws_access_key_id source: aws_access_key_id - id: aws_secret_access_key source: aws_secret_access_key - id: region_name source: region_name - id: endpoint_url source: endpoint_url - id: stac_catalog source: app/cropped out: - s3_catalog_output run: '#my-super-stage-out' scatter: stac_catalog scatterMethod: dotproduct - id: stage_out_1 in: - id: s3_bucket source: s3_bucket - id: sub_path source: sub_path - id: aws_access_key_id source: aws_access_key_id - id: aws_secret_access_key source: aws_secret_access_key - id: region_name source: region_name - id: endpoint_url source: endpoint_url - id: stac_catalog source: app/ndwi out: - s3_catalog_output run: '#my-super-stage-out' - id: stage_out_2 in: - id: s3_bucket source: s3_bucket - id: sub_path source: sub_path - id: aws_access_key_id source: aws_access_key_id - id: aws_secret_access_key source: aws_secret_access_key - id: region_name source: region_name - id: endpoint_url source: endpoint_url - id: stac_catalog source: app/water_bodies out: - s3_catalog_output run: '#my-super-stage-out' - http://commonwl.org/cwltool#original_cwlVersion: v1.2 id: my-asthonishing-stage-in-directory class: CommandLineTool inputs: - id: reference label: STAC Item URL doc: A STAC Item to stage type: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#URI - id: another_input label: Another Input doc: An additional input for demonstration purposes type: string outputs: - id: staged type: Directory outputBinding: glob: . requirements: - class: SchemaDefRequirement types: - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#Date fields: - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#Date/value type: string type: record - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#DateTime fields: - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#DateTime/value type: string type: record - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#Duration fields: - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#Duration/value type: string type: record - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#Email fields: - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#Email/value type: string type: record - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#Hostname fields: - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#Hostname/value type: string type: record - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#IDNEmail fields: - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#IDNEmail/value type: string type: record - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#IDNHostname fields: - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#IDNHostname/value type: string type: record - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#IPv4 fields: - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#IPv4/value type: string type: record - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#IPv6 fields: - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#IPv6/value type: string type: record - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#IRI fields: - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#IRI/value type: string type: record - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#IRIReference fields: - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#IRIReference/value type: string type: record - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#JsonPointer fields: - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#JsonPointer/value type: string type: record - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#Password fields: - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#Password/value type: string type: record - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#RelativeJsonPointer fields: - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#RelativeJsonPointer/value type: string type: record - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#UUID fields: - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#UUID/value type: string type: record - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#URI fields: - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#URI/value type: string type: record - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#URIReference fields: - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#URIReference/value type: string type: record - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#URITemplate fields: - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#URITemplate/value type: string type: record - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#Time fields: - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#Time/value type: string type: record - class: DockerRequirement dockerPull: ghcr.io/eoap/mastering-app-package/stage:1.0.0 - class: InlineJavascriptRequirement - class: InitialWorkDirRequirement listing: - entryname: stage.py entry: |- import pystac import stac_asset import asyncio import os import sys config = stac_asset.Config(warn=True) async def main(href: str): item = pystac.read_file(href) os.makedirs(item.id, exist_ok=True) cwd = os.getcwd() os.chdir(item.id) item = await stac_asset.download_item(item=item, directory=".", config=config) os.chdir(cwd) cat = pystac.Catalog( id="catalog", description=f"catalog with staged {item.id}", title=f"catalog with staged {item.id}", ) cat.add_item(item) cat.normalize_hrefs("./") cat.save(catalog_type=pystac.CatalogType.SELF_CONTAINED) return cat href = sys.argv[1] empty_arg = sys.argv[2] cat = asyncio.run(main(href)) cwlVersion: v1.2 baseCommand: - python - stage.py arguments: - $( inputs.reference.value ) - $( inputs.another_input ) - id: pattern-12 class: Workflow label: Water body detection based on NDWI and the otsu threshold doc: Water bodies detection based on NDWI and otsu threshold applied to Sentinel-2 or Landsat-9 staged acquisitions inputs: - id: aoi label: area of interest doc: area of interest as a bounding box type: https://raw.githubusercontent.com/eoap/schemas/main/ogc.yaml#BBox - id: bands label: bands used for the NDWI doc: bands used for the NDWI default: - green - nir type: name: _:b7fa8882-6bd2-464b-9990-e97ec8a2f7cf items: string type: array - id: item label: Landsat-8/9 acquisition reference doc: Landsat-8/9 acquisition reference type: Directory - id: cropped-collection label: cropped reflectances STAC Collection doc: STAC Collection URL for the cropped reflectances type: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#URI - id: ndwi-collection label: NDWI STAC Collection doc: STAC Collection URL for the NDWI type: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#URI - id: water-bodies-collection label: Water bodies STAC Collection doc: STAC Collection URL for the water bodies type: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#URI outputs: - id: cropped label: Cropped reflectances doc: Cropped reflectances outputSource: - node_crop/cropped type: name: _:adcb6c13-4094-494f-9c8c-a18d7c7c6a09 items: Directory type: array - id: ndwi label: Normalized Difference Water Index doc: Normalized Difference Water Index calculated from the input bands outputSource: - node_normalized_difference/ndwi type: Directory - id: water_bodies label: Water bodies detected doc: Water bodies detected based on the NDWI and otsu threshold outputSource: - node_otsu/water_bodies type: Directory requirements: - class: ScatterFeatureRequirement - class: SchemaDefRequirement types: - name: https://raw.githubusercontent.com/eoap/schemas/main/ogc.yaml#CRS symbols: - https://raw.githubusercontent.com/eoap/schemas/main/ogc.yaml#CRS/CRS84 - https://raw.githubusercontent.com/eoap/schemas/main/ogc.yaml#CRS/CRS84h type: enum - name: https://raw.githubusercontent.com/eoap/schemas/main/ogc.yaml#BBox fields: - name: https://raw.githubusercontent.com/eoap/schemas/main/ogc.yaml#BBox/bbox type: name: _:5f695a5a-372e-4cde-8309-a383aa119349 items: double type: array - name: https://raw.githubusercontent.com/eoap/schemas/main/ogc.yaml#BBox/crs type: https://raw.githubusercontent.com/eoap/schemas/main/ogc.yaml#CRS type: record - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#Date fields: - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#Date/value type: string type: record - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#DateTime fields: - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#DateTime/value type: string type: record - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#Duration fields: - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#Duration/value type: string type: record - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#Email fields: - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#Email/value type: string type: record - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#Hostname fields: - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#Hostname/value type: string type: record - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#IDNEmail fields: - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#IDNEmail/value type: string type: record - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#IDNHostname fields: - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#IDNHostname/value type: string type: record - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#IPv4 fields: - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#IPv4/value type: string type: record - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#IPv6 fields: - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#IPv6/value type: string type: record - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#IRI fields: - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#IRI/value type: string type: record - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#IRIReference fields: - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#IRIReference/value type: string type: record - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#JsonPointer fields: - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#JsonPointer/value type: string type: record - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#Password fields: - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#Password/value type: string type: record - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#RelativeJsonPointer fields: - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#RelativeJsonPointer/value type: string type: record - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#UUID fields: - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#UUID/value type: string type: record - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#URI fields: - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#URI/value type: string type: record - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#URIReference fields: - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#URIReference/value type: string type: record - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#URITemplate fields: - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#URITemplate/value type: string type: record - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#Time fields: - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#Time/value type: string type: record cwlVersion: v1.2 steps: - id: node_crop in: - id: item source: item - id: aoi source: aoi - id: epsg source: aoi - id: band source: bands - id: collection source: cropped-collection out: - cropped run: '#crop' scatter: node_crop/band scatterMethod: dotproduct - id: node_normalized_difference in: - id: rasters source: node_crop/cropped - id: item source: item - id: collection source: ndwi-collection out: - ndwi run: '#norm_diff' - id: node_otsu in: - id: raster source: node_normalized_difference/ndwi - id: item source: item - id: collection source: water-bodies-collection out: - water_bodies run: '#otsu' $namespaces: &id001 s: https://schema.org/ - http://commonwl.org/cwltool#original_cwlVersion: v1.2 id: my-super-stage-out class: CommandLineTool doc: Stage-out the results to S3 inputs: - id: s3_bucket type: string - id: sub_path type: string - id: aws_access_key_id type: string - id: aws_secret_access_key type: string - id: region_name type: string - id: endpoint_url type: string - id: stac_catalog label: STAC Catalog folder doc: The folder containing the STAC catalog to stage out type: Directory outputs: - id: s3_catalog_output type: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#URI outputBinding: loadContents: true glob: catalog-uri.txt outputEval: | ${ return { "value": self[0].contents, "type": "https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#URI" }; } requirements: - class: SchemaDefRequirement types: - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#Date fields: - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#Date/value type: string type: record - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#DateTime fields: - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#DateTime/value type: string type: record - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#Duration fields: - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#Duration/value type: string type: record - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#Email fields: - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#Email/value type: string type: record - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#Hostname fields: - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#Hostname/value type: string type: record - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#IDNEmail fields: - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#IDNEmail/value type: string type: record - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#IDNHostname fields: - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#IDNHostname/value type: string type: record - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#IPv4 fields: - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#IPv4/value type: string type: record - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#IPv6 fields: - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#IPv6/value type: string type: record - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#IRI fields: - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#IRI/value type: string type: record - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#IRIReference fields: - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#IRIReference/value type: string type: record - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#JsonPointer fields: - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#JsonPointer/value type: string type: record - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#Password fields: - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#Password/value type: string type: record - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#RelativeJsonPointer fields: - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#RelativeJsonPointer/value type: string type: record - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#UUID fields: - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#UUID/value type: string type: record - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#URI fields: - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#URI/value type: string type: record - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#URIReference fields: - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#URIReference/value type: string type: record - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#URITemplate fields: - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#URITemplate/value type: string type: record - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#Time fields: - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#Time/value type: string type: record - class: DockerRequirement dockerPull: ghcr.io/eoap/mastering-app-package/stage:1.0.0 - class: InlineJavascriptRequirement - class: EnvVarRequirement envDef: - envName: aws_access_key_id envValue: $( inputs.aws_access_key_id ) - envName: aws_secret_access_key envValue: $( inputs.aws_secret_access_key ) - envName: aws_region_name envValue: $( inputs.region_name ) - envName: aws_endpoint_url envValue: $( inputs.endpoint_url ) - class: ResourceRequirement - class: InitialWorkDirRequirement listing: - entryname: stage.py entry: |- import os import sys import pystac import botocore import boto3 import shutil from pystac.stac_io import DefaultStacIO, StacIO from urllib.parse import urlparse cat_url = sys.argv[1] bucket = sys.argv[2] subfolder = sys.argv[3] aws_access_key_id = os.environ["aws_access_key_id"] aws_secret_access_key = os.environ["aws_secret_access_key"] region_name = os.environ["aws_region_name"] endpoint_url = os.environ["aws_endpoint_url"] shutil.copytree(cat_url, "/tmp/catalog") cat = pystac.read_file(os.path.join("/tmp/catalog", "catalog.json")) class CustomStacIO(DefaultStacIO): """Custom STAC IO class that uses boto3 to read from S3.""" def __init__(self): self.session = botocore.session.Session() self.s3_client = self.session.create_client( service_name="s3", use_ssl=True, aws_access_key_id=aws_access_key_id, aws_secret_access_key=aws_secret_access_key, endpoint_url=endpoint_url, region_name=region_name, ) def write_text(self, dest, txt, *args, **kwargs): parsed = urlparse(dest) if parsed.scheme == "s3": self.s3_client.put_object( Body=txt.encode("UTF-8"), Bucket=parsed.netloc, Key=parsed.path[1:], ContentType="application/geo+json", ) else: super().write_text(dest, txt, *args, **kwargs) client = boto3.client( "s3", aws_access_key_id=aws_access_key_id, aws_secret_access_key=aws_secret_access_key, endpoint_url=endpoint_url, region_name=region_name, ) StacIO.set_default(CustomStacIO) for item in cat.get_items(): for key, asset in item.get_assets().items(): s3_path = os.path.normpath( os.path.join(os.path.join(subfolder, item.id, asset.href)) ) print(f"upload {asset.href} to s3://{bucket}/{s3_path}",file=sys.stderr) client.upload_file( asset.get_absolute_href(), bucket, s3_path, ) asset.href = f"s3://{bucket}/{s3_path}" item.add_asset(key, asset) cat.normalize_hrefs(f"s3://{bucket}/{subfolder}") for item in cat.get_items(): # upload item to S3 print(f"upload {item.id} to s3://{bucket}/{subfolder}", file=sys.stderr) pystac.write_file(item, item.get_self_href()) # upload catalog to S3 print(f"upload catalog.json to s3://{bucket}/{subfolder}", file=sys.stderr) pystac.write_file(cat, cat.get_self_href()) print(f"s3://{bucket}/{subfolder}/catalog.json", end="", file=sys.stdout) cwlVersion: v1.2 baseCommand: - python - stage.py arguments: - $( inputs.stac_catalog.path ) - $( inputs.s3_bucket ) - ${ var firstPart = (Math.random() * 46656) | 0; var secondPart = (Math.random() * 46656) | 0; firstPart = ("000" + firstPart.toString(36)).slice(-3); secondPart = ("000" + secondPart.toString(36)).slice(-3); return inputs.sub_path + "-" + firstPart + secondPart; } stdout: catalog-uri.txt - id: crop class: CommandLineTool inputs: - id: item type: Directory inputBinding: prefix: --input-item - id: aoi label: Area of interest doc: Area of interest defined as a bounding box type: https://raw.githubusercontent.com/eoap/schemas/main/ogc.yaml#BBox inputBinding: valueFrom: | ${ // Validate the length of bbox to be either 4 or 6 var bboxLength = inputs.aoi.bbox.length; if (bboxLength !== 4 && bboxLength !== 6) { throw "Invalid bbox length: bbox must have either 4 or 6 elements."; } // Convert bbox array to a space-separated string for echo return ["--aoi", inputs.aoi.bbox.join(",")]; } - id: epsg type: https://raw.githubusercontent.com/eoap/schemas/main/ogc.yaml#BBox inputBinding: valueFrom: | ${ const crs = inputs.epsg.crs; if (typeof crs !== "string") { throw "Invalid CRS: must be a string."; } if (["CRS84"].includes(crs)) { return ["--epsg", "EPSG:4326"]; } else { throw "Unsupported CRS: only CRS84 is currently supported."; } } - id: band type: string inputBinding: prefix: --band - id: collection type: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#URI inputBinding: valueFrom: | ${ // parse the URI provided in the input var product_uri = inputs.collection.value; // Validate the URI format var uriPattern = /^(https?|ftp):\/\/[^\s/$.?#].[^\s]*$/i; if (!uriPattern.test(product_uri)) { throw "Invalid URI format: " + product_uri; } // Return the URI as a string return ["--collection", product_uri]; } outputs: - id: cropped type: Directory outputBinding: glob: . requirements: - class: InlineJavascriptRequirement - class: EnvVarRequirement envDef: - envName: PATH envValue: /app/envs/runner/bin - class: ResourceRequirement coresMax: 1 ramMax: 512 - class: SchemaDefRequirement types: - name: https://raw.githubusercontent.com/eoap/schemas/main/ogc.yaml#CRS symbols: - https://raw.githubusercontent.com/eoap/schemas/main/ogc.yaml#CRS/CRS84 - https://raw.githubusercontent.com/eoap/schemas/main/ogc.yaml#CRS/CRS84h type: enum - name: https://raw.githubusercontent.com/eoap/schemas/main/ogc.yaml#BBox fields: - name: https://raw.githubusercontent.com/eoap/schemas/main/ogc.yaml#BBox/bbox type: name: _:5f695a5a-372e-4cde-8309-a383aa119349 items: double type: array - name: https://raw.githubusercontent.com/eoap/schemas/main/ogc.yaml#BBox/crs type: https://raw.githubusercontent.com/eoap/schemas/main/ogc.yaml#CRS type: record - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#Date fields: - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#Date/value type: string type: record - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#DateTime fields: - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#DateTime/value type: string type: record - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#Duration fields: - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#Duration/value type: string type: record - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#Email fields: - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#Email/value type: string type: record - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#Hostname fields: - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#Hostname/value type: string type: record - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#IDNEmail fields: - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#IDNEmail/value type: string type: record - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#IDNHostname fields: - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#IDNHostname/value type: string type: record - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#IPv4 fields: - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#IPv4/value type: string type: record - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#IPv6 fields: - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#IPv6/value type: string type: record - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#IRI fields: - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#IRI/value type: string type: record - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#IRIReference fields: - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#IRIReference/value type: string type: record - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#JsonPointer fields: - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#JsonPointer/value type: string type: record - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#Password fields: - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#Password/value type: string type: record - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#RelativeJsonPointer fields: - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#RelativeJsonPointer/value type: string type: record - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#UUID fields: - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#UUID/value type: string type: record - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#URI fields: - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#URI/value type: string type: record - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#URIReference fields: - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#URIReference/value type: string type: record - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#URITemplate fields: - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#URITemplate/value type: string type: record - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#Time fields: - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#Time/value type: string type: record - class: NetworkAccess networkAccess: true hints: - class: DockerRequirement dockerPull: ghcr.io/eoap/application-package-patterns/runner:0.2.0 cwlVersion: v1.2 baseCommand: - runner arguments: - crop-cli $namespaces: *id001 - id: norm_diff class: CommandLineTool inputs: - id: rasters type: name: _:8321b636-f22a-4917-b557-72c8dd7b7b63 items: Directory type: array inputBinding: valueFrom: | ${ var args = []; for (var i = 0; i < inputs.rasters.length; i++) { args.push(`--item-${i + 1}`); args.push(inputs.rasters[i].path); } return args; } - id: item type: Directory inputBinding: prefix: --item - id: collection type: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#URI inputBinding: valueFrom: | ${ // parse the URI provided in the input var product_uri = inputs.collection.value; // Validate the URI format var uriPattern = /^(https?|ftp):\/\/[^\s/$.?#].[^\s]*$/i; if (!uriPattern.test(product_uri)) { throw "Invalid URI format: " + product_uri; } // Return the URI as a string return ["--collection", product_uri]; } outputs: - id: ndwi type: Directory outputBinding: glob: . requirements: - class: InlineJavascriptRequirement - class: EnvVarRequirement envDef: - envName: PATH envValue: /app/envs/runner/bin - class: ResourceRequirement coresMax: 1 ramMax: 512 - class: SchemaDefRequirement types: - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#Date fields: - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#Date/value type: string type: record - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#DateTime fields: - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#DateTime/value type: string type: record - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#Duration fields: - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#Duration/value type: string type: record - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#Email fields: - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#Email/value type: string type: record - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#Hostname fields: - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#Hostname/value type: string type: record - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#IDNEmail fields: - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#IDNEmail/value type: string type: record - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#IDNHostname fields: - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#IDNHostname/value type: string type: record - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#IPv4 fields: - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#IPv4/value type: string type: record - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#IPv6 fields: - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#IPv6/value type: string type: record - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#IRI fields: - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#IRI/value type: string type: record - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#IRIReference fields: - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#IRIReference/value type: string type: record - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#JsonPointer fields: - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#JsonPointer/value type: string type: record - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#Password fields: - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#Password/value type: string type: record - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#RelativeJsonPointer fields: - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#RelativeJsonPointer/value type: string type: record - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#UUID fields: - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#UUID/value type: string type: record - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#URI fields: - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#URI/value type: string type: record - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#URIReference fields: - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#URIReference/value type: string type: record - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#URITemplate fields: - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#URITemplate/value type: string type: record - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#Time fields: - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#Time/value type: string type: record - class: NetworkAccess networkAccess: true hints: - class: DockerRequirement dockerPull: ghcr.io/eoap/application-package-patterns/runner:0.2.0 cwlVersion: v1.2 baseCommand: - runner arguments: - ndi-cli $namespaces: *id001 - id: otsu class: CommandLineTool inputs: - id: raster type: Directory inputBinding: prefix: --input-ndi - id: item type: Directory inputBinding: prefix: --item - id: collection type: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#URI inputBinding: valueFrom: | ${ // parse the URI provided in the input var product_uri = inputs.collection.value; // Validate the URI format var uriPattern = /^(https?|ftp):\/\/[^\s/$.?#].[^\s]*$/i; if (!uriPattern.test(product_uri)) { throw "Invalid URI format: " + product_uri; } // Return the URI as a string return ["--collection", product_uri]; } outputs: - id: water_bodies type: Directory outputBinding: glob: . requirements: - class: InlineJavascriptRequirement - class: EnvVarRequirement envDef: - envName: PATH envValue: /app/envs/runner/bin - class: ResourceRequirement coresMax: 1 ramMax: 512 - class: SchemaDefRequirement types: - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#Date fields: - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#Date/value type: string type: record - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#DateTime fields: - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#DateTime/value type: string type: record - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#Duration fields: - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#Duration/value type: string type: record - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#Email fields: - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#Email/value type: string type: record - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#Hostname fields: - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#Hostname/value type: string type: record - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#IDNEmail fields: - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#IDNEmail/value type: string type: record - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#IDNHostname fields: - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#IDNHostname/value type: string type: record - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#IPv4 fields: - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#IPv4/value type: string type: record - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#IPv6 fields: - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#IPv6/value type: string type: record - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#IRI fields: - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#IRI/value type: string type: record - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#IRIReference fields: - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#IRIReference/value type: string type: record - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#JsonPointer fields: - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#JsonPointer/value type: string type: record - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#Password fields: - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#Password/value type: string type: record - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#RelativeJsonPointer fields: - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#RelativeJsonPointer/value type: string type: record - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#UUID fields: - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#UUID/value type: string type: record - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#URI fields: - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#URI/value type: string type: record - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#URIReference fields: - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#URIReference/value type: string type: record - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#URITemplate fields: - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#URITemplate/value type: string type: record - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#Time fields: - name: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#Time/value type: string type: record - class: NetworkAccess networkAccess: true hints: - class: DockerRequirement dockerPull: ghcr.io/eoap/application-package-patterns/runner:0.2.0 cwlVersion: v1.2 baseCommand: - runner arguments: - otsu-cli $namespaces: *id001
Raw CWL document successfully updated! Now dereferencing the FQNs... Cleaning CommandLineTool my-super-stage-out... CWL document successfully dereferenced! Validating stage-in 'my-asthonishing-stage-in-directory'... stage-in 'my-asthonishing-stage-in-directory' is valid Validating stage-in 'my-asthonishing-stage-in-file'... stage-in 'my-asthonishing-stage-in-file' is valid Validating stage-out 'my-super-stage-out'... stage-out 'my-super-stage-out' is valid Building the CWL Orchestrator Workflow... Analyzing pattern-12 inputs... * pattern-12/aoi: https://raw.githubusercontent.com/eoap/schemas/main/ogc.yaml#BBox * pattern-12/bands: string[] * pattern-12/item: Directory Directory type detected, creating a related 'directory_stage_in_0'... Converting Directory to URL-compatible type... Directory converted to https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#URI Connecting 'app/item' to 'directory_stage_in_0' output... * pattern-12/cropped-collection: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#URI * pattern-12/ndwi-collection: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#URI * pattern-12/water-bodies-collection: https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#URI Analyzing pattern-12 outputs... * pattern-12/cropped: Directory[] Directory type detected, creating a related 'stage_out_0'... Converting Directory[] to URL-compatible type... Directory[] converted to https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#URI[] Array detected, scatter required for stac_catalog:app/cropped Connecting 'app/cropped' to 'stage_out_0' output... * pattern-12/ndwi: Directory Directory type detected, creating a related 'stage_out_1'... Converting Directory to URL-compatible type... Directory converted to https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#URI Connecting 'app/ndwi' to 'stage_out_1' output... * pattern-12/water_bodies: Directory Directory type detected, creating a related 'stage_out_2'... Converting Directory to URL-compatible type... Directory converted to https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#URI Connecting 'app/water_bodies' to 'stage_out_2' output... Orchestrator Workflow built in 0.0035 seconds