Pattern 6 - one input, no output¶
The CWL includes:
- one input parameter of type
Directory; - there are no output parameters of type
Directory.
This corner-case scenario takes as input an acquisition, applies an algorithm and generates an output that is not a STAC Catalog.
Implementation: derive the NDVI mean taking as input a Landsat-9 acquisition.
Building the Worflow¶
In [1]:
Copied!
from eoap_cwlwrap import wrap
from eoap_cwlwrap.loader import ( load_workflow, dump_workflow )
import sys
base_url = 'https://raw.githubusercontent.com/eoap/application-package-patterns/refs/heads/main'
workflow_id = 'pattern-6'
stage_in_cwl = load_workflow(path=f"{base_url}/templates/stage-in.cwl")
workflows_cwl = load_workflow(path=f"{base_url}/cwl-workflow/{workflow_id}.cwl")
stage_out_cwl = load_workflow(path=f"{base_url}/templates/stage-out.cwl")
main_workflow = wrap(
directory_stage_in=stage_in_cwl,
workflows=workflows_cwl,
workflow_id=workflow_id,
stage_out=stage_out_cwl
)
print('------------------------------------------------------------------------')
print('BUILD SUCCESS')
print('------------------------------------------------------------------------')
dump_workflow(main_workflow, sys.stdout)
from eoap_cwlwrap import wrap
from eoap_cwlwrap.loader import ( load_workflow, dump_workflow )
import sys
base_url = 'https://raw.githubusercontent.com/eoap/application-package-patterns/refs/heads/main'
workflow_id = 'pattern-6'
stage_in_cwl = load_workflow(path=f"{base_url}/templates/stage-in.cwl")
workflows_cwl = load_workflow(path=f"{base_url}/cwl-workflow/{workflow_id}.cwl")
stage_out_cwl = load_workflow(path=f"{base_url}/templates/stage-out.cwl")
main_workflow = wrap(
directory_stage_in=stage_in_cwl,
workflows=workflows_cwl,
workflow_id=workflow_id,
stage_out=stage_out_cwl
)
print('------------------------------------------------------------------------')
print('BUILD SUCCESS')
print('------------------------------------------------------------------------')
dump_workflow(main_workflow, sys.stdout)
Loading CWL document from https://raw.githubusercontent.com/eoap/application-package-patterns/refs/heads/main/templates/stage-in.cwl... Raw CWL document successfully loaded from https://raw.githubusercontent.com/eoap/application-package-patterns/refs/heads/main/templates/stage-in.cwl! Now updating the model to v1.2... Raw CWL document successfully updated! Now converting to the CWL model...
Raw CWL document successfully updated! Now dereferencing the FQNs... Cleaning CommandLineTool my-asthonishing-stage-in-directory... CWL document successfully dereferenced! Loading CWL document from https://raw.githubusercontent.com/eoap/application-package-patterns/refs/heads/main/cwl-workflow/pattern-6.cwl... Raw CWL document successfully loaded from https://raw.githubusercontent.com/eoap/application-package-patterns/refs/heads/main/cwl-workflow/pattern-6.cwl! Now updating the model to v1.2... Raw CWL document successfully updated! Now converting to the CWL model... Raw CWL document successfully updated! Now dereferencing the FQNs... Cleaning Workflow pattern-6... Cleaning CommandLineTool clt... CWL document successfully dereferenced! Loading CWL document from https://raw.githubusercontent.com/eoap/application-package-patterns/refs/heads/main/templates/stage-out.cwl... Raw CWL document successfully loaded from https://raw.githubusercontent.com/eoap/application-package-patterns/refs/heads/main/templates/stage-out.cwl! Now updating the model to v1.2... Raw CWL document successfully updated! Now converting to the CWL model...
------------------------------------------------------------------------
BUILD SUCCESS
------------------------------------------------------------------------
cwlVersion: v1.2
$graph:
- id: main
class: Workflow
label: Workflow pattern-6 orchestrator
doc: This Workflow is used to orchestrate the Workflow pattern-6
inputs:
- id: aoi
label: area of interest - pattern-6/aoi
doc: area of interest as a bounding box - This parameter is derived from
pattern-6/aoi
type: string
- id: epsg
label: EPSG code - pattern-6/epsg
doc: EPSG code - This parameter is derived from pattern-6/epsg
default: EPSG:4326
type: string
- id: another_input
label: Another Input - my-asthonishing-stage-in-directory/another_input
doc: An additional input for demonstration purposes - This parameter is
derived from my-asthonishing-stage-in-directory/another_input
type: string
- id: item
label: Landsat-8/9 acquisition reference - pattern-6/item
doc: Landsat-8/9 acquisition reference - This parameter is derived from
pattern-6/item
type:
https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#URI
outputs:
- id: mean
label: Vegetation index mean - app/mean
doc: Vegetation index mean - This output is derived from app/mean
outputSource:
- app/mean
type: float
requirements:
- class: SubworkflowFeatureRequirement
- class: SchemaDefRequirement
types:
- $import:
https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml
steps:
- id: directory_stage_in_0
in:
- id: reference
source: item
- id: another_input
source: another_input
out:
- staged
run: '#my-asthonishing-stage-in-directory'
- id: app
in:
- id: aoi
source: aoi
- id: epsg
source: epsg
- id: item
source: directory_stage_in_0/staged
out:
- mean
run: '#pattern-6'
- http://commonwl.org/cwltool#original_cwlVersion: v1.2
id: my-asthonishing-stage-in-directory
class: CommandLineTool
inputs:
- id: reference
label: STAC Item URL
doc: A STAC Item to stage
type:
https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#URI
- id: another_input
label: Another Input
doc: An additional input for demonstration purposes
type: string
outputs:
- id: staged
type: Directory
outputBinding:
glob: .
requirements:
- class: SchemaDefRequirement
types:
- name:
https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#Date
fields:
- name:
https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#Date/value
type: string
type: record
- name:
https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#DateTime
fields:
- name:
https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#DateTime/value
type: string
type: record
- name:
https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#Duration
fields:
- name:
https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#Duration/value
type: string
type: record
- name:
https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#Email
fields:
- name:
https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#Email/value
type: string
type: record
- name:
https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#Hostname
fields:
- name:
https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#Hostname/value
type: string
type: record
- name:
https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#IDNEmail
fields:
- name:
https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#IDNEmail/value
type: string
type: record
- name:
https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#IDNHostname
fields:
- name:
https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#IDNHostname/value
type: string
type: record
- name:
https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#IPv4
fields:
- name:
https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#IPv4/value
type: string
type: record
- name:
https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#IPv6
fields:
- name:
https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#IPv6/value
type: string
type: record
- name:
https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#IRI
fields:
- name:
https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#IRI/value
type: string
type: record
- name:
https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#IRIReference
fields:
- name:
https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#IRIReference/value
type: string
type: record
- name:
https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#JsonPointer
fields:
- name:
https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#JsonPointer/value
type: string
type: record
- name:
https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#Password
fields:
- name:
https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#Password/value
type: string
type: record
- name:
https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#RelativeJsonPointer
fields:
- name:
https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#RelativeJsonPointer/value
type: string
type: record
- name:
https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#UUID
fields:
- name:
https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#UUID/value
type: string
type: record
- name:
https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#URI
fields:
- name:
https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#URI/value
type: string
type: record
- name:
https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#URIReference
fields:
- name:
https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#URIReference/value
type: string
type: record
- name:
https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#URITemplate
fields:
- name:
https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#URITemplate/value
type: string
type: record
- name:
https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#Time
fields:
- name:
https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#Time/value
type: string
type: record
- class: DockerRequirement
dockerPull: ghcr.io/eoap/mastering-app-package/stage:1.0.0
- class: InlineJavascriptRequirement
- class: InitialWorkDirRequirement
listing:
- entryname: stage.py
entry: |-
import pystac
import stac_asset
import asyncio
import os
import sys
config = stac_asset.Config(warn=True)
async def main(href: str):
item = pystac.read_file(href)
os.makedirs(item.id, exist_ok=True)
cwd = os.getcwd()
os.chdir(item.id)
item = await stac_asset.download_item(item=item, directory=".", config=config)
os.chdir(cwd)
cat = pystac.Catalog(
id="catalog",
description=f"catalog with staged {item.id}",
title=f"catalog with staged {item.id}",
)
cat.add_item(item)
cat.normalize_hrefs("./")
cat.save(catalog_type=pystac.CatalogType.SELF_CONTAINED)
return cat
href = sys.argv[1]
empty_arg = sys.argv[2]
cat = asyncio.run(main(href))
cwlVersion: v1.2
baseCommand:
- python
- stage.py
arguments:
- $( inputs.reference.value )
- $( inputs.another_input )
- id: pattern-6
class: Workflow
label: NDVI mean
doc: NDVI mean from Landsat-8/9 acquisitions
inputs:
- id: aoi
label: area of interest
doc: area of interest as a bounding box
type: string
- id: epsg
label: EPSG code
doc: EPSG code
default: EPSG:4326
type: string
- id: item
label: Landsat-8/9 acquisition reference
doc: Landsat-8/9 acquisition reference
type: Directory
outputs:
- id: mean
label: Vegetation index mean
doc: Vegetation index mean
outputSource:
- step/mean
type: float
requirements:
- class: ScatterFeatureRequirement
cwlVersion: v1.2
steps:
- id: step
in:
- id: item
source: item
- id: aoi
source: aoi
- id: epsg
source: epsg
out:
- mean
run: '#clt'
$namespaces: &id001
s: https://schema.org/
- id: clt
class: CommandLineTool
inputs:
- id: item
type: Directory
inputBinding:
prefix: --input-item
- id: aoi
type: string
inputBinding:
prefix: --aoi
- id: epsg
type: string
inputBinding:
prefix: --epsg
outputs:
- id: mean
type: float
outputBinding:
loadContents: true
glob: message
outputEval: $(parseFloat(self[0].contents))
requirements:
- class: InlineJavascriptRequirement
- class: EnvVarRequirement
envDef:
- envName: PATH
envValue: $PATH:/app/envs/runner/bin
- class: ResourceRequirement
coresMax: 1
ramMax: 512
hints:
- class: DockerRequirement
dockerPull: docker.io/library/runner:latest
cwlVersion: v1.2
baseCommand:
- runner
arguments:
- pattern-6
stdout: message
$namespaces: *id001
Raw CWL document successfully updated! Now dereferencing the FQNs... Cleaning CommandLineTool my-super-stage-out... CWL document successfully dereferenced! Validating stage-in 'my-asthonishing-stage-in-directory'... stage-in 'my-asthonishing-stage-in-directory' is valid Validating stage-out 'my-super-stage-out'... stage-out 'my-super-stage-out' is valid Building the CWL Orchestrator Workflow... Analyzing pattern-6 inputs... * pattern-6/aoi: string * pattern-6/epsg: string * pattern-6/item: Directory Directory type detected, creating a related 'directory_stage_in_0'... Converting Directory to URL-compatible type... Directory converted to https://raw.githubusercontent.com/eoap/schemas/main/string_format.yaml#URI Connecting 'app/item' to 'directory_stage_in_0' output... Analyzing pattern-6 outputs... * pattern-6/mean: float Orchestrator Workflow built in 0.0010 seconds
