EO Exploitation Platform Common Architecture
License Manager Design Document
EOEPCA.SDD.xxx
COMMENTS and ISSUES |
PDF |
EUROPEAN SPACE AGENCY CONTRACT REPORT |
TELESPAZIO VEGA UK Ltd |
- AMENDMENT HISTORY
-
This document shall be amended by releasing a new edition of the document in its entirety.
The Amendment Record Sheet below records the history and issue status of this document.Table 1. Amendment Record Sheet ISSUE DATE REASON 0.1
dd/mm/yyyy
Initial in-progress draft
1. Introduction
1.1. Purpose and Scope
This document presents the License Manager Design for the Common Architecture.
1.2. Structure of the Document
1.3. Reference Documents
The following is a list of Reference Documents with a direct bearing on the content of this document.
Reference | Document Details | Version |
---|---|---|
EOEPCA - Use Case Analysis |
Issue 1.0, |
|
Exploitation Platform - Functional Model, |
Issue 1.0, |
|
Thematic Exploitation Platform Open Architecture, |
Issue 1, |
|
OGC Testbed-14: WPS-T Engineering Report, |
18-036r1, |
|
OGC WPS 2.0 REST/JSON Binding Extension, Draft, |
1.0-draft |
|
Common Workflow Language Specifications, |
v1.0.2 |
|
OGC Testbed-13, EP Application Package Engineering Report, |
17-023, |
|
OGC Testbed-13, Application Deployment and Execution Service Engineering Report, |
17-024, |
|
OGC Testbed-14, Application Package Engineering Report, |
18-049r1, |
|
OGC Testbed-14, ADES & EMS Results and Best Practices Engineering Report, |
18-050r1, |
|
OpenSearch GEO: OpenSearch Geo and Time Extensions, |
10-032r8, |
|
OpenSearch EO: OGC OpenSearch Extension for Earth Observation, |
13-026r9, |
|
OGC EO Dataset Metadata GeoJSON(-LD) Encoding Standard, |
17-003r1/17-084 |
|
OGC OpenSearch-EO GeoJSON(-LD) Response Encoding Standard, |
17-047 |
|
The Payment Card Industry Data Security Standard, |
v3.2.1 |
|
CEOS OpenSearch Best Practise, |
v1.2, |
|
OpenID Connect Core 1.0, |
v1.0, |
|
OGC Catalogue Services 3.0 Specification - HTTP Protocol Binding (Catalogue Services for the Web), |
v3.0, |
|
OGC Web Map Server Implementation Specification, |
v1.3.0, |
|
OGC Web Map Tile Service Implementation Standard, |
v1.0.0, |
|
OGC Web Feature Service 2.0 Interface Standard – With Corrigendum, |
v2.0.2, |
|
OGC Web Coverage Service (WCS) 2.1 Interface Standard - Core, |
v2.1, |
|
Web Coverage Processing Service (WCPS) Language Interface Standard, |
v1.0.0, |
|
Amazon Simple Storage Service REST API, |
API Version 2006-03-01 |
1.4. Terminology
The following terms are used in the Master System Design.
Term | Meaning |
---|---|
Admin |
User with administrative capability on the EP |
Algorithm |
A self-contained set of operations to be performed, typically to achieve a desired data manipulation. The algorithm must be implemented (codified) for deployment and execution on the platform. |
Analysis Result |
The Products produced as output of an Interactive Application analysis session. |
Analytics |
A set of activities aimed to discover, interpret and communicate meaningful patters within the data. Analytics considered here are performed manually (or in a semi-automatic way) on-line with the aid of Interactive Applications. |
Application Artefact |
The 'software' component that provides the execution unit of the Application Package. |
Application Deployment and Execution Service (ADES) |
WPS-T (REST/JSON) service that incorporates the Docker execution engine, and is responsible for the execution of the processing service (as a WPS request) within the ‘target’ Exploitation Platform. |
Application Descriptor |
A file that provides the metadata part of the Application Package. Provides all the metadata required to accommodate the processor within the WPS service and make it available for execution. |
Application Package |
A platform independent and self-contained representation of a software item, providing executable, metadata and dependencies such that it can be deployed to and executed within an Exploitation Platform. Comprises the Application Descriptor and the Application Artefact. |
Bulk Processing |
Execution of a Processing Service on large amounts of data specified by AOI and TOI. |
Code |
The codification of an algorithm performed with a given programming language - compiled to Software or directly executed (interpretted) within the platform. |
Compute Platform |
The Platform on which execution occurs (this may differ from the Host or Home platform where federated processing is happening) |
Consumer |
User accessing existing services/products within the EP. Consumers may be scientific/research or commercial, and may or may not be experts of the domain |
Data Access Library |
An abstraction of the interface to the data layer of the resource tier. The library provides bindings for common languages (including python, Javascript) and presents a common object model to the code. |
Development |
The act of building new products/services/applications to be exposed within the platform and made available for users to conduct exploitation activities. Development may be performed inside or outside of the platform. If performed outside, an integration activity will be required to accommodate the developed service so that it is exposed within the platform. |
Discovery |
User finds products/services of interest to them based upon search criteria. |
Execution |
The act to start a Processing Service or an Interactive Application. |
Execution Management Service (EMS) |
The EMS is responsible for the orchestration of workflows, including the possibility of steps running on other (remote) platforms, and the on-demand deployment of processors to local/remote ADES as required. |
Expert |
User developing and integrating added-value to the EP (Scientific Researcher or Service Developer) |
Exploitation Tier |
The Exploitation Tier represents the end-users who exploit the services of the platform to perform analysis, or using high-level applications built-in on top of the platform’s services |
External Application |
An application or script that is developed and executed outside of the Exploitation Platform, but is able to use the data/services of the EP via a programmatic interface (API). |
Guest |
An unregistered User or an unauthenticated Consumer with limited access to the EP’s services |
Home Platform |
The Platform on which a User is based or from which an action was initiated by a User |
Host Platform |
The Platform through which a Resource has been published |
Identity Provider (IdP) |
The source for validating user identity in a federated identity system, (user authentication as a service). |
Interactive Application |
A stand-alone application provided within the exploitation platform for on-line hosted processing. Provides an interactive interface through which the user is able to conduct their analysis of the data, producing Analysis Results as output. Interactive Applications include at least the following types: console application, web application (rich browser interface), remote desktop to a hosted VM. |
Interactive Console Application |
A simple Interactive Application for analysis in which a console interface to a platform-hosted terminal is provided to the user. The console interface can be provided through the user’s browser session or through a remote SSH connection. |
Interactive Remote Desktop |
An Interactive Application for analysis provided as a remote desktop session to an OS-session (or directly to a 'native' application) on the exploitation platform. The user will have access to a number of applications within the hosted OS. The remote desktop session is provided through the user’s web browser. |
Interactive Web Application |
An Interactive Application for analysis provided as a rich user interface through the user’s web browser. |
Key-Value Pair |
A key-value pair (KVP) is an abstract data type that includes a group of key identifiers and a set of associated values. Key-value pairs are frequently used in lookup tables, hash tables and configuration files. |
Kubernetes (K8s) |
Container orchestration system for automating application deployment, scaling and management. |
Login Service |
An encapsulation of Authenticated Login provision within the Exploitation Platform context. The Login Service is an OpenID Connect Provider that is used purely for authentication. It acts as a Relying Party in flows with external IdPs to obtain access to the user’s identity. |
EO Network of Resources |
The coordinated collection of European EO resources (platforms, data sources, etc.). |
Object Store |
A computer data storage architecture that manages data as objects. Each object typically includes the data itself, a variable amount of metadata, and a globally unique identifier. |
On-demand Processing Service |
A Processing Service whose execution is initiated directly by the user on an ad-hoc basis. |
Platform (EP) |
An on-line collection of products, services and tools for exploitation of EO data |
Platform Tier |
The Platform Tier represents the Exploitation Platform and the services it offers to end-users |
Processing |
A set of pre-defined activities that interact to achieve a result. For the exploitation platform, comprises on-line processing to derive data products from input data, conducted by a hosted processing service execution. |
Processing Result |
The Products produced as output of a Processing Service execution. |
Processing Service |
A non-interactive data processing that has a well-defined set of input data types, input parameterisation, producing Processing Results with a well-defined output data type. |
Products |
EO data (commercial and non-commercial) and Value-added products and made available through the EP. It is assumed that the Hosting Environment for the EP makes available an existing supply of EO Data |
Resource |
A entity, such as a Product, Processing Service or Interactive Application, which is of interest to a user, is indexed in a catalogue and can be returned as a single meaningful search result |
Resource Tier |
The Resource Tier represents the hosting infrastructure and provides the EO data, storage and compute upon which the exploitation platform is deployed |
Reusable Research Object |
An encapsulation of some research/analysis that describes all aspects required to reproduce the analysis, including data used, processing performed etc. |
Scientific Researcher |
Expert user with the objective to perform scientific research. Having minimal IT knowledge with no desire to acquire it, they want the effort for the translation of their algorithm into a service/product to be minimised by the platform. |
Service Developer |
Expert user with the objective to provide a performing, stable and reliable service/product. Having deeper IT knowledge or a willingness to acquire it, they require deeper access to the platform IT functionalities for optimisation of their algorithm. |
Software |
The compilation of code into a binary program to be executed within the platform on-line computing environment. |
Systematic Processing Service |
A Processing Service whose execution is initiated automatically (on behalf of a user), either according to a schedule (routine) or triggered by an event (e.g. arrival of new data). |
Terms & Conditions (T&Cs) |
The obligations that the user agrees to abide by in regard of usage of products/services of the platform. T&Cs are set by the provider of each product/service. |
Transactional Web Processing Service (WPS-T) |
Transactional extension to WPS that allows adhoc deployment / undeployment of user-provided processors. |
User |
An individual using the EP, of any type (Admin/Consumer/Expert/Guest) |
Value-added products |
Products generated from processing services of the EP (or external processing) and made available through the EP. This includes products uploaded to the EP by users and published for collaborative consumption |
Visualisation |
To obtain a visual representation of any data/products held within the platform - presented to the user within their web browser session. |
Web Coverage Service (WCS) |
OGC standard that provides an open specification for sharing raster datasets on the web. |
Web Coverage Processing Service (WCPS) |
OGC standard that defines a protocol-independent language for the extraction, processing, and analysis of multi-dimentional coverages representing sensor, image, or statistics data. |
Web Feature Service (WFS) |
OGC standard that makes geographic feature data (vector geospatial datasets) available on the web. |
Web Map Service (WMS) |
OGC standard that provides a simple HTTP interface for requesting geo-registered map images from one or more distributed geospatial databases. |
Web Map Tile Service (WMTS) |
OGC standard that provides a simple HTTP interface for requesting map tiles of spatially referenced data using the images with predefined content, extent, and resolution. |
Web Processing Services (WPS) |
OGC standard that defines how a client can request the execution of a process, and how the output from the process is handled. |
Workspace |
A user-scoped 'container' in the EP, in which each user maintains their own links to resources (products and services) that have been collected by a user during their usage of the EP. The workspace acts as the hub for a user’s exploitation activities within the EP |
1.5. Glossary
The following acronyms and abbreviations have been used in this report.
Term | Definition |
---|---|
AAI |
Authentication & Authorization Infrastructure |
ABAC |
Attribute Based Access Control |
ADES |
Application Deployment and Execution Service |
ALFA |
Abbreviated Language For Authorization |
AOI |
Area of Interest |
API |
Application Programming Interface |
CMS |
Content Management System |
CWL |
Common Workflow Language |
DAL |
Data Access Library |
EMS |
Execution Management Service |
EO |
Earth Observation |
EP |
Exploitation Platform |
FUSE |
Filesystem in Userspace |
GeoXACML |
Geo-specific extension to the XACML Policy Language |
IAM |
Identity and Access Management |
IdP |
Identity Provider |
JSON |
JavaScript Object Notation |
K8s |
Kubernetes |
KVP |
Key-value Pair |
M2M |
Machine-to-machine |
OGC |
Open Geospatial Consortium |
PDE |
Processor Development Environment |
PDP |
Policy Decision Point |
PEP |
Policy Enforcement Point |
PIP |
Policy Information Point |
RBAC |
Role Based Access Control |
REST |
Representational State Transfer |
SSH |
Secure Shell |
TOI |
Time of Interest |
UMA |
User-Managed Access |
VNC |
Virtual Network Computing |
WCS |
Web Coverage Service |
WCPS |
Web Coverage Processing Service |
WFS |
Web Feature Service |
WMS |
Web Map Service |
WMTS |
Web Map Tile Service |
WPS |
Web Processing Service |
WPS-T |
Transactional Web Processing Service |
XACML |
eXtensible Access Control Markup Language |
2. Overview
2.1. Building Block Overview
Content Description
This section contains:
|
The functionality of the License Manager will mainly be as support for the PDP Policy Engine when assessing policy checks that directly
relate to Licenses owned by the Platform or by the End-User requesting access to a specific resource.
-
Licence Managing Core, with the capability of recording which Licenses an End-User has available and
whether or not the limits applied to that license have been reached (i.e. if more than 5 users are
concurrently using a service with the same License key, a PDP request for Authorization could be
turned down). -
SCIM Client, allowing to retrieve user information that is local to the platform, whenever possible or to update User Profiles based on the
assignment of a specific license -
OIDC Client, allowing to authenticate the component as trusted within the architecture and used by the License Manager to identify itself as trusted party within the EP domain in
order to able to perform queries against the Login Service SCIM endpoints. -
Billing Connector allowing the generation of charging requests to the Billing Service, either timebased or volume-based.
-
Management API for License Owners to register Licenses into the system and to assign licenses to the Platform to specific End-Users
2.1.1. Initialization flow
The figure below, identifies the main workflows on which the License Manager participates, along with it’s components:
2.1.2. Exposed Interfaces
2.1.2.1. Licenses API
The API will be the platform where the owners will access in order to actually manage the licenses in use. For that an special credentials are needed since not every user has access, same as the login-service.
2.1.2.2. User Profile
In the User Profile web interface each user has access to their own license list stored into the login service user information endpoint. In order to retrieve that values it reuses the SCIM library of EOEPCA.
2.1.3. Consumed Interfaces
2.1.3.1. OIDC (to Login Service)
The PDP uses the OIDC protocol in order to authenticate itself as a valid UMA client, and uses this OIDC client in all UMA-related queries.It allows Clients to verify the identity of the End-User. (https://gluu.org/docs/gluu-server/4.0/admin-guide/openid-connect/)
These queries are done against the Login Service, and the endpoints used are:
-
Discovery Endpoint: /.well-known/openid-configuration
And the keys used from Well Known Handler:
-
Token Endpoint: KEY_OIDC_TOKEN_ENDPOINT
-
UserInfo Endpoint: KEY_OIDC_USERINFO_ENDPOINT
2.1.3.2. SCIM (to Login Service)
The PDP has the capability to auto-register itself as a client if there is no client pre-configured from previous starts or previous configuration. In order to do this, it utilizes the SCIM protocol which is designed to reduce the complexity of user management operations. (https://gluu.org/docs/gluu-server/3.1.1/user-management/scim2/)
The keys used from Well Known Handler:
-
User Atributes: KEY_SCIM_USER_ENDPOINT
-
Private Key JWT Key: ENDPOINT_AUTH_CLIENT_PRIVATE_KEY_JWT
2.2. Required resources
Content Description
This section contains:
|
2.2.1. Interdependencies
The following list organizes main identified features based on inter-dependencies and impact on the
overall reference implementation functionality:
-
UM-LIM-010: SCIM Client connection to License Manager
-
UM-LIM-020: OIDC Client connection to License Manager
-
UM-LIM-030: License Manage Core (Use Cases)
-
UM-LIM-040: Billing Connector
-
UM-LIM-050: License Management API
2.2.2. Software
The following Open-Source Software is required to support the deployment and integration of the Policy Enforcement Point:
-
EOEPCA’s SCIM Client - https://github.com/EOEPCA/um-common-scim-client
-
EOEPCA’s OpenID - https://github.com/EOEPCA/um-common-oidc-client
-
EOEPCA’s Well Known Handler - https://github.com/EOEPCA/well-known-handler
2.3. Use cases
2.3.1. Owner Run Operation
In this case the owner interacts with the API in order to operate against the licenses. This secuential diagram shows the different components the Building Block manages.

2.3.2. API Interaction

2.3.3. End User access to a Resource protected with Licenses
The interaction within the system can be shown in the following diagram that illustrates how the end user activates a license check when accessing a resource through the PEP and PDP

2.3.4. End User access to the API
The interaction within the system can be shown in the following diagram that illustrates how the end user interacts with the API

2.3.5. End User Licenese management through the User Profile
The User Profile offers an interface to manually edit the user attributes, in specific the Licenses associated.
The next diagram will show that interaction:

3. Design
3.1. Building Block Design
Content Description
This section contains:
When a breakdown is necessary, a general overview of the building block can be given. On the contrary, no breakdown indicates a single component development with the same expected sections. |
3.2. License Manager Core
3.2.1. Overview and Purpose
It is the main component of the building block, in which the scope of each license and its owner are determined. The objective is to recursively allocate licenses according to the privileges of the End User or the owner of the platform.
Within the core of the building block the information of the relations between licenses, resources and Users is stored and managed.
3.2.2. Software Reuse and Dependencies
All requirements for the executing of the reverse proxy are found under src/requirements.txt, and expect Python 3.6.9 or greater to work.
The most important are:
-
EOEPCA-SCIM: Used to auto-register itself as a client to the Auth. Server upon startup
-
EOEPCA-OIDC: Used to generate PAT tokens, validate OAuth tokens and JWTs.
-
WellKnownHandler: Used to dynamically check the configuration of the Authorization Server on each execution. For example, it can get the needed endpoints for any API the PEP needs, such as the token request for OIDC.
3.2.3. Interfaces
This component doesn’t have any internal interfaces. For a reference of external interfaces see [External Interfaces] on Section 2 Overview
3.2.4. Data
3.2.4.1. Configuration
The License Manager gets all its configuration from the file located under config/config.json
.
3.2.4.2. Data flow
The information that the License Manager handles is mainly user related since the authentication is propertly handled by the OIDC library and the user information from the SCIM Library.
As it is a component that serves the PDP, a policy check will run the data flow of the License Manager to allow or deny access to a requested resource.
The core also deals with the billing service when it comes to communicating the use of volume and time of the licenses and its consequent impact on the invoicing of the associated resource.
3.2.5. Applicable Resources
-
EOEPCA’s SCIM Client - https://github.com/EOEPCA/um-common-scim-client
-
EOEPCA’s UMA Client - https://github.com/EOEPCA/um-common-uma-client
-
EOEPCA’s Well Known Handler - https://github.com/EOEPCA/well-known-handler
<< End of Document >>