Copernicus Climate Change Serviceinspire.ec.europa.eu › events › conferences › inspire_2016...

Copernicus Climate Change Service

Climate Data Store (CDS) & Toolbox.

Angel Lopez Alos [email protected] Raoult [email protected] Bergeron [email protected]


2

Be an authoritative source of climate information for Europe

Build upon national investments and complement national climate service providers

Support the market for climate services in Europe

C3SOverall Vision and Value


3


Questions addressed in C3S:• How is climate changing?

• Earth observations• Reanalyses

• Will climate change continue/accelerate?• Predictions• Projections

• What are the societal impacts?• Climate indicators• Sectoral information

C3SOverall Vision and Value

Reliable and up-to-date Information for Adaptation and Mitigation


4

ClimateData Store

SectoralInformation

System

Evaluation andQuality Control

Outreach and Dissemination

C3SComponents

4

• ECVs past, present and future

• Observed, reanalysedand simulated

• Derived climate indicators

• Tools to support adaptation and mitigation at global and European level

• Monitors quality of C3S products and services

• Ensures C3S delivers state‐of‐the‐art climate information to end‐users

• Identifies gaps in service provision

• Bridges Copernicus with the research agenda in Europe (e.g. H2020, national research projects)

• Web content • Public outreach• Coordination with national outreach

• Liaison with public authorities

• Conferences, seminars• Training and education


5

ReanalysesMulti model seasonal

forecast products

Data reprocessing

Data collection and data rescue

Other ECVdatasets

Climateprojections

Products Landscape on the

Climate Data Store / ToolboxContent


6

Scientific basis:• Essential Climate Variables as defined

by GCOS • GCOS Status Report (GCOS-195)• IPCC, CMIP

ObservationsObservationsGlobal estimates

of ECVs from satellite and in-

situ data

Reprocessed CDRs, reference

observations

Support for data rescue, climate data collections

Climate reanalysisClimate

reanalysisGlobal

atmosphere, ocean, land

Regional reanalysis

Coupled climate reanalysis for

100 years

Seasonal forecasts and

climate projections

Seasonal forecasts and

climate projections

Multi-model seasonal

forecast products

Access to CMIP data and products

European multi-model climate

projections

Clim

ateIndicators

Climate Data Store / ToolboxContent


7

Stage 0/1 ‐ Proof of Concept/Pre‐operationalStage II ‐ Operational ~20 ECVs, ~5‐6 SectorsStage III ‐ Operational ~30 ECVs, ~8‐10 Sectors

2013 2014 2015 2016 2017 2018 2019 2020 2021

Stage 0 Stage I Stage II Stage III

PoC + Pre‐operational Phase

Operational Phase

C3S Timeline


8

The Climate Data Store

(CDS)


9

• Climate Data Store will be at the heart of the C3S infrastructure.

• Distributed System with a unique interface offering:• Information about past, present and future climate.• Essential Climate Variables and derived climate indicators. • Observations, global and regional climate reanalyses, global and regional

climate projections and seasonal forecasts.• Authoritative set of software (toolbox) that will allow the users to develop

applications that will make use of the content of the CDS.

• CDS will accommodate the needs of the highly diverse set of users that will include policy makers, experts as well as scientists.

CDS: Core Infrastructure supporting the

Climate Data Store / ToolboxVision


10

MAIN CHALLENGES FOR

• Diversity of Products • Raw to elaborated

• Diversity of Users and Use Cases• Scientist to policy maker.

• Diversity of Volumes• From KB to PB

• Future Scalability• Increment in the number of Users, Providers,

Requirements,…

Public Authority; Urgent Requirements;

Elaborated data (charts, graphics, …);

Mb

CDS

Scientist; Research Proyect;

Raw Data; Pb

Climate Data Store / ToolboxMain Challenges


11

• Integration of new requirements, data sources, functionalities, scenarios, …

• Guarantee high availability [24/7] for data access.

• Open to new platforms, software tools and methodologies.

• Provide with high level added-value on data services.

• Intuitive and friendly Front-end, robust and scalable backend.

• Data Processing closer to data sources

• Interoperability – Compliance with Standards.

Aim: Create an Open Framework further than a classical Web Portal

Climate Data Store / ToolboxMain Challenges


12

Climate Data Store / ToolboxArchitecture

Monitoring / M

etrics

Computation Exec. Results/Cache

Data/Products CatalogueDiscover/View/

Download

Data/ProductMetadata Management*Users Management*Authentication

Tools/WorkflowManagement*

Application Management*

Applications Use

WEB PORTAL

BACK END

Data/Products Catalogue

Tools/Workflow Inventory

User Settings

User Requests

Web PagesContent Management

System

BROKER / Scheduler Compute

QueuesStatus

Applicationsinventory

Staging

Adaptor Adaptor Adaptor Adaptor

Data/Products Catalogue

Iso19115 Export

Tools/Workflows Repository

Computation Exec.

Mon

itorin

g / M

etric

s

DATASite 2

Tools Repository

Computation

Data Elaborated

DATASite 1

DATADATADATASite nDATA

Site 4

*Creation Modification Deletion

CLIM

ATEDATA

STORE

Toolbox Infrastructure

DATASite 3

Tools Repository

Computation

Data Elaborated

End User CDS Administrator CDS EQCExternal Information

SystemsData Supplier

Adaptor

Browse Browse API CatalogueExport

DATA

SUPPLIERS

FMI SmartMet


13

• “Web Portal”

• Content Management System (articles, news, events, …)

• Catalogue of products • ISO19115, for interoperability with INSPIRE, GEOSS, WMO Information System, …

• Users’ settings• Profiles, preferences, licenses, …

• Users’ requests• Current and past users requests: data retrievals and computations• Request management: monitor, cancel, download results

Climate Data Store / ToolboxWeb Portal


14

• Broker/Scheduler• Dispatches data retrieval and computation requests

to the relevant data repositories• Implements quality of service (QoS).

• Adaptors• Ensure interoperability between the broker and each of the data

repositories.

• Data Repositories (distributed)• Located at each data providers, hold datasets available via CDS.• May implement some tools to perform analytics on local data


15

• Toolbox• Tools that run next to the data (e.g. at the data providers)

when possible• Workflows that orchestrates the invocation of tools in a

distributed fashion to combine data from different data repositories

• Web based applications that let users interact with workflows

• Compute layer• Will be used by the toolbox when tools cannot be run on the

data• Will make use of cloud technologies

• REST Based API• Provides batch access to products and tools.


16

• Monitoring• Reporting

• Capacity planning• Usage statistics

• Service level agreement• On-call and support• Help desk

• High-availability• Backup

Climate Data Store / ToolboxOperations


17

Climate Data Store / ToolboxToolbox Challenges

CDS-ToolBox particular challenges:

• Data residing at different locations

• Interoperability, efficiency

• User-defined workflows

• Variety of presentation methods

• Need for interactivity

• Access via API

maps

text

controls

graphs


18

Example #1 (hypothetical):Monthly average of temperature at selected location

• ERA5: 40 years reanalysis, hosted at ECMWF (GRIB, in Kelvin)• CIMP6: 2000 years climate projections, hosted in an ESGF node

(NetCDF, Kelvin)• Observation: time series of temperature measured at a given station,

hosted in ClimatDBase (SQL, imaginary dataset)


19

Stakeholders, UX Inputs, ..

System Architecture, Mockups, Flow Diagrams, …

Development and Implementation

Agile Methodology: Brings flexibility and adaptability during development process.

Climate Data Store / ToolboxAgile Development


20


21

KO Sprint 0 Sprints 1-6 Sprints 7-13

6Se

p 16

04 O

ct 1

6SC1 End

30 S

ep 1

7

5 Se

p 16

Initial Product Backlog

Development Environment

Roadmap Release 0.1

Roadmap Release 1.0

Q1 Review

Q1 Review

v0.1 Review

v0.1 Review

v1.0Review

v1.0Review

4 weeks 6 x 4 weeks 6 x 4 weeks

Review Meetings(at ECMWF)

Climate Data Store ToolboxDevelopment Timeline to release 1.0

04 M

ar 1

7


22

Sectorial Information Systems (SIS)


23

Proof Of Concept Development of

sectorial applications

Copernicus Climate Change

Service

Sectoral Information SystemsOverview


24


Seven proof of concept SIS contracts have been awarded:

SIS water management: SWICCA (Service for Water Indicators in Climate Change adaptation) – lead(Sweden) EDgE (End-to-End demonstrator for improved decision making in the water sector in

Europe) – Lead CEH (UK)

SIS energy: CLIM4ENERGY (Climate for Energy) – Lead CEA (France) ECEM (European Climatic Energy Mixes) – Lead UEA (UK)

SIS others: AgriCLASS (Agriculture Climate Advisory Services) – Lead Telespazio (UK) WISC (Windstorm Information Service) – Lead (UK) URBAN-SIS (touching health, infrastructure, water) – Lead SMHI (Sweden)

Sectoral Information SystemsOverview


25

•Simple presentation•Maps and time-series plots•Narrative films

•Uncertainty and model skill•For seasonal forecasts & long-term projections•How best to visualise/ disseminate?

•Access to underlying data•Subset of point data•Gridded data to drive models•NetCDF/ csv files/ GIS format•Standard Services (WXS)

SIS Interfaces


26

Focus Groups & Case Studies in 3 contrasting parts of Europe:

Spain

Norway

United Kingdom

SIS Use Cases

Use Case driving Approach (e.g. EDgE)


27

Predicting changes in disruption risks

Predicting changes in renewable resource

Predicting changes in conditions for operating, maintenance, commissioning

and decommissioning

Balancing grid load in an increasingly variable production mix

Energy: CLIM4ENERGY


28 http://swicca.eu

Water manager

Purveyor

Data Provider

(C3S)

28

C3S

Sectoral Information SystemsLife-cycle (eg. SWICCA)


29

Req

uire

men

ts

Constrained by CDS-ToolBox and SIS Projects Roadmaps

Current Status

CDS V0.1

-

CDS V1.0

Toolbox V1

CDS V2.0

Toolbox V2

Agile Development

CDS Vx.0

Toolbox Vx

Sectoral Information SystemsIntegration with CDS - ToolBox

SIS “A”

SIS “B”

Catalogue MetadataData Services


30

Thank You!

climate.copernicus.eu

Copernicus Climate Change Serviceinspire.ec.europa.eu › events › conferences › inspire_2016...

Documents

Transcript of Copernicus Climate Change Serviceinspire.ec.europa.eu › events › conferences › inspire_2016...