Pull subscriptions to CREODIAS EODATA catalogue
Push and pull subscriptions allow you to receive notifications about products published in EODATA catalogue or a subset of such products.
This article covers pull subscriptions. For each of them, a queue stored in the cloud is created. The notifications are added to that queue as they appear and the user can view and acknowledge them.
The other type, push subscriptions, is covered in Push subscriptions to CREODIAS EODATA catalogue
What We Are Going To Cover
Introduction
What should a HTTP request contain?
Headers
Escaping special characters
Response to the HTTP call
HTTP request - Python example
Limitations of subscriptions
Basic information about pull subscriptions
Pull subscription entity description
Creating pull subscription
Status of created subscription
Viewing the queue
What does a notification look like?
Acking notifications
Using ack to read further notifications
Editing pull subscription
Example - changing the status of a pull subscription
Pull subscription - Example workflow
Listing current subscriptions
Deleting a subscription
Prerequisites
No. 1 Account
You need an active CREODIAS account https://new.cloudferro.com/.
No. 2 Ability to send HTTP requests
You need to be able to send HTTP requests, including POST and PATCH, containing headers and body.
No. 3 Knowledge how to filter products
You need to filter products only if you want to get notifications about a subset of the products in the catalogue.
Product filtering uses parameter $filter described in EOData Catalogue API Manual on CREODIAS.
No. 4 Generated Keycloak token
To prove your identity while using pull subscriptions, you need a Keycloak token. Here is how to generate it: Using curl and wget to download EODATA products from CREODIAS
Keycloak token is valid only for a limited time; you can generate new token if the current one expires and you want to continue using the subscriptions API.
Introduction
What should a HTTP request contain?
You can create and manage your pull subscriptions by sending HTTP requests.
When sending such a HTTP request, always use:
Type of request (usually GET, PATCH, DELETE)
Headers
URL
All these three parts of a HTTP request must be present together. Certain types of HTTP requests must also contain a body.
Headers
Requests in this article should contain the following headers:
{
"Content-Type": "application/json",
"Authorization": "Bearer <<access_token>>"
}
where <<access_token>> needs to be replaced with Keycloak token from Prerequisite No. 4 in order to authenticate you.
We will call them “the obligatory headings” and will refer to them as such throughout the article.
Escaping special characters
Requests in this article often contain special characters, for example, ‘ and &. Within a HTTP call, such characters must be escaped properly but the exact way of doing it will depend on the tool used. For example:
- URL
Use URL encoding (%27 for ‘ and %22 for “).
- JSON
Escape double quotes with backslashes (").
- Bash & curl
To escape single quotes, combine single and double quotes. To escape double quotes, use backslash inside double quotes.
- PHP
Use json_encode() and urlencode() functions.
Response to the HTTP call
Responses to these requests are either empty or consist of JSON data. Variations also exist: some JSON tools use “ to indicate strings, while others use ‘. Line formatting (or lack thereof) may vary.
HTTP request - Python example
In Python, the best way to send a HTTP request is using Python module requests. For JSON, you usually use pprint instead of print.
Make sure to have the requests module installed.
As mentioned, in this article we will show excerpts of code and you should add the mandatory rest of the code. For your reference, here is full code for request described in section Listing current subscriptions later in this article:
import pprint
import requests
import json
access_token = "<<your_access_token>>"
headers = {
"Content-Type": "application/json",
"Authorization": f"Bearer {access_token}",
}
url = "https://datahub.creodias.eu/odata/v1/Subscriptions/Info"
response = requests.get(url=url, headers=headers)
try:
pprint.pprint(response.json())
except requests.exceptions.JSONDecodeError:
print(response.text)
Assign Keycloak token obtained by following Prerequisite No. 4 to variable access_token.
Save the file as http_request_test.py.
To run the script in that file, execute the following command (assuming you are in the same folder as the script):
python3 http_request_test.py
The script should return
response code,
response reason and
response text (if any).
Limitations of subscriptions
Your account can only have 1 running subscription. It can be either a pull subscription (which is described in this article) or a push subscription (as mentioned in the beginning of this text).
The total number of subscriptions (both running and paused) is 10.
Basic information about pull subscriptions
Pull subscription creates a queue in which your notifications will appear as new products are being published. This queue is stored on the API servers.
The contents of the queue can be viewed by sending a HTTP request and will be returned as JSON.
To remove a notification from that queue, you need to ACK (acknowledge) it. It is done by sending another HTTP request. Such a request will remove all notifications that came chronologically before that notification as well.
The queue stores up to 100000 of such notifications. If there are more, the oldest notification or notifications will be removed to make space for the newer ones. Therefore, make sure to ACK notifications regularly.
Pull subscription entity description
| Property name | Type | Description | Example |
|---|---|---|---|
| Id | Guid | It is a universally unique identifier (UUID). The Id is a local identifier for a Subscription instance within the Catalogue, assigned upon Subscription creation. | 9249dde5-4838-4a34-8925-bac8c1d53f09 |
| Status | Subscription Status enumeration |
The allowed values of a subscription status are:
|
running |
| SubscriptionEvent | Subscription Event enumeration |
The subscription event to be monitored and for which notification is provided:
|
created |
| FilterParam | String | The filter parameters of the Subscription (refers to the $filter= parameter of any The same filtering parameters as described for EOData Catalogue are available. | Collection/Name eq 'SENTINEL-1' and Attributes/OData.CSC.StringAttribute/any (att:att/Name eq 'productType' and att/OData.CSC.StringAttribute/Value eq 'IW_SLC__1S') |
| SubmissionDate | DateTimeOffset | Date and time at which the Subscription was received by the Catalogue. Time is in UTC in the format YYYY-MMDDThh:mm:ss.sssZ | 2024-01-17T09:13:04.654Z |
| LastNotificationDate | DateTimeOffset | Date and time at which the Subscription was received by the Catalogue. Time is in UTC in the format YYYY-MMDDThh:mm:ss.sssZ | 2024-01-17T09:50:10.654Z |
| StageOrder | Boolean | Automatically orders the staging of products fulfilling the subscription. Only used if SubscriptionEvent = created Currently, the order of staging products is not feasible as all products in EOData Catalogue have status set to 'Online' and can be accessed immediately without setting order. | true |
| Priority | Int64 | Priority of the created orders resulting from the subscription. Currently automatically fixed to '1' without the possibility to change the value. | 1 |
Creating pull subscription
Send POST request to the following URL:
https://datahub.creodias.eu/odata/v1/Subscriptions
REMINDER: Include the obligatory headers.
Include the following body to this request (replace the values as instructed):
{
"StageOrder": true,
"FilterParam": "Collection/Name eq 'SENTINEL-1' and Attributes/OData.CSC.StringAttribute/any(att:att/Name eq 'productType' and att/OData.CSC.StringAttribute/Value eq 'IW_SLC__1S')",
"Priority": 1,
"Status": "running",
"SubscriptionEvent": [
"created"
]
}
In this body, change the value of key FilterParam to an appropriate filter query of your choice. See Prerequisite No. 3 for more information.
You can also get notifications about all products published in the catalogue, not only a subset of them. To that end, you can either
leave the value of key FilterParam empty, or
do not include that key with your request at all.
REMINDER: Include the obligatory headers.
When you’re ready, send the request.
Response code to that request should be 201 Created. Response should contain JSON data like this:
{
"Id": "991c4730-cf6f-432a-9f6c-47be0230ff45",
"FilterParam": "Collection/Name eq 'SENTINEL-1' and Attributes/OData.CSC.StringAttribute/any(att:att/Name eq 'productType' and att/OData.CSC.StringAttribute/Value eq 'IW_SLC__1S')",
"StageOrder": true,
"Priority": 1,
"Status": "running",
"SubscriptionEvent": [
"created"
],
"SubmissionDate": "2024-03-13T09:39:49.404Z",
"@odata.context": "$metadata#Subscriptions/$entity"
}
Status of the created subscription
A subscription (either push or pull) can have one of the following statuses:
running - the subscription receives new notifications
paused - the subscription temporarily does not receive new notifications
cancelled - the subscription no longer receives new notifications and will be removed
Status: running is the default value, meaning that that will be the status of a subscription for which no status was specified during its creation.
If you already have a subscription with this status, this operation will fail since one user at any time can only have one active subscription.
To resolve, you can pause your current subscription (more information about it is later on in this article) and then create a new subscription.
If you, however, don’t want to pause your current subscription but want to create a new, inactive subscription, you can create a new subscription with status paused. To do that, while following section Creating pull subscription, use paused for Status, instead of running:
"Status": "paused",
Viewing the queue
To view the notification queue, send GET request to the following URL:
https://datahub.creodias.eu/odata/v1/Subscriptions(<<subscription_id>>)/Read$top=20
Replace <<subscription_id>> with the ID of the subscription you obtained when creating it.
The value of parameter $top represents the number of notifications you want to see. Its maximum vale is 20. If you do not provide this parameter, it will be set to 1 by default.
REMINDER: Include the obligatory headers.
You should get a list of zero to twenty oldest notifications, depending on the length of the queue and value of $top parameter. Response code should be 200 OK.
Each of these notifications is presented as a dictionary which contains information about the product, notification and subscription.
If you do not have any notifications in the queue, the query above should return an empty list:
[]
What does a notification look like?
This is an example of pull notification:
[
{
"@odata.context": "$metadata#Notification/$entity",
"AckId": "MTcxODYyNTIxNTA1Ny0wOjliODZiYWVkLTY0YzEtNDdjZi05NGFlLWI1ODY2NjZkYTYyMg==",
"NotificationDate": "2024-06-17T11:53:35.000Z",
"ProductId": "c739f940-f8b5-443d-98a0-0e2b2ea2cf90",
"ProductName": "S5P_NRTI_L2__AER_LH_20240617T100024_20240617T100524_34603_03_020600_20240617T114720.nc",
"SubscriptionEvent": "created",
"SubscriptionId": "9b86baed-64c1-47cf-94ae-b586666da622",
"value": {
"@odata.context": "$metadata#Products(Attributes())(Assets())(Locations())/$entity",
"@odata.mediaContentType": "application/octet-stream",
"Assets": [],
"Attributes": [
{
"@odata.type": "#OData.CSC.IntegerAttribute",
"Name": "orbitNumber",
"Value": 34603,
"ValueType": "Integer"
},
{
"@odata.type": "#OData.CSC.StringAttribute",
"Name": "processorName",
"Value": "TROPNLL2DP",
"ValueType": "String"
},
{
"@odata.type": "#OData.CSC.DateTimeOffsetAttribute",
"Name": "processingDate",
"Value": "2024-06-17T11:47:20+00:00",
"ValueType": "DateTimeOffset"
},
{
"@odata.type": "#OData.CSC.StringAttribute",
"Name": "processingMode",
"Value": "NRTI",
"ValueType": "String"
},
{
"@odata.type": "#OData.CSC.StringAttribute",
"Name": "processingLevel",
"Value": "L2",
"ValueType": "String"
},
{
"@odata.type": "#OData.CSC.StringAttribute",
"Name": "parentIdentifier",
"Value": "urn:ogc:def:EOP:ESA:SENTINEL.S5P_TROP_L2__AER_LH",
"ValueType": "String"
},
{
"@odata.type": "#OData.CSC.StringAttribute",
"Name": "processingCenter",
"Value": "PDGS-OP",
"ValueType": "String"
},
{
"@odata.type": "#OData.CSC.StringAttribute",
"Name": "processorVersion",
"Value": "020600",
"ValueType": "String"
},
{
"@odata.type": "#OData.CSC.StringAttribute",
"Name": "platformShortName",
"Value": "SENTINEL-5P",
"ValueType": "String"
},
{
"@odata.type": "#OData.CSC.StringAttribute",
"Name": "instrumentShortName",
"Value": "TROPOMI",
"ValueType": "String"
},
{
"@odata.type": "#OData.CSC.StringAttribute",
"Name": "productType",
"Value": "L2__AER_LH",
"ValueType": "String"
},
{
"@odata.type": "#OData.CSC.DateTimeOffsetAttribute",
"Name": "beginningDateTime",
"Value": "2024-06-17T10:00:18.000Z",
"ValueType": "DateTimeOffset"
},
{
"@odata.type": "#OData.CSC.DateTimeOffsetAttribute",
"Name": "endingDateTime",
"Value": "2024-06-17T10:03:00.000Z",
"ValueType": "DateTimeOffset"
}
],
"Checksum": [
{
"Algorithm": "MD5",
"ChecksumDate": "2024-06-17T11:53:34.158020Z",
"Value": "9c78781abe2bf0125c9b001700d4c369"
},
{
"Algorithm": "BLAKE3",
"ChecksumDate": "2024-06-17T11:53:34.182568Z",
"Value": "842e321aa7dd8bd1f5780491f980f5cecf2473392f26b10d96565f8ad93d5e22"
}
],
"ContentDate": {
"End": "2024-06-17T10:03:00.000Z",
"Start": "2024-06-17T10:00:18.000Z"
},
"ContentLength": 3349004,
"ContentType": "application/octet-stream",
"EvictionDate": "",
"Footprint": "geography'SRID=4326;POLYGON ((-77.09505 63.7031, -76.65646 63.44439, -74.85055 63.9593, -71.98157 64.82188, -68.925896 65.622986, -65.680305 66.3564, -62.24578 67.01546, -60.579784 67.29585, -60.692856 67.36853, -63.370583 68.951614, -67.1073 70.796936, -70.23037 72.09402, -72.96789 73.09068, -75.45267 73.90361, -77.774956 74.59668, -80.003265 75.208565, -82.194855 75.76458, -84.40237 76.28251, -86.67906 76.77569, -86.89144 76.81968, -89.08415 77.25481, -91.689476 77.72902, -94.589386 78.206505, -97.916595 78.69464, -101.86985 79.199394, -106.76488 79.72298, -113.1327 80.25677, -121.912926 80.75976, -134.80223 81.0876, -147.34233 80.98266, -150.8459 80.87634, -148.04048 79.53462, -145.92802 78.16995, -144.30133 76.78949, -143.02618 75.397484, -142.00768 73.96712, -140.7294 71.14397, -140.13257 69.83876, -139.89313 69.85107, -134.90149 70.03338, -128.26067 70.06861, -123.465004 69.95807, -119.72813 69.79932, -116.654495 69.62488, -114.01812 69.446045, -111.67769 69.26598, -109.537926 69.08445, -107.52979 68.89955, -105.5993 68.70847, -105.42602 68.690674, -103.70045 68.50759, -101.79022 68.2923, -99.82416 68.05647, -97.75174 67.79154, -95.510254 67.48465, -93.01531 67.11531, -90.14401 66.64836, -86.700714 66.017654, -82.33445 65.08131, -78.17603 64.01363, -77.09505 63.7031))'",
"GeoFootprint": {
"coordinates": [
[
[
-77.09505,
63.7031
],
[
-76.65646,
63.44439
],
[
-74.85055,
63.9593
],
[
-71.98157,
64.82188
],
[
-68.925896,
65.622986
],
[
-65.680305,
66.3564
],
[
-62.24578,
67.01546
],
[
-60.579784,
67.29585
],
[
-60.692856,
67.36853
],
[
-63.370583,
68.951614
],
[
-67.1073,
70.796936
],
[
-70.23037,
72.09402
],
[
-72.96789,
73.09068
],
[
-75.45267,
73.90361
],
[
-77.774956,
74.59668
],
[
-80.003265,
75.208565
],
[
-82.194855,
75.76458
],
[
-84.40237,
76.28251
],
[
-86.67906,
76.77569
],
[
-86.89144,
76.81968
],
[
-89.08415,
77.25481
],
[
-91.689476,
77.72902
],
[
-94.589386,
78.206505
],
[
-97.916595,
78.69464
],
[
-101.86985,
79.199394
],
[
-106.76488,
79.72298
],
[
-113.1327,
80.25677
],
[
-121.912926,
80.75976
],
[
-134.80223,
81.0876
],
[
-147.34233,
80.98266
],
[
-150.8459,
80.87634
],
[
-148.04048,
79.53462
],
[
-145.92802,
78.16995
],
[
-144.30133,
76.78949
],
[
-143.02618,
75.397484
],
[
-142.00768,
73.96712
],
[
-140.7294,
71.14397
],
[
-140.13257,
69.83876
],
[
-139.89313,
69.85107
],
[
-134.90149,
70.03338
],
[
-128.26067,
70.06861
],
[
-123.465004,
69.95807
],
[
-119.72813,
69.79932
],
[
-116.654495,
69.62488
],
[
-114.01812,
69.446045
],
[
-111.67769,
69.26598
],
[
-109.537926,
69.08445
],
[
-107.52979,
68.89955
],
[
-105.5993,
68.70847
],
[
-105.42602,
68.690674
],
[
-103.70045,
68.50759
],
[
-101.79022,
68.2923
],
[
-99.82416,
68.05647
],
[
-97.75174,
67.79154
],
[
-95.510254,
67.48465
],
[
-93.01531,
67.11531
],
[
-90.14401,
66.64836
],
[
-86.700714,
66.017654
],
[
-82.33445,
65.08131
],
[
-78.17603,
64.01363
],
[
-77.09505,
63.7031
]
]
],
"type": "Polygon"
},
"Id": "c739f940-f8b5-443d-98a0-0e2b2ea2cf90",
"Locations": [
{
"Checksum": [
{
"Algorithm": "MD5",
"ChecksumDate": "2024-06-17T11:53:34.158020Z",
"Value": "9c78781abe2bf0125c9b001700d4c369"
},
{
"Algorithm": "BLAKE3",
"ChecksumDate": "2024-06-17T11:53:34.182568Z",
"Value": "842e321aa7dd8bd1f5780491f980f5cecf2473392f26b10d96565f8ad93d5e22"
}
],
"ContentLength": 3349004,
"DownloadLink": "https://datahub.creodias.eu/odata/v1/Products(c739f940-f8b5-443d-98a0-0e2b2ea2cf90)/$value",
"FormatType": "Extracted",
"S3Path": "/eodata/Sentinel-5P/TROPOMI/L2__AER_LH/2024/06/17/S5P_NRTI_L2__AER_LH_20240617T100024_20240617T100524_34603_03_020600_20240617T114720"
}
],
"ModificationDate": "2024-06-17T11:53:34.402Z",
"Name": "S5P_NRTI_L2__AER_LH_20240617T100024_20240617T100524_34603_03_020600_20240617T114720.nc",
"Online": true,
"OriginDate": "2024-06-17T11:48:17.858Z",
"PublicationDate": "2024-06-17T11:53:33.053Z",
"S3Path": "/eodata/Sentinel-5P/TROPOMI/L2__AER_LH/2024/06/17/S5P_NRTI_L2__AER_LH_20240617T100024_20240617T100524_34603_03_020600_20240617T114720"
}
}
]
Explanation of keys:
| Property name | Type | Description | Example |
|---|---|---|---|
| ProductId | Guid | It is a universally unique identifier (UUID) of the product being notified. | 8d654e59-d7b6-4a53-b086-c9de764e506b |
| ProductName | String | It is the name of the product being notified assigned within the Catalogue. | S2A_MSIL2A_20240129T062121_N0510_R034_T44VMN_20240129T093752.SAFE |
| SubscriptionId | Guid | It is a universally unique identifier (UUID). The Id is a local identifier for the Subscription instance within the Catalogue, assigned upon Subscription creation. | 991c4730-cf6f-432a-9f6c-47be0230ff4 |
| SubscriptionEvent | Subscription Event enumeration |
The subscription event to be monitored and for which notification is provided:
|
created |
| NotificationDate | DateTimeOffset | Date and time at which the notification was generated. Time is in UTC in the format YYYYMM-DDThh:mm:ss.sssZ | 2024-01-29T10:59:08.698Z |
| SubmissionDate | DateTimeOffset | Date and time at which the Subscription was received by the Catalogue. Time is in UTC in the format YYYY-MMDDThh:mm:ss.sssZ | 2024-01-17T09:13:04.654Z |
| LastNotificationDate | DateTimeOffset | Date and time at which the Subscription was received by the Catalogue. Time is in UTC in the format YYYY-MMDDThh:mm:ss.sssZ | 2024-01-17T09:50:10.654Z |
| StageOrder | Boolean | Automatically orders the staging of products fulfilling the subscription. Only used if SubscriptionEvent = created. Currently, the order of staging products is not feasible as all products in EOData Catalogue have status set to ‘Online’ and can be accessed immediately without setting order. | true |
| Priority | Int64 | Priority of the created orders resulting from the subscription. Currently automatically fixed to ‘1’ without the possibility to change the value. | 1 |
| AckId | String | Acknowledge Id assigned to each product notification. Required for acknowledging the notification messages from the client's queue. | MTcxMDc1NjcwNjUzMi0wOjk5MWM0NzMwLWNmNmYtNDMyYS05ZjZjLTQ3YmUwMjMwZmY0NQ== |
| value | Dictionary | It contains a standard Catalogue AP response with all metadata and content details about a product. |
Full information about a product is kept inside a notification for three days. After that, the notification will only contain a subset of its original keys. A notification after three days could look like this:
[{
"@odata.context": "$metadata#Notification/$entity",
"SubscriptionEvent": "created",
"ProductId": "d2ff986b-9454-43d6-95e0-1a0ae27019d7",
"SubscriptionId": "991c4730-cf6f-432a-9f6c-47be0230ff45",
"NotificationDate": "2024-03-13T13:39:59.000Z",
"AckId": "MTcxMDc1NjcwNjUzMi0wOjk5MWM0NzMwLWNmNmYtNDMyYS05ZjZjLTQ3YmUwMjMwZmY0NQ=="
}]
Acking notifications
Acknowledging (acking) a notification removes (from the queue) both that notification and all of the notifications that came chronologically before it.
Each notification contains key AckID. Its value has the following type: String.
You will need that value to ack that notification (and, of course, all notifications that came before it). You can see that AckID in the notification queue.
To ack a notification, send POST request to URL below. In it, replace <<subscription_id>> with the ID of your subscription and <<AckID>> with the appropriate AckID:
https://datahub.creodias.eu/odata/v1/Subscriptions(<<subscription_id>>)/Ack?$ackid=<<AckID>>
REMINDER: Include the obligatory headers.
You should get response 200 OK. The response should also contain JSON like this:
{
"@odata.context": "$metadata#Notification/$entity",
"AckMessagesNum": 2,
"CurrentQueueLength": 2115,
"MaxQueueLength": 100000
}
which contains the following information:
| Key | Type | Description | Example |
|---|---|---|---|
| AckMessagesNum | Int64 | The number of the notifications that were acknowledged by your request. | 20 |
| CurrentQueueLength | Int64 | Number of notifications currently in your queue. | 2115 |
| MaxQueueLength | Int64 | The maximum number of notifications which can be stored in your queue | 100000 |
Using ack to read further notifications
Request used to view the queue can only return up to 20 oldest notifications. The maximum number of notifications which can exist in a queue is 100000. Therefore, that request might be insufficient to fetch all notifications.
To read the remaining messages, you can use ack.
View the current list of notifications - set $top parameter to 20.
Save these notifications if needed.
Ack the last of them.
Once again, have a look at the current list of notifications - it should now contain notifications which came after the notification you acked. If there are more than 20 notifications in your queue, you should now see notifications which you did not see previously.
Once again, save notifications if needed and ack the last of them.
Do so until your queue is empty.
Editing pull subscription
For pull subscription, you can only change its status.
As stated in subsection Status of created subscription, the three possible statuses are running, paused and cancelled.
To edit status of the subscription, send HTTP PATCH request to this URL:
https://datahub.creodias.eu/odata/v1/Subscriptions(<<subscription_id>>)
Replace <<subscription_id>> with the ID of the subscription you want to edit.
The body should be like this:
{
"Status": "<<new_status>>"
}
Replace <<new_status>> with the status you want your subscription to set to.
REMINDER: Include the obligatory headers.
Some parameters, such as Id, cannot be changed. If you try to change them anyway, their values should remain unchanged.
You should get a response with updated information about your subscription, alongside with the response code 200 OK.
Example - changing the status of a pull subscription
If the ID of your subscription is 7ca682c3-7b21-4e9b-952e-874798181340 and you want to set its status to paused, send the PATCH request to the following URL:
https://datahub.creodias.eu/odata/v1/Subscriptions(7ca682c3-7b21-4e9b-952e-874798181340)
The body of the request is:
{
"Status": "paused"
}
REMINDER: Include the obligatory headers.
Pull subscription - Example workflow
Here is what a concrete workflow could look like when you are interacting with a pull subscription.
We demonstrate the process of ACKing a notification and its effect on a notification queue. However, most requests presented in this section are not intended to be directly copied, pasted and executed because your notifications and subscriptions will be different, with different IDs and other parameters.
REMINDER: Include the obligatory headers.
Let’s assume that we want to track products based on the following query:
Collection/Name eq 'SENTINEL-5P' and Attributes/OData.CSC.StringAttribute/any(att:att/Name eq 'productType' and att/OData.CSC.StringAttribute/Value eq 'L2__AER_LH')
As of writing of this article, this query is good for testing because the queue should populate with notifications relatively slowly.
To create notification queue for such a query, we send a POST request to the following URL:
https://datahub.creodias.eu/odata/v1/Subscriptions
The body of this request is as follows:
{
"FilterParam": "Collection/Name eq 'SENTINEL-5P' and Attributes/OData.CSC.StringAttribute/any(att:att/Name eq 'productType' and att/OData.CSC.StringAttribute/Value eq 'L2__AER_LH')",
"Status": "running"
}
We then get a confirmation similar to this:
{
"@odata.context": "$metadata#Subscriptions/$entity",
"FilterParam": "Collection/Name eq 'SENTINEL-5P' and Attributes/OData.CSC.StringAttribute/any(att:att/Name eq 'productType' and att/OData.CSC.StringAttribute/Value eq 'L2__AER_LH')",
"Id": "5e8a716e-467e-42cf-8493-0b8987586a2d",
"Priority": 1,
"StageOrder": false,
"Status": "running",
"SubmissionDate": "2024-05-14T12:15:22.150Z",
"SubscriptionEvent": [
"created"
]
}
From this output, we learn that the ID of our subscription is 5e8a716e-467e-42cf-8493-0b8987586a2d
Now, we can check whether there are any notifications in our queue. We send GET request to the following URL:
https://datahub.creodias.eu/odata/v1/Subscriptions(5e8a716e-467e-42cf-8493-0b8987586a2d)/Read$top=20
The value of $top parameter is 20. This is the maximum possible value and we want to see as many notifications as possible.
Since no notifications have appeared on our queue yet, we get an empty list as a response:
[]
We wait some time until our queue gets populated with notifications.
Eventually, there are seven notifications on our list.
The code block below contains them, with the value of key value made empty to increase legibility. This simplification was applied to all blocks in this section.
[{"@odata.context": "$metadata#Notification/$entity",
"AckId": "MTcxNTY5MDMxNzk0Mi0wOjVlOGE3MTZlLTQ2N2UtNDJjZi04NDkzLTBiODk4NzU4NmEyZA==",
"NotificationDate": "2024-05-14T12:38:37.000Z",
"ProductId": "a9b506be-bbfd-468d-9d88-19914d68393f",
"ProductName": "S5P_NRTI_L2__AER_LH_20240514T102229_20240514T102729_34121_03_020600_20240514T122820.nc",
"SubscriptionEvent": "created",
"SubscriptionId": "5e8a716e-467e-42cf-8493-0b8987586a2d",
"value": {}},
{"@odata.context": "$metadata#Notification/$entity",
"AckId": "MTcxNTY5MDM4OTA0Mi0wOjVlOGE3MTZlLTQ2N2UtNDJjZi04NDkzLTBiODk4NzU4NmEyZA==",
"NotificationDate": "2024-05-14T12:39:49.000Z",
"ProductId": "e3ddbc79-c009-4392-b3aa-7f9abaf79dd3",
"ProductName": "S5P_NRTI_L2__AER_LH_20240514T103729_20240514T104229_34121_03_020600_20240514T122938.nc",
"SubscriptionEvent": "created",
"SubscriptionId": "5e8a716e-467e-42cf-8493-0b8987586a2d",
"value": {}},
{"@odata.context": "$metadata#Notification/$entity",
"AckId": "MTcxNTY5MDM5MDc4NS0wOjVlOGE3MTZlLTQ2N2UtNDJjZi04NDkzLTBiODk4NzU4NmEyZA==",
"NotificationDate": "2024-05-14T12:39:50.000Z",
"ProductId": "8d6b1395-cd47-45c9-bc6d-7f4b134725f2",
"ProductName": "S5P_NRTI_L2__AER_LH_20240514T102729_20240514T103229_34121_03_020600_20240514T122753.nc",
"SubscriptionEvent": "created",
"SubscriptionId": "5e8a716e-467e-42cf-8493-0b8987586a2d",
"value": {}},
{"@odata.context": "$metadata#Notification/$entity",
"AckId": "MTcxNTY5MDM5MjQxNC0wOjVlOGE3MTZlLTQ2N2UtNDJjZi04NDkzLTBiODk4NzU4NmEyZA==",
"NotificationDate": "2024-05-14T12:39:52.000Z",
"ProductId": "40a07db8-166d-43ba-b22e-9ae74f6a7af8",
"ProductName": "S5P_NRTI_L2__AER_LH_20240514T103229_20240514T103729_34121_03_020600_20240514T122927.nc",
"SubscriptionEvent": "created",
"SubscriptionId": "5e8a716e-467e-42cf-8493-0b8987586a2d",
"value": {}},
{"@odata.context": "$metadata#Notification/$entity",
"AckId": "MTcxNTY5MDU0Mzc4Mi0wOjVlOGE3MTZlLTQ2N2UtNDJjZi04NDkzLTBiODk4NzU4NmEyZA==",
"NotificationDate": "2024-05-14T12:42:23.000Z",
"ProductId": "c3947151-bae3-4879-be5f-c6a680f69209",
"ProductName": "S5P_NRTI_L2__AER_LH_20240514T112729_20240514T113229_34122_03_020600_20240514T123452.nc",
"SubscriptionEvent": "created",
"SubscriptionId": "5e8a716e-467e-42cf-8493-0b8987586a2d",
"value": {}},
{"@odata.context": "$metadata#Notification/$entity",
"AckId": "MTcxNTY5MDU0Njk5NS0wOjVlOGE3MTZlLTQ2N2UtNDJjZi04NDkzLTBiODk4NzU4NmEyZA==",
"NotificationDate": "2024-05-14T12:42:26.000Z",
"ProductId": "aad1aac7-7a63-41f5-b100-30d0d8b213c5",
"ProductName": "S5P_NRTI_L2__AER_LH_20240514T104229_20240514T104729_34121_03_020600_20240514T123438.nc",
"SubscriptionEvent": "created",
"SubscriptionId": "5e8a716e-467e-42cf-8493-0b8987586a2d",
"value": {}},
{"@odata.context": "$metadata#Notification/$entity",
"AckId": "MTcxNTY5MDYxNTMzMy0wOjVlOGE3MTZlLTQ2N2UtNDJjZi04NDkzLTBiODk4NzU4NmEyZA==",
"NotificationDate": "2024-05-14T12:43:35.000Z",
"ProductId": "c0608f85-7a4a-41c1-a31b-2750c96f5902",
"ProductName": "S5P_NRTI_L2__AER_LH_20240514T113229_20240514T113729_34122_03_020600_20240514T123604.nc",
"SubscriptionEvent": "created",
"SubscriptionId": "5e8a716e-467e-42cf-8493-0b8987586a2d",
"value": {}}]
Each of these notifications starts with the key @odata.context. After that, we get AckId which can be used to ack (and remove) that notification (and possibly others). There are also other pieces of information there, including basic information about a product and the date when the notification was received. The value of key value should contain more detailed information about product, but as stated previously, here it was collapsed to increase legibility.
The notifications are in chronological order, with the earliest being in the beginning.
Let’s say that we want to ack (and remove) the notification about product S5P_NRTI_L2__AER_LH_20240514T112729_20240514T113229_34122_03_020600_20240514T123452.nc - the fifth in the queue. Ack ID (value of key AckId) for notification about this product is MTcxNTY5MDU0Mzc4Mi0wOjVlOGE3MTZlLTQ2N2UtNDJjZi04NDkzLTBiODk4NzU4NmEyZA==.
This should cause the first five notifications to be removed.
We send a POST request to the following URL:
https://datahub.creodias.eu/odata/v1/Subscriptions(5e8a716e-467e-42cf-8493-0b8987586a2d)/Ack?$ackid=MTcxNTY5MDU0Mzc4Mi0wOjVlOGE3MTZlLTQ2N2UtNDJjZi04NDkzLTBiODk4NzU4NmEyZA==
We get the following response:
{"@odata.context": "$metadata#Notification/$entity",
"AckMessagesNum": 5,
"CurrentQueueLength": 2,
"MaxQueueLength": 100000}
According to the response, 5 notifications were acked as a result of our request and there are 2 notifications in the queue remaining.
When we view the queue again, we see that it is indeed the case:
[{"@odata.context": "$metadata#Notification/$entity",
"AckId": "MTcxNTY5MDU0Njk5NS0wOjVlOGE3MTZlLTQ2N2UtNDJjZi04NDkzLTBiODk4NzU4NmEyZA==",
"NotificationDate": "2024-05-14T12:42:26.000Z",
"ProductId": "aad1aac7-7a63-41f5-b100-30d0d8b213c5",
"ProductName": "S5P_NRTI_L2__AER_LH_20240514T104229_20240514T104729_34121_03_020600_20240514T123438.nc",
"SubscriptionEvent": "created",
"SubscriptionId": "5e8a716e-467e-42cf-8493-0b8987586a2d",
"value": {}},
{"@odata.context": "$metadata#Notification/$entity",
"AckId": "MTcxNTY5MDYxNTMzMy0wOjVlOGE3MTZlLTQ2N2UtNDJjZi04NDkzLTBiODk4NzU4NmEyZA==",
"NotificationDate": "2024-05-14T12:43:35.000Z",
"ProductId": "c0608f85-7a4a-41c1-a31b-2750c96f5902",
"ProductName": "S5P_NRTI_L2__AER_LH_20240514T113229_20240514T113729_34122_03_020600_20240514T123604.nc",
"SubscriptionEvent": "created",
"SubscriptionId": "5e8a716e-467e-42cf-8493-0b8987586a2d",
"value": {}}]
Listing current subscriptions
To list subscriptions which are currently associated with your account, send GET request to the following URL:
https://datahub.creodias.eu/odata/v1/Subscriptions/Info
REMINDER: Include the obligatory headers.
The status of the response should be 200 OK. You should get a list of your current subscriptions. This includes:
push subscriptions, as well as
pull subscriptions.
For example, it can look similarly to this:
[
{
"@odata.context": "$metadata#Subscriptions/$entity",
"FilterParam": "",
"Id": "6a52c01c-9c43-4624-875e-561a2752b710",
"LastNotificationDate": "2025-06-17T09:30:49.959Z",
"NotificationEndpoint": "https://example.com:8000",
"Priority": 1,
"StageOrder": false,
"Status": "running",
"SubmissionDate": "2024-05-29T10:01:05.793Z",
"SubscriptionEvent": [
"created"
]
},
{
"@odata.context": "$metadata#Subscriptions/$entity",
"FilterParam": "Collection/Name eq 'SENTINEL-5P' and Attributes/OData.CSC.StringAttribute/any(att:att/Name eq 'productType' and att/OData.CSC.StringAttribute/Value eq 'L2__AER_LH')",
"Id": "9b86baed-64c1-47cf-94ae-b586666da622",
"Priority": 1,
"StageOrder": false,
"Status": "paused",
"SubmissionDate": "2024-05-15T14:25:43.940Z",
"SubscriptionEvent": [
"created"
]
}
]
In this example, we see that the user has two subscriptions.
The first one is a running push subscription. It includes a key NotificationEndpoint - its value is the URL of the notification endpoint. Also, it has key LastNotificationDate which shows when was the last time that a notification was delivered successfully. The value for key FilterParam is in this case an empty string - it means that the user should receive notifications about all products published in the catalogue.
The second subscription is a pull subscription. You can learn that it is a pull subscription from the fact that it does not have key NotificationEndpoint. This time, the value of key FilterParam is not empty and contains a meaningful query for the catalogue which means that it should receive notifications about only a subset of products published in the catalogue.
As stated previously, push subscriptions are outside of scope of this article.
If you don’t currently have any subscriptions, the query should return an empty list:
[]
Deleting a subscription
Send the DELETE request to the URL below. In it, replace <<subscription_id>> with the ID of the subscription you want to delete.
https://datahub.creodias.eu/odata/v1/Subscriptions(<<subscription_id>>)
REMINDER: Include the obligatory headers.
You should get response status: 204 No Content.
Listing subscriptions (as explained in section Listing current subscriptions of this article) should show that this subscription no longer exists.
What To Do Next
Once you have received a notification, you might want to download that product. The ways of achieving this goal include:
Copying product from the /eodata folder on a virtual machine running on CREODIAS cloud: How to mount eodata using s3fs in Linux on CREODIAS. Value of key S3Path in a notification contains the location of that product within the EODATA repository.
Downloading product as explained in Using curl and wget to download EODATA products from CREODIAS. Value of key ProductId contains the ID of the product which you can use to perform the download as explained in that article.
These articles cover some of the available ways of processing EODATA: