Pull subscriptions to CREODIAS EODATA catalogue

Push and pull subscriptions allow you to receive notifications when products in EODATA catalogue (or a subset of such products) are added, modified, or deleted.

In this article, we discuss pull subscriptions. For each of these subscriptions, a cloud-based queue is created to store incoming notifications. Users can view, save and delete notifications from this queue.

There are also the so-called push subscriptions, which are a method of receiving notifications from the CREODIAS EODATA catalogue straight to an endpoint managed by the user. They are covered in Push subscriptions to CREODIAS EODATA catalogue.

Prerequisites

No. 1 Account

You need an active CREODIAS account https://new.cloudferro.com/.

No. 2 Ability to send HTTP requests

To follow this article, you need to have a method of sending the following types of HTTP requests:

• GET

• POST

• PATCH

• DELETE

including corresponding headers and bodies. You also need to be able to read responses to these requests.

This article will include Python in two optional examples. Aside from these, the content remains “software-neutral.” The focus of the article is not on Python itself, but on HTTP requests, which you can send using any software of your choice:

curl, wget, httpie, JavaScript Fetch API, Axios (Node.js), Netcat/nc, PowerShell Invoke-RestMethod, Go (net/http), Java (HttpClient) and so on.

No. 3 Running Python code (optional)

You should have some experience using Python in order to run the above mentioned examples. Basics of Python are outside of scope of this article.

Apart from that, you will need to have Python and requests module installed.

If you are using Ubuntu 24.04, execute

sudo apt install python3-requests

to install this module. This should also install Python interpreter if you don’t have it yet.

On Windows, you can follow this article How to install Python in Windows on CREODIAS. After that, install the requests module.

Other operating systems might also work, but adjustments might be required.

No. 4 Knowledge how to filter products

If you want to be notified about changes to only a subset of products (as opposed to receiving data for all products), you need to use the appropriate filter.

Filters used in this article are described in EOData Catalogue API Manual on CREODIAS (they are the same as the ones used for $filter parameter for OData).

• For filters regarding created and/or modified products, check section about Products OData endpoint in EODATA Catalogue API Manual article.

• For filters regarding deleted products, check section about DeletedProducts OData endpoint in EODATA Catalogue API Manual article.

Products and DeletedProducts endpoints differ when it comes to available options, for example, the DeletedProducts allows you to search by cause of deletion of EODATA products while for Products endpoint there is no equivalent option.

No. 5 Generated Keycloak token

To prove your identity while using pull subscriptions, you need a Keycloak token. Here is how to generate it: Using curl and wget to download EODATA products from CREODIAS

Keycloak token is valid only for a limited time; you can generate new token if the current one expires and you want to continue using the subscriptions API.

If you are using Python or Bash and two-factor authentication is enabled on your account, you can also generate Keycloak token automatically without having to provide six-digit TOTP code. Learn more:

• Use Bash to automate generating API tokens for accessing and downloading EODATA when 2FA is enabled on CREODIAS

• Use Python to automate generating API tokens for accessing and downloading EODATA when 2FA is enabled on CREODIAS

Introduction

What should a HTTP request contain?

You can create and manage your pull subscriptions by sending HTTP requests.

When sending HTTP requests described in this article, always use:

• Type of request (GET, POST, PATCH or DELETE)

• Headers

• URL

All these three parts of a HTTP request must be present together. Some HTTP requests must also include a JSON body.

Headers

Requests in this article should contain the following headers:

{
    "Content-Type": "application/json",
    "Authorization": "Bearer <<access_token>>"
}

where <<access_token>> needs to be replaced with Keycloak token from Prerequisite No. 5. This token will be used to prove your identity.

We will call them the obligatory headers and will refer to them as such throughout the article.

JSON data in respones

Some requests described in this article return JSON data. For the purposes of this article, we formatted them as multiple lines so that they are more legible.

Basic information about pull subscriptions

Pull subscription creates a queue in which your notifications will appear. Each of these notifications will announce an event occuring in the catalogue which fits the criteria specified during the creation of that subscription. This queue is stored on API servers.

Full or partial contents of that queue (depending on the circumstances) can be viewed by sending a HTTP request - it will be returned as JSON.

To remove a notification from the queue, you must acknowledge (ACK) it by sending an HTTP request. This request will also clear all notifications that were received chronologically before it.

The queue stores up to 100000 of such notifications. If there are more, the oldest notification or notifications will be removed to make space for the newer ones. Therefore, make sure to view and ACK notifications regularly.

Pull subscription entity description

Property name	Type	Description
Id	Guid	It is a universally unique identifier (UUID). The Id is a local identifier for a Subscription instance within the Catalogue, assigned upon Subscription creation. Example 9249dde5-4838-4a34-8925-bac8c1d53f09c
Status	Subscription Status enumeration	The allowed values of a subscription status are: running (subscription is active and sends notifications to the endpoint) paused (subscription does not send notifications to the endpoint) cancelled (setting this status suspends and deletes a subscription) The default value set to 'Status', if not provided by the user, is 'running'. Example running
SubscriptionEvent	Subscription Event enumeration	The subscription event to be monitored and for which a notification is provided. Can have the following values: created modified created, modified deleted The default value set to SubscriptionEvent, if not provided by the user, is "created". For "SubscriptionEvent" = "created" the notifications about newly added products to the EOData Catalogue will be sent. For "SubscriptionEvent" = "modified" the notifications about updated products in the EOData Catalogue will be sent. For "SubscriptionEvent" = "deleted" the notifications about removed products from the EOData Catalogue will be sent. This will happen if one of the following values of the "DeletionCause" parameter is met: Duplicated product Missing checksum Corrupted product Obsolete product or Other Example created
FilterParam	String	The filter parameters of the Subscription. If "SubscriptionEvent" = "created" or "modified", this query refers to the $filter= parameter of any Products? query. If "SubscriptionEvent" = "deleted", this query refers to $filter= parameter of any DeletedProducts query. The same filtering parameters as described for EOData Catalogue are available - see Prerequisite No. 3 for more details. Example Collection/Name eq 'SENTINEL-1' and Attributes/OData.CSC.StringAttribute/any (att:att/Name eq 'productType' and att/OData.CSC.StringAttribute/Value eq 'IW_SLC__1S')
SubmissionDate	DateTimeOffset	Date and time at which the Subscription was received by the Catalogue. Time is in ISO 8601 format (UTC+0): YYYY-MMDDThh:mm:ss.sssZ Example 2024-01-17T09:13:04.654Z
LastNotificationDate	DateTimeOffset	Date and time corresponding to the last time a notification from the Subscription was acked. Time is in ISO 8601 format (UTC+0) YYYY-MMDDThh:mm:ss.sssZ Example 2024-01-17T09:50:10.654Z
StageOrder	Boolean	Automatically orders the staging of products fulfilling the subscription. Only used if value of SubscriptionEvent is created. Currently, the order of staging products is not feasible as all products in EOData Catalogue have status set to Online and can be accessed immediately without setting order. Example true
Priority	Int64	Priority of the created orders resulting from the subscription. Currently automatically set to 1 without the possibility to change the value. Example 1

Limitations of subscriptions

Your account can only have up to 2 (two) running subscriptions at the same time. These can either be

• two pull subscriptions (which are described in this article), or

• two push subscriptions (which are described in Push subscriptions to CREODIAS EODATA catalogue), or

• a pull subscription and a push subscription.

The maximum total number of subscriptions associated with a CREODIAS account is 10. This includes subscriptions of all types:

• running and paused

• push and pull

Creating pull subscription

This is a typical body of request to create a pull subscription:

{
    "StageOrder": true,
    "FilterParam": "Collection/Name eq 'SENTINEL-1' and Attributes/OData.CSC.StringAttribute/any(att:att/Name eq 'productType' and att/OData.CSC.StringAttribute/Value eq 'IW_SLC__1S')",
    "Priority": 1,
    "Status": "running",
    "SubscriptionEvent": [
        "created"
    ]
}

Set up your own values to the following three keys in that JSON body:

SubscriptionEvent

Value of this key is an array containing only one string. With this string, you choose which types of events do you wanted to be notified about. It can have one of the following values:

• created

• modified

• created, modified

• deleted

FilterParam

Change the value of key FilterParam to query which defines subset of products about which you want to be notified. See Prerequiste No. 3 to learn more.

To get information

• about created and/or modified products, use the same query as you would send to Products endpoint

• about deleted products, use the same query as you would send to DeletedProducts endpoint

You can also get notifications about selected types of events associated with all products from the catalogue, not only a subset of them. To that end, you can either

• leave the value of key FilterParam empty, or

• do not include that key with your request at all.

Status

If you set value of this key to running, the created subscription will immediately start receiving notifications. If you, however, want to create a paused (inactive) subscription and activate it later, set this value to paused.

If you do not add this key to your request, the created subscription will have running status.

If you already have 2 running subscriptions and you attempt to create another subscription with the same status, this operation will fail. See section Limitations of subscriptions above for more details.

To resolve, you can either

• pause one of your current subscriptions and then create a new subscription, or

• create a new subscription with paused status.

REMINDER: Include the obligatory headers.

This is the URL to which you should send this request:

https://datahub.creodias.eu/odata/v1/Subscriptions

Response code to that request should be 201 Created. The response should contain JSON data like this:

{
    "Id": "991c4730-cf6f-432a-9f6c-47be0230ff45",
    "FilterParam": "Collection/Name eq 'SENTINEL-1' and Attributes/OData.CSC.StringAttribute/any(att:att/Name eq 'productType' and att/OData.CSC.StringAttribute/Value eq 'IW_SLC__1S')",
    "StageOrder": true,
    "Priority": 1,
    "Status": "running",
    "SubscriptionEvent": [
        "created"
    ],
    "SubmissionDate": "2024-03-13T09:39:49.404Z",
    "@odata.context": "$metadata#Subscriptions/$entity"
}

Example: Creating pull subscription using Python

Here is the code in Python that will send a request with the parameters described as above:

import requests
import json

access_token = ""

headers = {
    "Content-Type": "application/json",
    "Authorization": f"Bearer {access_token}",
}

body = {
    "StageOrder": True,
    "FilterParam": "Collection/Name eq 'SENTINEL-1' and Attributes/OData.CSC.StringAttribute/any(att:att/Name eq 'productType' and att/OData.CSC.StringAttribute/Value eq 'IW_SLC__1S')",
    "Priority": 1,
    "Status": "running",
    "SubscriptionEvent": [
        "created"
    ]
}

url = "https://datahub.creodias.eu/odata/v1/Subscriptions"

response = requests.post(url=url, headers=headers, json=body)

print(f"{response.status_code} {response.reason}\n")
try:
    print(json.dumps(json.loads(response.text), indent=4))
except Exception:
    print(response.text)

This script:

• Sends an HTTP request to the specified URL.

• Retrieves and displays the results, which consist of:

  • The response status code (e.g., 200) and reason (e.g., OK).

  • A blank lane (for readibility).

  • Response code, if present. (If it is a valid JSON, it will be formatted to make it more legible.)

Note that in this script, the request body is sent to the requests module using json parameter.

Assign Keycloak token obtained by following Prerequisite No. 4 to variable access_token.

Save the file as http_request_test_create.py

Ubuntu 24.04Windows

python3 http_request_test_create.py

Example: Creating a subscription to track created and modified products

In this example, we will create a subscription which tracks both created and modified products that meet the following criteria (FilterParam):

Collection/Name eq 'SENTINEL-1' and Attributes/OData.CSC.StringAttribute/any(att:att/Name eq 'productType' and att/OData.CSC.StringAttribute/Value eq 'IW_SLC__1S')

To do it, send POST HTTP request with the following body:

{
    "StageOrder": true,
    "FilterParam": "Collection/Name eq 'SENTINEL-1' and Attributes/OData.CSC.StringAttribute/any(att:att/Name eq 'productType' and att/OData.CSC.StringAttribute/Value eq 'IW_SLC__1S')",
    "Priority": 1,
    "Status": "running",
    "SubscriptionEvent": [
        "created, modified"
    ]
}

to this URL:

https://datahub.creodias.eu/odata/v1/Subscriptions

REMINDER: Include the obligatory headers.

You should get status code 201 Created and response like this:

{
    "@odata.context": "$metadata#OData.CSC.Subscription",
    "FilterParam": "Collection/Name eq 'SENTINEL-1' and Attributes/OData.CSC.StringAttribute/any(att:att/Name eq 'productType' and att/OData.CSC.StringAttribute/Value eq 'IW_SLC__1S')",
    "Id": "3de3bcf4-4990-4e7f-8d89-67d80deb378c",
    "Priority": 1,
    "StageOrder": false,
    "Status": "running",
    "SubmissionDate": "2025-03-18T17:46:01.709952Z",
    "SubscriptionEvent": [
        "created",
        "modified"
    ]
}

Example: Attempting to create a subscription with using invalid FilterParam

Let’s say that we accidentally attempted to create a subscription with an invalid query, for example, by using Colection/Name instead of Collection/Name:

Colection/Name eq 'SENTINEL-1' and Attributes/OData.CSC.StringAttribute/any(att:att/Name eq 'productType' and att/OData.CSC.StringAttribute/Value eq 'IW_SLC__1S')

The body of the request which we will send contains our invalid query:

{
    "StageOrder": true,
    "FilterParam": "Colection/Name eq 'SENTINEL-1' and Attributes/OData.CSC.StringAttribute/any(att:att/Name eq 'productType' and att/OData.CSC.StringAttribute/Value eq 'IW_SLC__1S')",
    "Priority": 1,
    "Status": "running",
    "SubscriptionEvent": [
        "created"
    ]
}

As previously, we send our request to the following URL:

https://datahub.creodias.eu/odata/v1/Subscriptions

As a response, we get response code 400 Bad Request and the following JSON data:

{
    "detail": "Your filter is incorrect. We suggest to verify it with catalogue API. Detailed error message -> Invalid field: Colection"
}

This response will vary depending on the type of error you make in your query.

Viewing the queue

To view the oldest notifications on a notification queue, send GET request to the following URL:

https://datahub.creodias.eu/odata/v1/Subscriptions(<<subscription_id>>)/Read$top=20

Replace <<subscription_id>> with the ID of the subscription.

The $top parameter specifies the number of oldest notifications to display, with a maximum limit of 20. If not provided, it defaults to 1.

If the number of notifications in your queue is smaller or the same as $top, you will get all notifications.

An example request here would be a GET request to the following URL:

https://datahub.creodias.eu/odata/v1/Subscriptions(991c4730-cf6f-432a-9f6c-47be0230ff45)/Read?$top=2

Depending on circumstances, the response will vary.

REMINDER: Include the obligatory headers.

Subscription is Cancelled

The process of cancelling a subscription actually deletes it. Therefore, if you attempt to view notification queue of such a subscription, you will get response code 404 Not Found with JSON like this:

{
    "detail": {
        "message": "Subscription with id 991c4730-cf6f-432a-9f6c-47be0230ff45 not found.",
        "request_id": "aed89ead-b495-4f76-b6e9-51870973a44e"
    }
}

Subscription is Active or Paused

If subscription is Active or Paused, you will get an array of JSON objects where each represents a notification. Response code should then be 200 OK.

Each of these objects contains information about

• the notification,

• the product, as well as

• the subscription.

Full content of a notification is kept on users’s queue for three days. After that time, keys value and ProductName are removed.

Example 1

The following downloadable JSON file

example-notification.json

contains a successful response for SubscriptionEvent set to created (status 200 OK).

The complete response is too long to be published verbatim in this article.

Here is what the value -> Attributes array looks like. It starts on line 67 and ends on 248 while the entire file has 293 lines; the red rectangle on the right of the image shows the position of the Attributes array proportionally to the length of the entire text.

Example 2

Here is an example of a successful response (status 200 OK) for SubscriptionEvent set to modified. The values omitted are:

• value -> GeoFootprint -> coordinates

• value -> Attributes

• value -> Locations

[
    {
        "@odata.context": "$metadata#Notification/$entity",
        "SubscriptionEvent": "modified",
        "ProductId": "b6801a5d-0e66-43d7-afac-1280f0895bf1",
        "ProductName": "S1A_IW_SLC__1SDV_20241123T044906_20241123T044930_056677_06F42F_97B3.SAFE",
        "SubscriptionId": "991c4730-cf6f-432a-9f6c-47be0230ff45",
        "NotificationDate": "2025-02-10T08:30:48.905365Z",
        "AckId": "MTcxMDc1NjcwNjUzMi0wOjk5MWM0NzMwLWNmNmYtNDMyYS05ZjZjLTQ3YmUwMjMwZmY0NQ==",
        "value": {
            "@odata.context": "$metadata#Products(Attributes(),Assets(),Locations())/$entity",
            "@odata.mediaContentType": "application/octet-stream",
            "Id": "b6801a5d-0e66-43d7-afac-1280f0895bf1",
            "Name": "S1A_IW_SLC__1SDV_20241123T044906_20241123T044930_056677_06F42F_97B3.SAFE",
            "ContentType": "application/octet-stream",
            "ContentLength": 8217756705,
            "OriginDate": "2024-11-23T06:28:41.550000Z",
            "Checksum": [
                {
                    "Value": "40fbe8ddd886f24a323906af1e442155",
                    "Algorithm": "MD5",
                    "ChecksumDate": "2024-11-23T06:37:33.947488Z"
                },
                {
                    "Value": "5c128944c135faefd8eb15b28f776541c1e4d0b21fa5064906b1882ca590527e",
                    "Algorithm": "BLAKE3",
                    "ChecksumDate": "2024-11-23T06:37:43.810075Z"
                }
            ],
            "ContentDate": {
                "Start": "2024-11-23T04:49:06.776458Z",
                "End": "2024-11-23T04:49:30.129633Z"
            },
            "Footprint": "geography'SRID=4326;POLYGON ((15.389621 5.429769, 15.680731 6.842505, 13.45454 7.296846, 13.16956 5.8881, 15.389621 5.429769))'",
            "GeoFootprint": {
                "type": "Polygon",
                "coordinates": []
            },
            "Attributes": [],
            "ModificationDate": "2025-02-10T08:30:48.665109Z",
            "PublicationDate": "2024-11-23T06:36:54.654519Z",
            "Online": true,
            "EvictionDate": "2025-03-12T08:28:16.409489Z",
            "S3Path": "/eodata/Sentinel-1/SAR/IW_SLC__1S/2024/11/23/S1A_IW_SLC__1SDV_20241123T044906_20241123T044930_056677_06F42F_97B3.SAFE",
            "Assets": [
                {
                    "Type": "QUICKLOOK",
                    "Id": "b43f60cb-3869-4c1d-b1e5-336e0d057f43",
                    "DownloadLink": "https://datahub.creodias.eu/odata/v1/Assets(f4cfe4c9-5b56-47c2-87ce-05b090c7a4ba)/$value",
                    "S3Path": "/eodata/Sentinel-1/SAR/IW_SLC__1S/2024/03/13/S1A_IW_SLC__1SDV_20240313T054626_20240313T054653_052959_066921_3759.SAFE"
                }
            ],
            "Locations": []
        }
    }
]

Example 3

This is an example of a successful response (status 200 OK) for SubscriptionEvent set to deleted.

[
    {
        "@odata.context": "$metadata#Notification/$entity",
        "SubscriptionEvent": "deleted",
        "ProductId": "232d796a-0552-4679-a1ed-b8cd3d002db2",
        "ProductName": "S1A_IW_SLC__1SDV_20241111T044907_20241111T044931_056502_06ED2E_7F0A",
        "SubscriptionId": "991c4730-cf6f-432a-9f6c-47be0230ff45",
        "NotificationDate": "2025-02-10T08:30:04.127132Z",
        "AckId": "MTcxMDc1NjcwNjUzMi0wOjk5MWM0NzMwLWNmNmYtNDMyYS05ZjZjLTQ3YmUwMjMwZmY0NQ==",
        "value": {
            "@odata.context": "$metadata#DeletedProducts(Attributes())/$entity",
            "@odata.mediaContentType": "application/image-tiff",
            "Id": "232d796a-0552-4679-a1ed-b8cd3d002db2",
            "Name": "S1A_IW_SLC__1SDV_20241111T044907_20241111T044931_056502_06ED2E_7F0A",
            "ContentType": "application/image-tiff",
            "ContentLength": 999999,
            "OriginDate": "2025-02-10T08:28:16.409494Z",
            "Checksum": [
                {
                    "Value": "1efb9cf89aa30086e114705c20831606",
                    "Algorithm": "MD5",
                    "ChecksumDate": "2025-02-10T08:28:16.409494Z"
                },
                {
                    "Value": "7a8e7280a1b37f14ecf6cf9216ec3f9e3824e5a7098d266168b8ce6b32598551",
                    "Algorithm": "BLAKE3",
                    "ChecksumDate": "2025-02-10T08:28:16.409494Z"
                }
            ],
            "Footprint": "geography'SRID=4326;POLYGON ((14.412463 5.424546, 15.703652 6.837446, 13.453771 7.296696, 13.168763 5.887819, 14.412463 5.424546))'",
            "GeoFootprint": {
                "type": "Polygon",
                "coordinates": []
                },
                "ContentDate": {
                    "Start": "2025-02-09T08:28:16.409468Z",
                    "End": "2025-02-10T08:28:16.409494Z"
                },
                "Attributes": [],
                "DeletionDate": "2025-02-10T08:30:03.907283Z",
                "DeletionCause": "Corrupted product"
            }
        }
    }
]

Example 4

An example of a successful response (status 200 OK) or notifications which are 3 days old or older:

[
    {
        "@odata.context": "$metadata#Notification/$entity",
        "SubscriptionEvent": "created",
        "ProductId": "d2ff986b-9454-43d6-95e0-1a0ae27019d7",
        "SubscriptionId": "991c4730-cf6f-432a-9f6c-47be0230ff45",
        "NotificationDate": "2024-03-13T13:39:59.000Z",
        "AckId": "MTcxMDc1NjcwNjUzMi0wOjk5MWM0NzMwLWNmNmYtNDMyYS05ZjZjLTQ3YmUwMjMwZmY0NQ=="
    },
    {
        "@odata.context": "$metadata#Notification/$entity",
        "SubscriptionEvent": "created",
        "ProductId": "33ad5694-9208-4001-ae4f-6d7cd0579ce0",
        "SubscriptionId": "991c4730-cf6f-432a-9f6c-47be0230ff45",
        "NotificationDate": "2024-03-13T13:39:59.000Z",
        "AckId": "MTcxMDc1NjcwNjY2NC0wOjk5MWM0NzMwLWNmNmYtNDMyYS05ZjZjLTQ3YmUwMjMwZmY0NQ=="
    }
]

Example 5

If you do not have any notifications in the queue, you should get status 200 OK and an empty array:

[]

Explanation of keys in a notification

Property name	Type	Description
ProductId	Guid	It is a universally unique identifier (UUID) of the product being notified. Example 8d654e59-d7b6-4a53-b086-c9de764e506b
ProductName	String	It is the name of the product being notified assigned within the Catalogue. The value of ProductName can be lengthy and will not usually fit into a single table cell; the concrete example is provided below the table.
SubscriptionId	Guid	It is a universally unique identifier (UUID). The Id is a local identifier for the Subscription instance within the Catalogue, assigned upon Subscription creation. Example 991c4730-cf6f-432a-9f6c-47be0230ff4
SubscriptionEvent	Subscription Event enumeration	The subscription event to be monitored and for which notification is provided: created modified created, modified deleted The default value set to SubscriptionEvent, if not provided by the user, is "created". For "SubscriptionEvent" = "created" the notifications about newly added products to the EOData Catalogue will be sent. For "SubscriptionEvent" = "modified" the notifications about updated products in the EOData Catalogue will be sent. For "SubscriptionEvent" = "deleted" the notifications about removed products from the EOData Catalogue will be sent. Example created
NotificationDate	DateTimeOffset	Date and time at which the notification was generated. Time is in ISO 8601 format (UTC+0): YYYYMM-DDThh:mm:ss.sssZ Example 2024-01-29T10:59:08.698Z
AckId	String	Acknowledge Id assigned to each product notification. Required for acknowledging the notification messages from the client's queue. Because the AckId is long, its example is below this table.
value	JSON object	It contains a standard Catalogue API response with all metadata and content details about a product.

Example of ProductName

S2A_MSIL2A_20240129T062121_N0510_R034_T44VMN_20240129T093752.SAFE

Example of AckID

MTcxMDc1NjcwNjUzMi0wOjk5MWM0NzMwLWNmNmYtNDMyYS05ZjZjLTQ3YmUwMjMwZmY0NQ==

Acking notifications

Acknowledging (acking) a notification removes (from the queue) both that notification and all of the notifications that came chronologically before it.

To do it, send POST request to the URL below. Replace:

• <<subscription_id>> with the ID of your subscription

• <<AckId>> with value of key AckId for your notification found in the notification queue

https://datahub.creodias.eu/odata/v1/Subscriptions(<<subscription_id>>)/Ack?$ackid=<<AckID>>

REMINDER: Include the obligatory headers.

For example, this could be a POST request to the following URL:

https://datahub.creodias.eu/odata/v1/Subscriptions(991c4730-cf6f-432a-9f6c-47be0230ff45)/Ack?$ackid=MTcxMDc1NjcwNjY2NC0wOjk5MWM0NzMwLWNmNmYtNDMyYS05ZjZjLTQ3YmUwMjMwZmY0NQ==

You should get response 200 OK. The response should also contain JSON like this:

{
     "@odata.context": "$metadata#Notification/$entity",
     "AckMessagesNum": 2,
     "CurrentQueueLength": 2115,
     "MaxQueueLength": 100000
}

which contains the following information:

Key	Type	Description
AckMessagesNum	Int64	The number of the notifications that were acknowledged by your request.
CurrentQueueLength	Int64	Number of notifications currently in your queue.
MaxQueueLength	Int64	The maximum number of notifications which can be stored in your queue

If the subscription was previously Cancelled, you should get response code 404 Not Found and JSON like this:

{
    "detail": {
        "message": "Subscription with id 991c4730-cf6f-432a-9f6c-47be0230ff4 not found.",
        "request_id": "ae19864a-5bda-4893-9f02-a62eba95fd7a"
    }
}

Using ack to read further notifications

Request used to view the queue can only return up to 20 oldest notifications. The maximum number of notifications which can exist in a queue is 100000. Therefore, that request might be insufficient to fetch all notifications.

To read the remaining messages, you can use ack:

• View the current list of notifications - set $top parameter to 20.

• Save these notifications if needed.

• Ack the last of them.

Once again, have a look at the current list of notifications - it should now contain notifications which came after the notification you acked. If there are more than 20 notifications in your queue, you should now see notifications which you did not see previously.

Once again, save notifications if needed and ack the last of them.

Do so until your queue is empty.

Changing pull subscription status

For pull subscription, you can only change its status.

A subscription can have one of the following statuses:

• running - the subscription receives new notifications

• paused - the subscription temporarily does not receive new notifications

• cancelled - setting this status suspends and deletes a subscription

To edit status of the subscription, send HTTP PATCH request with the following body (replace <<new_status>> with the status you want your subscription to set to):

{
    "Status": "<<new_status>>"
}

REMINDER: Include the obligatory headers.

Send this request to the URL below (replace <<subscription_id>> with the ID of the subscription you want to edit).

https://datahub.creodias.eu/odata/v1/Subscriptions(<<subscription_id>>)

You should get a response with updated information about your subscription, alongside with the response code 200 OK. The exception is setting status to cancelled - this should give response code 204 No Content.

Example: Non-destructive changing the status of a pull subscription

If the ID of your subscription is 7ca682c3-7b21-4e9b-952e-874798181340 and you want to set its status to paused, use the following body for your POST HTTP request:

{
    "Status": "paused"
}

This is the URL:

https://datahub.creodias.eu/odata/v1/Subscriptions(7ca682c3-7b21-4e9b-952e-874798181340)

REMINDER: Include the obligatory headers.

Example: Setting subscription status to cancelled

Let’s say that we want to delete subscription from example above by setting its status to cancelled.

For this purpose, we will use the following JSON object as a body of our HTTP PATCH request:

{
   "Status": "cancelled"
}

The URL remains the same:

https://datahub.creodias.eu/odata/v1/Subscriptions(7ca682c3-7b21-4e9b-952e-874798181340)

We get response code 204 No Content. Our subscription should now be deleted.

Listing current subscriptions

To list all subscriptions currently associated with your account, send GET request to the following URL:

https://datahub.creodias.eu/odata/v1/Subscriptions/Info

REMINDER: Include the obligatory headers.

The status of the response should be 200 OK. You should get a list of your current subscriptions. This includes:

• pull subscriptions, as well as

• push subscriptions.

Here is an example:

[
  {
    "Id": "0a449cc9-d6d4-4db7-af6a-d10105bedd2b",
    "FilterParam": "Collection/Name eq 'SENTINEL-2' and Attributes/OData.CSC.StringAttribute/any(att:att/Name eq 'productType' and att/OData.CSC.StringAttribute/Value eq 'S2MSI2A')",
    "StageOrder": false,
    "Priority": 1,
    "Status": "paused",
    "NotificationEndpoint": "https://example.com/notification",
    "SubscriptionEvent": [
      "created"
    ],
    "SubmissionDate": "2025-04-10T13:11:11.203179Z",
    "@odata.context": "$metadata#OData.CSC.Subscription"
  },
  {
    "Id": "e9d0b04b-cb58-4963-814c-e6d81be4219e",
    "FilterParam": "Collection/Name eq 'SENTINEL-1' and Attributes/OData.CSC.StringAttribute/any(att:att/Name eq 'productType' and att/OData.CSC.StringAttribute/Value eq 'IW_SLC__1S')",
    "StageOrder": false,
    "Priority": 1,
    "Status": "running",
    "SubscriptionEvent": [
      "created",
      "modified"
    ],
    "SubmissionDate": "2025-04-10T13:13:51.988546Z",
    "@odata.context": "$metadata#OData.CSC.Subscription"
  }
]

In this example, we see that the user has two subscriptions:

• A push subscription which tracks created products

• A pull subscription which tracks both created and modified products

A push subscription, contrary to a pull subscription, includes key NotificationEndpoint which shows URL of endpoint to which the notifications are to be delivered. You can learn more about this type of subscriptions here: Push subscriptions to CREODIAS EODATA catalogue

If you don’t currently have any subscriptions, the query should return an empty array:

[]

Example: Listing subscriptions using Python

Here is a complete Python code for listing subscriptions:

import requests
import json

access_token = ""

headers = {
    "Content-Type": "application/json",
    "Authorization": f"Bearer {access_token}",
}

url = "https://datahub.creodias.eu/odata/v1/Subscriptions/Info"

response = requests.get(url=url, headers=headers)

print(f"{response.status_code} {response.reason}\n")
try:
    print(json.dumps(json.loads(response.text), indent=4))
except Exception as e:
    print(e)
    print(response.text)

This script:

• Sends an HTTP request to the specified URL.

• Retrieves and displays the results, which consist of:

  • The response status code (e.g., 200) and reason (e.g., OK).

  • A blank lane (for readibility).

  • Response code, if present. (If it is a valid JSON, it will be formatted to make it more legible.)

Assign Keycloak token obtained by following Prerequisite No. 4 to variable access_token.

Save the file as http_request_test_list.py

Ubuntu 24.04Windows

python3 http_request_test_list.py

Deleting a subscription

Method 1: Send a DELETE HTTP request

Send a DELETE request to the URL below. In it, replace <<subscription_id>> with the ID of the subscription you want to delete.

https://datahub.creodias.eu/odata/v1/Subscriptions(<<subscription_id>>)

REMINDER: Include the obligatory headers.

You should get response status: 204 No Content.

Listing subscriptions (as explained in section Listing current subscriptions of this article) should show that this subscription no longer exists.

Method 2: Change subscription status to cancelled

This process is described in section Changing pull subscription status of this article.

Pull subscription - Example workflow

In this section, we will cover an example worfkflow with a pull subscription.

Creating a subscription

Let’s begin by listing subscriptions associated with our account.

We do that by sending GET request to the following URL:

https://datahub.creodias.eu/odata/v1/Subscriptions/Info

REMINDER: Include the obligatory headers.

This request returns status code 200 OK and an empty array. This means that our account does not have any subscriptions yet:

[]

We now create our first subscription. It will track products being added to the catalogue. We will use the following filter:

Collection/Name eq 'SENTINEL-1' and Attributes/OData.CSC.StringAttribute/any(att:att/Name eq 'productType' and att/OData.CSC.StringAttribute/Value eq 'IW_SLC__1S') and Online eq true and Attributes/OData.CSC.IntegerAttribute/any(att:att/Name eq 'orbitNumber' and att/OData.CSC.IntegerAttribute/Value gt 0) and Attributes/OData.CSC.IntegerAttribute/any(att:att/Name eq 'orbitNumber' and att/OData.CSC.IntegerAttribute/Value lt 100000) and OData.CSC.Intersects(area=geography'SRID=4326;POLYGON((-180.0000 90.0000,180.00 90.00,180 -90,-180.0 -90.0,-180.0000 90.0000))')

This query tracks products which:

• Belong to SENTINEL-2 collection

• Have the following product type: IW_SLC__1S

• Have Online set to true

• Have orbitNumber greater than 0

• Have orbitNumber smaller than 100000

• Intersect the following polygon: POLYGON((-180.0000 90.0000,180.00 90.00,180 -90,-180.0 -90.0,-180.0000 90.0000))

This is the body of request we send to achieve this goal:

{
    "StageOrder": true,
    "FilterParam": "Collection/Name eq 'SENTINEL-1' and Attributes/OData.CSC.StringAttribute/any(att:att/Name eq 'productType' and att/OData.CSC.StringAttribute/Value eq 'IW_SLC__1S') and Online eq true and Attributes/OData.CSC.IntegerAttribute/any(att:att/Name eq 'orbitNumber' and att/OData.CSC.IntegerAttribute/Value gt 0) and Attributes/OData.CSC.IntegerAttribute/any(att:att/Name eq 'orbitNumber' and att/OData.CSC.IntegerAttribute/Value lt 100000) and OData.CSC.Intersects(area=geography'SRID=4326;POLYGON((-180.0000 90.0000,180.00 90.00,180 -90,-180.0 -90.0,-180.0000 90.0000))')",
    "Priority": 1,
    "Status": "running",
    "SubscriptionEvent": [
        "created"
    ]
}

This body contains information about our subscription, such as:

FilterParam

Query to the catalogue which filter products.

Status

Status of the subscription being created (we want the subscription to start operating right after its creation, so we set it to running).

SubscriptionEvent

What type of changes in the Catalogue do you want to track. For more information, see table in section Pull subscription entity description of this article. In this example, we chose created because we want to get notifications about products being created.

REMINDER: Include the obligatory headers.

We send this request to this URL:

https://datahub.creodias.eu/odata/v1/Subscriptions

Upon sending this request, we get status code 201 Created and the following JSON object:

{
    "Id": "32ee6df9-3008-4279-92e6-200828e532d1",
    "FilterParam": "Collection/Name eq 'SENTINEL-1' and Attributes/OData.CSC.StringAttribute/any(att:att/Name eq 'productType' and att/OData.CSC.StringAttribute/Value eq 'IW_SLC__1S') and Online eq true and Attributes/OData.CSC.IntegerAttribute/any(att:att/Name eq 'orbitNumber' and att/OData.CSC.IntegerAttribute/Value gt 0) and Attributes/OData.CSC.IntegerAttribute/any(att:att/Name eq 'orbitNumber' and att/OData.CSC.IntegerAttribute/Value lt 100000) and OData.CSC.Intersects(area=geography'SRID=4326;POLYGON((-180.0000 90.0000,180.00 90.00,180 -90,-180.0 -90.0,-180.0000 90.0000))')",
    "StageOrder": false,
    "Priority": 1,
    "Status": "running",
    "SubscriptionEvent": [
      "created"
    ],
    "SubmissionDate": "2025-03-26T12:19:03.492271Z",
    "@odata.context": "$metadata#OData.CSC.Subscription"
}

This JSON object confirms details about our new subscription, such as query used to filter products and type of events being tracked.

For interacting with this subscription, we will later need its ID, which in this example is 32ee6df9-3008-4279-92e6-200828e532d1.

This output also contains confirmation that the Status of our subscription is running which means that it should already be able to receive notifications.

To double check that this subscription was created, we once again list subscriptions associated with our account by sending the following GET request to the following URL:

https://datahub.creodias.eu/odata/v1/Subscriptions/Info

REMINDER: Include the obligatory headers.

The response code is once again 200 OK. The data received is an array with our objects, each of which represents one subscription.

[
    {
        "Id": "32ee6df9-3008-4279-92e6-200828e532d1",
        "FilterParam": "Collection/Name eq 'SENTINEL-1' and Attributes/OData.CSC.StringAttribute/any(att:att/Name eq 'productType' and att/OData.CSC.StringAttribute/Value eq 'IW_SLC__1S') and Online eq true and Attributes/OData.CSC.IntegerAttribute/any(att:att/Name eq 'orbitNumber' and att/OData.CSC.IntegerAttribute/Value gt 0) and Attributes/OData.CSC.IntegerAttribute/any(att:att/Name eq 'orbitNumber' and att/OData.CSC.IntegerAttribute/Value lt 100000) and OData.CSC.Intersects(area=geography'SRID=4326;POLYGON((-180.0000 90.0000,180.00 90.00,180 -90,-180.0 -90.0,-180.0000 90.0000))')",
        "StageOrder": false,
        "Priority": 1,
        "Status": "running",
        "SubscriptionEvent": [
            "created"
        ],
        "SubmissionDate": "2025-03-26T12:19:03.492271Z",
        "@odata.context": "$metadata#OData.CSC.Subscription"
    }
]

Since we now have one subscription, there is one object. It includes detailed information about our subscription, which is identical to output when it was created.

Should we ever forget something about our subscription (such as its ID or query used), we can execute this command to retrieve it.

Obviously, while you can have only one active subscription, you can have multiple paused subscriptions. This request should list them all, if they exist.

Acking one notification

Let’s now check the notification queue associated with this subscription. To do that, we send GET request to the following URL:

https://datahub.creodias.eu/odata/v1/Subscriptions(32ee6df9-3008-4279-92e6-200828e532d1)/Read$top=20

This URL contains data such as:

• ID of our subscription (32ee6df9-3008-4279-92e6-200828e532d1)

• maximum number of notifications we want to see (20, this value can’t be higher)

REMINDER: Include the obligatory headers.

This request returns response code 200 OK and an empty array as data:

[]

This means that our queue does not have any notifications yet.

We wait some time until the first notification appears.

After that, we send the same HTTP request again. This time, we also get response code 200 OK. The array we get as a response now contains one notification.

(Value of JSON key value was trimmed - made empty - for legibility purposes.)

[
    {
        "@odata.context": "$metadata#Notification/$entity",
        "SubscriptionEvent": "created",
        "ProductId": "f2587c70-1104-4e92-960b-640495331d2e",
        "ProductName": "S1A_IW_SLC__1SDV_20250326T101553_20250326T101621_058474_073BCB_A673.SAFE",
        "SubscriptionId": "32ee6df9-3008-4279-92e6-200828e532d1",
        "NotificationDate": "2025-03-26T12:20:43.000000Z",
        "AckId": "MTc0Mjk5MTY0Mzg5MC0wOjMyZWU2ZGY5LTMwMDgtNDI3OS05MmU2LTIwMDgyOGU1MzJkMQ==",
        "value": {
        }
    }
]

This notification announces publication of product named:

S1A_IW_SLC__1SDV_20250326T101553_20250326T101621_058474_073BCB_A673.SAFE

For future use (eg. processing), we save information that interests us from this notification.

Once we have done it, this notification is no longer needed and we want to delete it by acking (acknowledging) it.

For this purpose, we need the AckId of the notification, which in this example is:

MTc0Mjk5MTY0Mzg5MC0wOjMyZWU2ZGY5LTMwMDgtNDI3OS05MmU2LTIwMDgyOGU1MzJkMQ==

To Ack this notification, we send POST request to the URL below. In this URL, we are using the ID of the subscription, alongside with that AckId:

https://datahub.creodias.eu/odata/v1/Subscriptions(32ee6df9-3008-4279-92e6-200828e532d1)/Ack?$ackid=MTc0Mjk5MTY0Mzg5MC0wOjMyZWU2ZGY5LTMwMDgtNDI3OS05MmU2LTIwMDgyOGU1MzJkMQ==

REMINDER: Include the obligatory headers.

Response code returned is 200 OK. JSON data fetched by this request contain information about results of this operation:

{
    "@odata.context": "$metadata#Notification/$entity",
    "AckMessagesNum": 1,
    "CurrentQueueLength": 0,
    "MaxQueueLength": 100000
}

From this JSON object we learn that

• this Ack removed 1 notification (AckMessagesNum)

• 0 notifications remains in the queue (CurrentQueueLength)

It also contains a reminder that maximum number of notifications stored in a queue is 100000 (value of key MaxQueueLength).

We once again check the current notification queue by sending the GET request to this URL:

https://datahub.creodias.eu/odata/v1/Subscriptions(32ee6df9-3008-4279-92e6-200828e532d1)/Read$top=20

REMINDER: Include the obligatory headers.

The request returns response code 200 OK and an empty JSON array as data:

[]

Previously, we had one notification. We removed that notification by acking it and so now the list of notifications is empty.

Acking multiple notifications at once

We now wait some time until our queue gets populated with more notifications.

After that, we check the notification queue by sending the GET request to the same URL as before:

https://datahub.creodias.eu/odata/v1/Subscriptions(32ee6df9-3008-4279-92e6-200828e532d1)/Read$top=20

REMINDER: Include the obligatory headers.

We get response code 200 OK and three notifications. Note that, for legibility, value of key value was trimmed.

[
    {
        "@odata.context": "$metadata#Notification/$entity",
        "SubscriptionEvent": "created",
        "ProductId": "ea5e8c98-a3bd-4cab-86c9-7ea48f21f356",
        "ProductName": "S1A_IW_SLC__1SDV_20250326T101258_20250326T101325_058474_073BCA_A4FE.SAFE",
        "SubscriptionId": "32ee6df9-3008-4279-92e6-200828e532d1",
        "NotificationDate": "2025-03-26T12:21:27.000000Z",
        "AckId": "MTc0Mjk5MTY4NzE1MS0wOjMyZWU2ZGY5LTMwMDgtNDI3OS05MmU2LTIwMDgyOGU1MzJkMQ==",
        "value": {
        }
    },
    {
        "@odata.context": "$metadata#Notification/$entity",
        "SubscriptionEvent": "created",
        "ProductId": "9ab913a7-0445-4466-9825-90a36c9541e7",
        "ProductName": "S1A_IW_SLC__1SDV_20250326T101323_20250326T101350_058474_073BCA_EAEE.SAFE",
        "SubscriptionId": "32ee6df9-3008-4279-92e6-200828e532d1",
        "NotificationDate": "2025-03-26T12:21:41.000000Z",
        "AckId": "MTc0Mjk5MTcwMTQ5OC0wOjMyZWU2ZGY5LTMwMDgtNDI3OS05MmU2LTIwMDgyOGU1MzJkMQ==",
        "value": {
        }
    },
    {
        "@odata.context": "$metadata#Notification/$entity",
        "SubscriptionEvent": "created",
        "ProductId": "a05f5274-cad5-451c-b1d3-b86017251c57",
        "ProductName": "S1A_IW_SLC__1SDV_20250326T101348_20250326T101415_058474_073BCA_6916.SAFE",
        "SubscriptionId": "32ee6df9-3008-4279-92e6-200828e532d1",
        "NotificationDate": "2025-03-26T12:21:59.000000Z",
        "AckId": "MTc0Mjk5MTcxOTU3MS0wOjMyZWU2ZGY5LTMwMDgtNDI3OS05MmU2LTIwMDgyOGU1MzJkMQ==",
        "value": {
        }
    }
]

Each of these three notifications represents one product. They are ordered chronologically, from the earliest timestamp on which the notification was received (NotificationDate) to the latest.

Let’s say that we want to Ack the second notification which was sent on the following timestamp: 2025-03-26T12:21:41.000000Z

Its AckId is

MTc0Mjk5MTcwMTQ5OC0wOjMyZWU2ZGY5LTMwMDgtNDI3OS05MmU2LTIwMDgyOGU1MzJkMQ==

To perform this operation, we send POST request to this URL:

https://datahub.creodias.eu/odata/v1/Subscriptions(32ee6df9-3008-4279-92e6-200828e532d1)/Ack?$ackid=MTc0Mjk5MTY0Mzg5MC0wOjMyZWU2ZGY5LTMwMDgtNDI3OS05MmU2LTIwMDgyOGU1MzJkMQ==

REMINDER: Include the obligatory headers.

The request returns response code 200 OK and JSON data which summarizes what happened as a result of this request:

{
    "@odata.context": "$metadata#Notification/$entity",
    "AckMessagesNum": 2,
    "CurrentQueueLength": 1,
    "MaxQueueLength": 100000
}

Acking a notification removes both that notification as well as all of the notifications that came before it. In this case, there was one older notification than the one being acked so both were removed. Two notifications were removed in total - that is what value of key AckMessagesNum is showing.

In our queue there was also a third notification, which survives this process because it appeared chronologically later than the notification being acked. This notification is counted in value of key CurrentQueueLength.

We can view that remaining notification by once again checking the notification queue. We send GET request to this URL:

https://datahub.creodias.eu/odata/v1/Subscriptions(32ee6df9-3008-4279-92e6-200828e532d1)/Read$top=20

REMINDER: Include the obligatory headers.

We get response code 200 OK and an array with our one remaining notification (again, for legibility purposes, value of key value was trimmed):

[
    {
        "@odata.context": "$metadata#Notification/$entity",
        "SubscriptionEvent": "created",
        "ProductId": "a05f5274-cad5-451c-b1d3-b86017251c57",
        "ProductName": "S1A_IW_SLC__1SDV_20250326T101348_20250326T101415_058474_073BCA_6916.SAFE",
        "SubscriptionId": "32ee6df9-3008-4279-92e6-200828e532d1",
        "NotificationDate": "2025-03-26T12:21:59.000000Z",
        "AckId": "MTc0Mjk5MTcxOTU3MS0wOjMyZWU2ZGY5LTMwMDgtNDI3OS05MmU2LTIwMDgyOGU1MzJkMQ==",
        "value": {
        }
    }
]

Its AckId is:

MTc0Mjk5MTcxOTU3MS0wOjMyZWU2ZGY5LTMwMDgtNDI3OS05MmU2LTIwMDgyOGU1MzJkMQ==

To ack that notification as well, we send POST request to this URL:

https://datahub.creodias.eu/odata/v1/Subscriptions(32ee6df9-3008-4279-92e6-200828e532d1)/Ack?$ackid=MTc0Mjk5MTcxOTU3MS0wOjMyZWU2ZGY5LTMwMDgtNDI3OS05MmU2LTIwMDgyOGU1MzJkMQ==

REMINDER: Include the obligatory headers.

The request once again returns response code 200 OK and results of this operation as JSON data:

{
    "@odata.context": "$metadata#Notification/$entity",
    "AckMessagesNum": 1,
    "CurrentQueueLength": 0,
    "MaxQueueLength": 100000
}

From that JSON object we can learn that this was the last notification from our queue. To confirm that it is empty, we check our notification queue again. We send GET request to this URL:

https://datahub.creodias.eu/odata/v1/Subscriptions(32ee6df9-3008-4279-92e6-200828e532d1)/Read$top=20

REMINDER: Include the obligatory headers.

Response code 200 OK is returned alongside an empty JSON:

[]

Since we deleted our only notification from that queue, the queue is indeed empty.

Deleting subscription

Let’s now assume that after some time we no longer need our subscription and want to remove it.

To do that, we send a DELETE request to the following URL:

https://datahub.creodias.eu/odata/v1/Subscriptions(32ee6df9-3008-4279-92e6-200828e532d1)

REMINDER: Include the obligatory headers.

Response code 204 No Content is returned (with no JSON data included).

To check the current list of subscriptions, we send GET request to this URL:

https://datahub.creodias.eu/odata/v1/Subscriptions/Info

REMINDER: Include the obligatory headers.

We get response code 200 OK and an empty array:

[]

This confirms that no subscriptions remain on our account.

What To Do Next

Once you have received a notification, you might want to download that product. The approaches to reaching this objective include:

• Copying product from /eodata folder on a virtual machine running on CREODIAS cloud: How to mount eodata using s3fs in Linux on CREODIAS. Value of key S3Path in a notification contains the location of that product within the EODATA repository.

• Downloading product as explained in Using curl and wget to download EODATA products from CREODIAS. Value of key ProductId contains the ID of the product which you can use to perform the download as explained in that article.

These articles cover some of the available ways of processing EODATA:

• Analyzing and monitoring floods using Python and Sentinel-2 satellite imagery on CREODIAS

• Processing Sentinel-5P data on air pollution using Jupyter Notebook on CREODIAS

• Sample Deep Learning workflow using vGPU and EO DATA on CREODIAS