Using Python to send EODATA Catalogue requests on CREODIAS
This article covers an example way of sending EODATA Catalogue requests described in the article EOData Catalogue API Manual on CREODIAS.
URLs used for sending these requests might cause problems when using with, say, curl command on Bash. For example, the following URL:
https://datahub.creodias.eu/odata/v1/Products?$filter=not contains(Name,'S2') and ContentDate/Start gt 2022-05-03T00:00:00.000Z and ContentDate/Start lt 2022-05-03T00:10:00.000Z
contains among others a dollar sign, spaces and brackets. They need to be appropriately escaped and/or encoded to make sure that the request is valid. For instance, Bash might interpret $filter as variable $filter when it is, in fact, part of URL.
In this article, we circumvent this problem by
pasting the URL to one file,
pasting the body of request to another file (if needed),
creating another file with Python code
and then executing it.
Prerequisites
No. 1 Installed Python environment
This article is written for simple Python environments running on Ubuntu 22.04 and Windows Server 2022. Other operating systems (such as Windows 11) and environments (such as PyCharm) might also work, but are outside of scope of this article.
The following articles contain sections on how to install Python:
If you are running Ubuntu, you can also use virtualenvwrapper: How to install Python virtualenv or virtualenvwrapper on CREODIAS
No. 2 Python library called requests
You need to install Python package called requests. You can do that using pip:
pip install requests
On Ubuntu 22.04, you can also install this library globally using apt:
sudo apt install python3-requests
What We Are Going To Cover
Sending GET requests
Sending POST requests
Sending GET requests
Most links found in API manuals are GET requests which do not have any body.
Create script named send_get.py and enter the following content to it:
import requests
from requests.utils import requote_uri
# Open URL from file
url = open('link1.txt').read()
# Encode URL
url_encoded = requote_uri(url)
# Remove unnecessasary characters from encoded URL
url_encoded_cleared = url_encoded.replace('%0A', '')
# Obtain and print the response
response = requests.get(url_encoded_cleared)
print(response.json())
Create file named link1.txt in the same folder as the script send_get.py which contains the link you want to access. This file can have only one URL.
Navigate to the folder in which both script and link1.txt file are located using your terminal and run the script:
python3 send_get.py
python send_get.py
Output should consist of the response from Catalogue API.
Example 1: OData API
Enter the link below to file link1.txt (note that this file must have exactly one URL):
https://datahub.creodias.eu/odata/v1/Products?$filter=contains(Name,'S1A') and ContentDate/Start gt 2022-05-03T00:00:00.000Z and ContentDate/Start lt 2022-05-21T00:00:00.000Z
This link searches for products which
contain phrase S1A in their name, and
have their starting sensing date is later than 2022-05-03T00:00:00.000Z and earlier than 2022-05-21T00:00:00.000Z.
It should return top 20 results.
Execute the script. In this example, the GNOME Terminal and gedit on Ubuntu 22.04 were used. Note that gedit describes this code as Python 2 even though actually Python 3 was used.
Note that screenshot above contains only the beginning of the response.
Example 2: Deleted Products ODATA API
Enter the link below to file link1.txt (note that this file must have exactly one URL):
https://datahub.creodias.eu/odata/v1/DeletedProducts?$filter=not contains(Name,'S1') and DeletionDate gt 2023-04-01T00:00:00.000Z and DeletionDate lt 2023-05-30T23:59:59.999Z&$orderby=DeletionDate
This link searches for deleted products which
have a name that does not contain phrase S1, and
have deletion date set to later than 2023-04-01T00:00:00.000Z and earlier than 2023-05-30T23:59:59.999Z
The results are sorted by deletion date. Top 20 results should be returned.
Execute the script:
Note that screenshot above contains only the beginning of the response.
Example 3: OpenSearch API
Enter the link below to file link1.txt (note that this file must have exactly one URL):
https://datahub.creodias.eu/resto/api/collections/Sentinel2/search.json?startDate=2021-07-01&completionDate=2021-07-31
This link searches for Sentinel-2 products which have 2021-07-01 and 2021-07-31 as their starting and finishing observation date, respectively.
Execute the script:
Note that screenshot above contains only the beginning of the response.
Sending POST requests
Create a file named send_post.py and enter the following code to it:
import requests
import json
from requests.utils import requote_uri
# Open URL from file
url = open('link2.txt').read()
# Open body from file and convert it to JSON object
query = open('body.json')
body = json.load(query)
# Encode URL
url_encoded = requote_uri(url)
# Remove unnecessasary characters from encoded URL
url_encoded_cleared = url_encoded.replace('%0A', '')
# Obtain and print the response
response = requests.post(url_encoded_cleared, json=body)
print(response.json())
Script presented above works similarly as the one presented in the previous section.
The URL should, however, be this time put to file link2.txt and not link1.txt.
Body of the request should be put in file body.json.
All files mentioned in this section should be in the same folder.
Navigate to that folder using your terminal and run Python code:
python3 send_post.py
python send_post.py
Output should consist of the response from Catalogue API.
Example: OData API
Let’s use link from section Query by list of /eodata/OData-API-Manual-for-Creodias-EODATA-Catalogue to search for multiple product
names at once:
https://datahub.creodias.eu/odata/v1/Products/OData.CSC.FilterList
Place this link to file link2.txt.
Put the body found in that article to file body.json:
{
"FilterProducts":
[
{"Name": "S1A_IW_GRDH_1SDV_20141031T161924_20141031T161949_003076_003856_634E.SAFE"},
{"Name": "S3B_SL_1_RBT____20190116T050535_20190116T050835_20190117T125958_0179_021_048_0000_LN2_O_NT_003.SEN3"},
{"Name": "xxxxxxxx.06.tar"}
]
}
Execute the script:
