CUTTING EDGE

Note

Availability of Cutting Edge capabilities and articles across the Creodias regions:

WAW3-1	Available now
WAW3-2	Available now
FRA1-2	Available now
CF-2	Not available

• Install TensorFlow on Docker Running on CREODIAS vGPU Virtual Machine
  • Prerequisites
  • What We Are Going To Cover
  • Step 1: Update software on your VM and verify that the NVIDIA graphics card is working
  • Step 2: Install Docker
  • Step 3: Install and verify the NVIDIA Container Toolkit (ndivia-docker2)
  • Step 4 Install TensorFlow with vGPU support
  • What To Do Next
• Sample Deep Learning Workflow Using TensorFlow Running on Docker on CREODIAS vGPU Virtual Machine
  • What We Are Going To Do
  • Prerequisites
  • Code Explanation
    • Step 1: Data Preparation
    • Step 2: Defining and Training of the Model
    • Step 3: Generating Predictions for the Test Data
    • Step 4: Summarizing the Results
  • Practical Workflow
    • Step 1: Download the resources to your virtual machine
    • Step 2: Move the downloaded resources to the appropriate location
    • Step 3: Create a Dockerfile and file requirements.txt
    • Step 4: Unzip and delete the archive containing the training data
    • Step 5: Verify That All the Needed Files are in the Correct Place
    • Step 6: Build a Docker Image and Enter it
    • Step 7: Run the Python code
    • Step 8: Extract the Files From the Container
    • Step 9: Download the File with the Results to Your Local Machine
  • Performance Comparison
  • What Can Be Done Next
• Install TensorFlow on vGPU enabled VM on CREODIAS
  • What Are We Going To Do
  • Prerequisites
  • Step 1 Verify the availability of the toolkit
  • Step 2 Install cuDNN
  • Step 3 Install TensorFlow
  • What To Do Next
• Sample Deep Learning workflow using vGPU and EO DATA on CREODIAS
  • What We Are Going To Do
  • The Prerequisites
  • Code Explanation
    • Step 1: Data Preperation
    • Step 2: Defining and Training of the Model
    • Step 3: Generating Predictions for the Test Data
    • Step 4: Summarizing the Results
  • Practical Workflow
    • Step 1: Loading and Formatting the Data
    • Step 2: Copy the downloaded files to your virtual machine
    • Step 3: Install additional required packages
    • Step 4: Prepare the files
    • Step 5: Execute the file with Python code
    • Step 6: Download the results and the trained model to your local computer
  • Performance Comparison
  • What Can Be Done Next
• Sample SLURM Cluster on CREODIAS Cloud with ElastiCluster
  • What We Are Going To Cover
  • Prerequisites
  • Always use the latest value of image id
  • Preparation Step 1 Create an OpenStack Keystone User
  • Preparation Step 2 Create a New Key Pair
  • Preparation Step 3 Create Security Group
  • Preparation Step 4 How to set up Python and virtual env
  • Preparation Step 5 Source OpenStack RC File
  • Preparation Step 6 Install ElastiCluster
  • Installation Step 1 Create ~/.elasticluster/config
  • Installation Step 2 How to Enter Correct Values into ElastiCluster Config Template
  • Installation Step 3 Run and verify cluster setup
  • Step 4 Troubleshooting and Debugging
  • What To Do Next
• Sample Workflow: Running EO Processing MPI jobs on a SLURM Cluster on CREODIAS Cloud
  • What We Are Going To Cover
  • Prerequisites
  • Step 1 Install mpi4py
  • Step 2 Install snappy
  • Step 3 Add EO Data Capabilities to the Cluster
  • Step 4 Install Additional Python Modules
  • Step 5 Run an Image Processing MPI Job
  • What To Do Next
• How to Prepare and Use Open Data Cube on CREODIAS Cloud
  • What We Are Going To Cover
  • Prerequisites
  • Step 1 Create virtual machine for Open Data Cube
  • Step 2 Add a security group for Jupiter Lab
  • Step 3 Add floating IP to connect to your instance
  • Step 4 Connect to the VM using SSH
  • Step 5 Prepare Sentinel Hub credentials
  • Step 6 Test Data Cube and Sentinel Hub integration
  • Step 7 Connect to jupyter-lab and test the integration
  • Troubleshooting
• How to Login to Data Explorer on CREODIAS Cloud
  • What We Are Going To Cover
  • Prerequisites
  • Step 1 Open the Data Explorer website
  • Step 2 Log in to Data Explorer
  • What To Do Next
• How to Download a Single Product Using Data Explorer on CREODIAS Cloud
  • What We Are Going To Cover
  • Prerequisites
  • Step 1: Find a product that you are interested in
  • Step 2: Downloading a product
  • What To Do Next
• Processing Products with Data Explorer on CREODIAS Cloud
  • What We Are Going To Cover
  • Prerequisites
  • Step 1 Finding a product ready for processing
  • Step 2 Adding a product to your workspace
  • Step 3 Queuing up your products for processing
  • What To Do Next
• Processing Sentinel-5P data on air pollution using Jupyter Notebook on CREODIAS
  • Prerequisites
  • PART #1 - Download and prepare data
  • PART #2 - Conversion and zonal statistics