How to mount object storage container from CREODIAS as file system on local Windows computer
This article describes how to configure direct access to object storage containers from CREODIAS cloud in This PC window on your local Windows computer. Such containers will be mounted as network drives, for example:
You will configure mounting using an account which has administrative privileges obtained using UAC (User Account Control). After this process, the container should be also be available on accounts which do not have such administrative privileges.
Prerequisites
No. 1 Account
You need a CREODIAS hosting account with access to the Horizon interface https://horizon.cloudferro.com.
No. 2. Object storage container
You need at least one object storage container on the CREODIAS cloud. If you do not have one yet, please follow this article: How to use Object Storage on CREODIAS
No. 3. Generated EC2 Credentials
You need to generate EC2 credentials for your account.
The following article contains information how to do it on Linux: How to generate and manage EC2 credentials on CREODIAS.
If instead you want to do it on Windows, you will need to install the OpenStack CLI client first. Check one of these articles to learn more.
Once you have installed the OpenStack CLI client on Windows, use it to perform the workflow from the article How to generate and manage EC2 credentials on CREODIAS - adjust the commands found there accordingly.
No. 4 A local computer running Microsoft Windows
You need a local computer running Microsoft Windows. This article was written for Windows 10 Pro.
You need to have access to an account which can be elevated to administrative privileges through User Account Control (UAC). An account like this should be created by default during standard installation of Microsoft Windows.
Other accounts, including accounts named Administrator found on Windows Server are outside of scope of this article.
What We Are Going To Cover
Mounting an object storage container as a Windows drive using open source software
Testing the connection to the container
Tweaking the –dir-cache-time option
Setting automatic mounting for a container
Removing software used for mounting
Software tools used in this article: Rclone, WinFSP, and NSSM
Rclone has multiple functions such as managing files in cloud storage and syncing between file systems. In this article, you will use its rclone mount command to mount object storage on your Windows VM.
WinFSP enables accessing custom file systems on Microsoft Windows. In this workflow, it will allow Rclone to mount the S3 storage.
NSSM is a service manager. Here, it will be used for configuring automatic mounting of object storage. You will run it from the command line.
How to use the Rclone configuration file
By default, Rclone creates and uses configuration file in the folder \.config\rclone in home directory of the current user.
In order to streamline the configuration process, you will instead manually create a configuration file called rclone.conf in folder where Rclone binary is stored. There, you will paste the code provided in this article. Each time Rclone is executed, the location of that file will be passed to it in a parameter.
Warning
All users of your computer will be able to access and modify the configuration file. It contains the login credentials written as plain text.
Step 1 Download and install the appropriate software
Note
Skip this step if you have these software tools already configured.
Download and extract Rclone
Start PowerShell normally, not as an Administrator.
Use the following commands to
create the directory in which Rclone will be stored and
navigate to it.
mkdir C:\rclone; cd C:\rclone
Without closing PowerShell, open a web browser on your virtual machine. Navigate to the following website: https://rclone.org/downloads. It should contain the table with links to its different versions:
Download the version of Rclone for the Intel/AMD - 64 Bit platform (for Windows operating system).
Download the zip archive. In Microsoft Edge browser it should look like this:
Enter that zip archive, it should contain one directory. Double-click to enter it. Its content should look like this:
Copy the content of that directory to C:\rclone folder which you have previously created using PowerShell.
Download and install WinFsp
Return to Internet browser and navigate to https://winfsp.dev/rel/. Click Download WinFsp Installer:
Run the downloaded installer. The installation process will be similar to installation of other Windows programs. Its step called Custom Setup, however, will require you to make a choice:
Leave the default values intact to install only the Core section and click Next. Complete the installation.
Download and extract NSSM
Navigate to https://nssm.cc/download in your browser.
Click the link in the Latest relase section:
A zip file should be downloaded. It should contain one folder. The content of that folder should look like this:
Navigate to win64 folder. It should have one executable file called nssm:
Copy that file to C:\rclone
Step 2: Enter the connection data
Open folder C:\rclone using Windows file manager. File rclone.conf in that folder will be used to store the connection data for object storage. If that file does not exist, create it. Open it using Notepad.
Each section containing the object storage connection data starts with a line containing its name written in square brackets. Below it, there are lines containing different parameters used to connect to that object storage. Sections will have access to all object storage containers with the same pair of access and secret keys. This means that you do not need to create separate sections for different containers using the same pairs of credentials.
Add the following section to the end of this file:
[remote-config]
type = s3
provider = Other
access_key_id = 1234
secret_access_key = 4321
endpoint = s3.waw3-1.cloudferro.com
[remote-config]
type = s3
provider = Other
access_key_id = 1234
secret_access_key = 4321
endpoint = s3.waw3-2.cloudferro.com
[remote-config]
type = s3
provider = Other
access_key_id = 1234
secret_access_key = 4321
endpoint = s3.fra1-2.cloudferro.com
[remote-config]
type = s3
provider = Other
access_key_id = 1234
secret_access_key = 4321
endpoint = s3.waw2-1.cloudferro.com
In the above block, replace 1234 and 4321 with the access and secret key you obtained while following Prerequisite No. 3, respectively.
This code will create a connection called remote-config. If you want to use a different name for your connection than remote-config, replace it in the code above. This name does not have to be the same as the name of one of your containers.
If you want to access other object storage containers using different credentials on your computer, create a separate section with a different name as described above for each key pair.
Step 3: Perform a test mount
In this step you will perform a test mount to make sure that the configuration was successful and rclone is able to mount your object storage container.
Return to PowerShell you ran in Step 1. If you closed it, open it again as a normal user and not as an Administrator, then navigate to the C:rclone directory. Execute the command below. Replace remote-config with the name of the connection you just configured.
.\rclone.exe --config "C:\rclone\rclone.conf" lsd remote-config:
You should see the list of object storage containers associated with your credentials, for example:
-1 2023-01-18 12:53:14 -1 second-test-container
-1 2023-01-16 13:23:03 -1 test-container
Repeat this process for each key pair you added in Step 2.
To test the mounting of one of your containers, execute the command below without leaving the PowerShell. Replace remote-config with the name of your connection, test-container with the name of your container and E: with the drive letter under which you wish to mount it.
.\rclone.exe mount --config "C:\rclone\rclone.conf" --network-mode remote-config:test-container E: --vfs-cache-mode full --dir-cache-time 1m0s
Note
If you enter a name of a non-existing object storage container by mistake, the command will instead attempt to both create and mount it, subject to other potential constraints (missing permissions etc.)
The option in this command –vfs-cache-mode full should make the mount support standard file system operations.
The option –dir-cache-time 1m0s will be explained in the next step.
The option –network-mode is optional. It was added so that the mounted container will be visually represented as a network drive in This PC window. Removing it will cause the container to be represented as a normal drive and might solve some problems which you might have with mounting.
You should now get the following output:
The service rclone has been started.
Go to This PC window. You should see the mounted container there:
Enter it and you should see its content there:
To stop the test mount, press CTRL+C in the PowerShell. You should get the following output:
The service rclone has been stopped.
The container should no longer be visible in This PC window.
If pressing CTRL+C does not stop the test mount, make sure that the PowerShell window is focused by left-clicking it. Press a letter on your keyboard, for example A, and try pressing CTRL+C again.
You can perform such tests for all object storage containers you wish to use on your computer.
Do not close PowerShell yet.
Step 4: Tweak the –dir-cache-time option
In Step 3, you performed a test mount of your object storage container using the following command:
.\rclone.exe mount --network-mode remote-config:test-container E: --vfs-cache-mode full --dir-cache-time 1m0s
Tweaking the option –dir-cache-time is important especially if you intend to use your container on multiple physical and/or virtual machines. This includes using the container on your virtual machine and the Horizon dashboard. You might discover that the changes made to the bucket on another computer do not appear on your Windows VM. Using the Refresh option of the Windows File Explorer might not synchronize that change either.
That is because the Refresh option in this case does not pull the changes directly from the container, but from the cache. If the option –dir-cache-time is not specified during mounting, the cache is automatically synchronized every 5 minutes. Therefore, if you for example change a name of the folder on your other device, you will be able to pull that change after up to about 5 minutes.
Specifying this option overwrites this default value of 5 minutes. In this example, the automatic refresh of cache was set to 1 minute (1m0s). It is also possible to set this value to for example 1 second (0m1s). You can replace 1m0s in the command above with the value of your choice.
You can now perform a few test mounts as explained in Step 3 and find the –dir-cache-time value that suits you.
Step 5: Configure automatic mounting of your container
Open PowerShell as Administrator and execute the following command there to navigate to the C:\rclone directory:
cd C:\rclone
Now, start the service installer of NSSM with the following command:
.\nssm.exe install
You should get the following window:
Click the … button next to the Path: text field.
Choose the location of Rclone. If you followed this tutorial, this location is as follows:
C:\rclone\rclone.exe
In the Arguments text field enter the following code. Replace remote-config with the name of your connection, test-container with the name of your container, E: with the drive letter under which you wish to mount it and 1m0s with the value you chose in Step 3.
mount --config "C:\rclone\rclone.conf" --network-mode remote-config:test-container E: --vfs-cache-mode full --dir-cache-time 1m0s
Warning
Make sure that you do not have anything mounted under the drive letter you choose here.
In the text field Service name: enter the name for your mounting service. It can be different than the name of your connection you set in Step 1 and the name of your S3 container. In this example, the name mounting-service will be used.
Navigate to the Log on tab. Make sure that the option Local System account is selected.
Click Install service. You should get the following message:
Restart your computer and check whether the drive gets automatically mounted in the This PC window. If it is, the service works as intended.
You should now be able to work with your files.
Add a separate service with a different name as described above for each object storage container you wish to have automatically mounted on your computer. Memorize or write somewhere down the names of those services to be able to easily stop and remove them in the future if needed.
If you find yourself unable to delete files or folders on the object storage, you can remove them from the Object Store -> Containers option in the Horizon dashboard:
Removing software responsible for automatic mounting of object storage
If you no longer want to access the object storage containers from CREODIAS on a particular machine, you can remove the appropriate configuration and software.
The software stack described here might also be used for mounting the EODATA repository using the Remote Transfer for EODATA service. If you follow this section correctly, it will also get removed.
The whole procedure will be covered in this section:
Removal of automatic mounting created in NSSM
Removal of saved object storage in Rclone
Uninstalling WinFsp
Removal of C:\rclone folder containing Rclone and NSSM
Open PowerShell as administrator. Navigate to the C:rclone directory by executing the following command:
cd C:\rclone
To check the status of your automatic mounting service, execute the following command. In it, replace mounting-service with the name of your automatic mounting service you set previously.
.\nssm.exe status mounting-service
You should get the following output:
SERVICE_RUNNING
To stop the automatic mounting of your container, execute the command below. Replace mounting-service as previously.
.\nssm.exe stop mounting-service
Delete the service by executing the command below. Replace mounting-service as previously.
.\nssm.exe remove mounting-service confirm
You should now get the output similar to this:
Service "mounting-service" removed successfully.
If you have used that installation of rclone for mounting, for instance, other object storage containers from CREODIAS or the EODATA repository using the Remote Transfer for EODATA service, remove their services using NSSM in the same way as well.
Delete the C:\rclone folder you created.
Click the Start menu and type control panel.
You should now see the following search result:
Click on it to enter the Control Panel.
Make sure that the drop-down menu View by is set to Category.
In the section Programs select Uninstall a program.
On the list which appeared find the WinFsp entry:
Right-click it and choose Uninstall:
You will now get the following question:
Click Yes. You should get the following window:
Close all your open program and File Explorer windows (for example a This PC or Documents window). Make sure that the option Automatically close applications and attempt to restart them after setup is complete. is selected and click OK.
You will now be prompted to reboot your computer:
Make sure that it is safe to reboot, including that you
saved your work and
closed the programs.
After that, click on Yes in the prompt shown above.
Your computer should now get rebooted.
Once you have logged in again, open This PC window.
Enter your C: drive.
Remove the rclone folder in it - you might need administrative privileges for it.
Rclone and other software used for mounting object storage should now be removed.
What To Do Next
Object storage containers on the CREODIAS cloud can be mounted both on physical and virtual machines running Windows and Linux. To mount the object storage container on a different platform, please follow one of the articles below:
How To Mount Object Storage Container as File System on Windows VM on CREODIAS
How to Mount Object Storage Container as a File System in Linux Using s3fs on CREODIAS
How to access private object storage using S3cmd or boto3 on CREODIAS
You can also mount EODATA repository onto your local computer using the Remote Transfer for EODATA service.
Click here to learn more about that service, including its pricing: How to access EODATA from your own infrastructure on CREODIAS.