Ideas for local access of Cloudinary content to application server?
Our company is in the process of transitioning from a legacy DAM system to Cloudinary. Although the majority of our content can be migrated seamlessly, we are encountering some challenges with a specific type of file that requires local accessibility, i.e., it needs to be mounted onto specific application servers.
These files, considered as raw
formats by Cloudinary, are essentially templates contained within a zip package with a custom extension. The application utilizing these files only recognizes paths presented as local paths, for instance:
file:////mnt/files/directory/subdirectory/templatename.custom
We are trying to figure out the best method to synchronize these specific files within our Cloudinary environment. The goal is to have them automatically copied to a location that can be mounted to our application server upon uploading. These assets in Cloudinary are expected to have a range of structured metadata fields, enabling us to employ queries based on these fields to identify the appropriate file to be used as a template. Consequently, a file path would be derived from the corresponding Cloudinary URL.
However, it's important to note that not all content in our Cloudinary environment requires this process. We have numerous image formats that do not necessitate this treatment. We are only interested in synchronizing the file formats that serve as templates.
Currently, we're contemplating the following solutions:
- Whenever a file of this type is uploaded to Cloudinary, it could be simultaneously copied to an additional S3 bucket. This bucket could then be mounted to the application server, thereby emulating a local mount.
- Another possibility is to trigger an automated process that downloads the file to a local storage mount on the server whenever a file of this type is uploaded. This would ensure that the files are always synchronized.
In either case, the paths should ideally retain a similar structure with related folders and subfolders. Example:
- Cloudinary:
https://res.cloudinary.com/{subaccount}/image/upload/templates/subfolder/template1.custom
- Mount:
file:////mnt/cld/{subaccount}/image/upload/templates/subfolder/template1.custom
If anyone has encountered a similar situation to this, or has any suggestions, your input would be appreciated. Thanks.
Comments
-
Hi,
Thank you for reaching out! If I understood you correctly, you would want a reference where the template file is located in the mounted drive. The easiest way to achieve this is through Structured Metadata. Here is my suggested workflow and granted that
file_path
structured metadata is created:1.App is uploading a template file to Cloudinary, along with the upload request is to pass the
notification_url
parameter which will trigger a notification to your webhook URL endpoint.2.Your webhook URL will catch this notification with a payload similar below:
{ "notification_type": "upload", "timestamp": "2023-05-17T20:46:50+00:00", "request_id": "4552ef6551eee51dd097991e37496507", "asset_id": "28aa71376825ddba90a8847f552c6a34", "public_id": "test2.txt", "version": 1684356409, "version_id": "ea4af35aad33bd6461e6976a684e74db", "resource_type": "raw", "created_at": "2023-05-17T20:46:49Z", "tags": [], "bytes": 113, "type": "upload", "etag": "40fae2ead5e8a1ad97932042246105a4", "placeholder": false, "url": "http://tdatu-res.cloudinary.com/raw/upload/v1684356409/test2.txt", "secure_url": "https://tdatu-res.cloudinary.com/raw/upload/v1684356409/test2.txt",
3.You may download this resource using either the
url
orsecure_url
.4.Once the download is successful, use our Upload API's Explicit to update the
file_path
metadata. Thefile_path
is a string that contains the file's path to your mounted drive:res = Cloudinary::Uploader.explicit("test2.txt", {:resource_type => "raw", :type = "upload", :metadata => {:file_path => "/mnt/test2.template"}})
5.When retrieving the test2.txt asset, you may then use Search API and pass the optional
with_field("metadata")
to retrieve the structured metadata value:res = Cloudinary::Search\ .expression("resource_type: raw AND public_id=test2.txt")\ .with_field("metadata")\ .execute()
The response object will look like the following:
{"total_count"=>1, "time"=>12, "resources"=> [{"asset_id"=>"a36b9f90c8039da57213e56eed02d07c", "public_id"=>"test.txt", "folder"=>"", ... "status"=>"active", "access_mode"=>"public", "access_control"=>nil, "etag"=>"40fae2ead5e8a1ad97932042246105a4", "metadata"=>{"file_path"=>"/mnt/test.template"},
I hope the above helps in finding a solution to your application. For further reading on Metadata API, please visit:
https://cloudinary.com/documentation/metadata_api#update_a_metadata_field_by_external_idLet us know if you need further assistance.
Regards,
AnthonyDeveloper Support Engineer
CloudinaryJoin the Cloudinary Community
1 -
Thanks, Anthony. That sounds like a good approach. We'll look into implementing and testing your idea.
0 -
Hi,
Thank you for the update. We will wait for your feedback.
Best regards,
AnthonyDeveloper Support Engineer
CloudinaryJoin the Cloudinary Community
0