How to correctly delete folders with mixed contents by prefix?

nk9

I have a bunch of test folders with both images and raw files in them which I want to delete. I have this code:

    let folders = await cloudinary.api.sub_folders('albums').then(resp => resp.folders)
    var folders_to_delete = folders.filter((f) => f.name.startsWith('test'))

    console.log("folders to delete: ", folders_to_delete)

    for (const folder of folders_to_delete) {
        console.log("prefix: ", folder)
        cloudinary.api
            .delete_resources_by_prefix(folder.path, { resource_type: "image" })
            .then(images_result => {
                console.log(images_result);

                cloudinary.api.delete_resources_by_prefix(folder.path,
                    { resource_type: "raw" })
            })
            .finally(info_result => {
                console.log(info_result)
                
                // Once all the folder's contents are gone, then delete the folder itself
                cloudinary.api
                    .delete_folder(folder.path)
                    .then(folder_result => console.log(folder_result))
                    .catch(err => console.log("ERROR: ", err))
            })

    }

But I'm not sure if this is the clearest/most correct way to do what I want. Also, I now have 4 folders which appear to be empty in the Media Library but which the API is refusing to delete because "Folder is not empty." Might this have something to do with derived assets? And regardless, how do I convince the API to actually delete these empty folders?

nk9

I've got something a bit clearer now, but it's still not deleting the supposedly empty folders. I've even turned on skip_backup, but that hasn't helped.

    let types = ["image", "raw"]

    for (const type of types) {
        let resp = await cloudinary.api.resources({
            resource_type: "image",
            max_results: 500,
            prefix: "albums/test",
            type: "upload"
        })
        let public_ids = resp.resources.map(r => r.public_id)
        cloudinary.api.delete_resources(public_ids, { resource_type: type })
    }

    let folders = await cloudinary.api.sub_folders('albums').then(resp => resp.folders)
    var folders_to_delete = folders.filter((f) => f.name.startsWith('test'))

    console.log("folders to delete: ", folders_to_delete)

    for (const folder of folders_to_delete) {
        // Once all the folder's contents are gone, then delete the folder itself
        cloudinary.api
            .delete_folder(folder.path, { skip_backup: true })
            .then(folder_result => console.log(folder_result))
            .catch(err => console.log("ERROR: ", err))
    }

I appreciate that this will only delete the first 500 resources found, and would fail if you subsequently tried to delete folders which still have resources inside. Those limitations are fine, but I do want the empty folders to actually go away…

nk9

I have something a bit clearer now, and I've tried using skip_backup to delete folders even if there are backed up resources hiding. Unfortunately, it's still not deleting the empty folders:

    let types = ["image", "raw"]

    for (const type of types) {
        let resp = await cloudinary.api.resources({
            resource_type: "image",
            max_results: 500,
            prefix: "albums/test",
            type: "upload"
        })
        let public_ids = resp.resources.map(r => r.public_id)
        cloudinary.api.delete_resources(public_ids, { resource_type: type })
            .then(res => console.log("delete resources success:", res))
    }

    let folders = await cloudinary.api.sub_folders('albums').then(resp => resp.folders)
    var folders_to_delete = folders.filter((f) => f.name.startsWith('test'))

    console.log("folders to delete: ", folders_to_delete)

    for (const folder of folders_to_delete) {
        // Once all the folder's contents are gone, then delete the folder itself
        cloudinary.api
            .delete_folder(folder.path, { skip_backup: true })
            .then(folder_result => console.log(folder_result))
            .catch(err => console.log("ERROR: ", err))
    }

I understand that this would only remove the first 500 resources returned, and would fail if it then tried to delete a non-empty folder. That limitation is fine, but I do still need the empty folders to actually be deleted…

nk9

Here is part of the response I'm getting back from delete_resources:

delete resources success: {
  deleted: {
    'albums/test1/cabinet_handle': 'deleted',
    'albums/test1/fBurma-002': 'deleted',
    'albums/test1/intro_cgi_with_caption': 'deleted',
    'albums/test4/fBurma-011': 'deleted',
...
  deleted_counts: {
    'albums/test1/cabinet_handle': { original: 1, derived: 0 },
    'albums/test1/fBurma-002': { original: 1, derived: 0 },
    'albums/test1/intro_cgi_with_caption': { original: 1, derived: 0 },
    'albums/test4/fBurma-011': { original: 1, derived: 0 },

It says the files are deleted, but then it won't delete the folders containing them. And when I run the code again, without having uploaded anything else, the same files are always returned from the api.resources() call. And many of these aren't even in the existing folders. For example, "test1" doesn't show up in the Media Library, but "test4" is there and empty.

I'm really confused at the way Cloudinary is treating deleted files.

Cloudinary_John_H

Hi there,

Thanks for reaching out, and for all of the details.

This behavior is likely related to having records of backed-up assets in the folders you are trying to delete. When assets have been backed up to the folder, the folder will not be deletable unless the backups are cleared. We do not expose the deletion of backups to end users, in order to help prevent the accidental or intentional malicious deletion of backups.

Can you share with me the complete list of folder names that are not deleting due to the records of backed-up assets being present?

Is this the entire list of folders?

 ['albums/test1/','albums/test4']

In order to delete these assets, we will need to assist with the clearing of asset backups associated with these folders. Can you share with me your cloud name please, so I can confirm my suspicions about records of backed-up assets being present in these folders?

Looking forward to your response,

John

nk9

Thanks for the reply! Unfortunately, I don't have a complete list of the folders that aren't deleting. I turned off backups at some point, but it was long after I created a bunch of folders in the process of testing. It would be all the ones which start with "test". Anyway, I've opened a support ticket about this just now. Hopefully it will be possible to remove the test*/** backups.

Cloudinary_John_H

Sounds good - we will assist you further in the support ticket, thank you!

nk9

Just to circle back on this, I've decided that it's preferable if I have a way to filter these out, since it's possible to get these sorts of files returned from the resource Admin methods. Filtering out deleted-but-backed-up items is a more futureproof solution vs having to remember never to delete anything, or forcing accounts which use this code to turn off backups.

I've found that the deleted items have a created_by and uploaded_by property of null. Can you confirm that this is a reliable way to determine whether an asset returned from e.g. search() is deleted or not? And, even better, is there a way to add a predicate to a search which would exclude these from the results before they're returned?

Cloudinary_John_H

Hey there,

We provide a field in the Search API expression field called "status", which can be used to filter assets if they are deleted or not.

https://cloudinary.com/documentation/search_api#expression_fields

status=deleted
status=deleted OR active

Possible values: active, deleted
Notes:
- Resource data is stored in the database with status deleted only if the resource has a backup.
- By default, when you conduct a search, the response includes only resources with active status. If you want the response to include resources with a deleted status, you must use this field to specify it.
Exact match only. Case-sensitive.

Hopefully, that will be useful for you. Based on the description, it sounds like we only return assets that are "active" by default. If you are seeing results that are contrary to that, please let us know the search expression you are using.