Some images are 404, some are gone totally

Options
kalesagobeyond
kalesagobeyond Member Posts: 5
edited November 2023 in Developer APIs

We have been doing some testing on our production instance where we notice that there are so many 404s that has been working before, then suddenly is not. Also, there are some folders where the images inside was completely deleted.

  1. We are using the upload API endpoint for uploading (https://api.cloudinary.com/v1_1/<cloud_name>/image/upload)
  2. The "secure_url" from the response is being saved in the database for reference
  3. All other information is saved on the database. 1 sample of the missing image below

{

                "asset_id": "4ccdb5424ee05cdf0b21dc72b5f5b68b",

                "public_id": "prod/feature/the-mark-twain-house--museum/hstc1kfxbnwbhrqzqa9x",

                "version": 1699156080,

                "version_id": "0dc9bdbddc2480b288c4d98276a04c1f",

                "signature": "b6622df197635b6786d36897f262cf8c4fec367c",

                "width": 853,

                "height": 731,

                "format": "png",

                "resource_type": "image",

                "created_at": "2023-11-05T03:48:00Z",

                "tags": [],

                "bytes": 1824174,

                "type": "upload",

                "etag": "5fcd80fb0b3c78f28d179417c651b1a1",

                "placeholder": false,

                "url": "http://res.cloudinary.com/kalesa-dev/image/upload/v1699156080/prod/feature/the-mark-twain-house--museum/hstc1kfxbnwbhrqzqa9x.png",

                "secure_url": "https://res.cloudinary.com/kalesa-dev/image/upload/v1699156080/prod/feature/the-mark-twain-house--museum/hstc1kfxbnwbhrqzqa9x.png",

                "folder": "prod/feature/the-mark-twain-house--museum",

                "access_mode": "public",

                "original_filename": "blob"

            }

This one is an example where the image is COMPLETELY missing.

There are others where the images are there but somehow, the "secure_url" being used by our webapps is suddenly not working, even the images are on the folder.

Saved in DB: https://res.cloudinary.com/kalesa-dev/image/upload/v1699156080/prod/feature/the-mark-twain-house--museum/hstc1kfxbnwbhrqzqa9x.png

We are a week away from production and this is showing up on our testing. Please let me know what are we doing wrong.

Tagged:

Answers

  • Cloudinary Team
    Cloudinary Team Administrator, Cloudinary Staff Posts: 123 admin
    Options

    Hi there,

    Thanks for reaching out.

    I checked the logs for the asset with the public id prod/feature/the-mark-twain-house--museum/hstc1kfxbnwbhrqzqa9x. I see that it was uploaded on November 5th at 03:48:01.
    A destroy call was made for this asset at 04:42:21, also on November 5th.

    So it was deleted right after it was uploaded. I suspect the same thing has happened for the other assets that you say are missing. The user agent and IP for the upload and deletion are the same. I would advise having a look at your code to see where that destroy call is being made.

    I hope this helps. Let me know if you have any questions.

    Kind regards,

    Tia

    Helpful Links For You
    💬 Share questions, connect with other users in our Cloudinary Community forums and Discord server!
    🧑‍🎓 Join our Cloudinary Academy for free courses, workshops and other educational resources.
    📄 Read our documentation for in-depth details on Cloudinary product features and capabilities
    📰 Check out the Cloudinary blog for the latest company news and insights

  • kalesagobeyond
    kalesagobeyond Member Posts: 5
    Options

    It was working for a few days and then it was destroyed? I haven't issued any DESTROY commands for this image.

    How about the intermittent 404s? I checked our production instance right now and voila, some of the images are back... but some other images are returning 404s. Example below, this image was returning 404 earlier, now it is showing up properly (Main cover image above).


  • Cloudinary Team
    Cloudinary Team Administrator, Cloudinary Staff Posts: 123 admin
    Options

    Hi there,

    The image with the public id of prod/feature/the-mark-twain-house--museum/hstc1kfxbnwbhrqzqa9x was destroyed on the same day that it was uploaded. If it worked for you for a few days, it could have been due to browser caching. All I can tell you is that someone sent a destroy call. I am not able to see who. But as I mentioned earlier, the IP and the user agent are the same for the upload call and the destroy call.

    I'm afraid I will not be able to check anything from an image. If you can share some public ids with me, I would be happy to check when they were uploaded/destroyed.

    Kind regards,

    Tia

    Helpful Links For You
    💬 Share questions, connect with other users in our Cloudinary Community forums and Discord server!
    🧑‍🎓 Join our Cloudinary Academy for free courses, workshops and other educational resources.
    📄 Read our documentation for in-depth details on Cloudinary product features and capabilities
    📰 Check out the Cloudinary blog for the latest company news and insights

  • kalesagobeyond
    kalesagobeyond Member Posts: 5
    Options

    This particular image below was not working properly earlier and it's just showing 404. I tried to refresh my browser but nothing is working. Suddenly, it's ok now. This is happening for several images from time to time.

    {

                    "asset_id": "9c63f4cc4837f985f9bfd1496efd7435",

                    "public_id": "prod/feature/suvarnabhumi-airport/np10jpwwga9j0lomrmo5",

                    "version": 1699535109,

                    "version_id": "36cf7c31c5be810d831749dcdfc920d7",

                    "signature": "610c85bdaa82a490241af7fe17f4a81ffac92896",

                    "width": 599,

                    "height": 426,

                    "format": "png",

                    "resource_type": "image",

                    "created_at": "2023-11-09T13:05:09Z",

                    "tags": [],

                    "bytes": 380599,

                    "type": "upload",

                    "etag": "2b8b8e8af619d2009cb329de7d041bea",

                    "placeholder": false,

                    "url": "http://res.cloudinary.com/kalesa-dev/image/upload/v1699535109/prod/feature/suvarnabhumi-airport/np10jpwwga9j0lomrmo5.png",

                    "secure_url": "https://res.cloudinary.com/kalesa-dev/image/upload/v1699535109/prod/feature/suvarnabhumi-airport/np10jpwwga9j0lomrmo5.png",

                    "folder": "prod/feature/suvarnabhumi-airport",

                    "access_mode": "public",

                    "original_filename": "blob"

                }

  • Cloudinary Team
    Cloudinary Team Administrator, Cloudinary Staff Posts: 123 admin
    Options

    Hi there,

    I checked our logs for the asset with the public id prod/feature/suvarnabhumi-airport/np10jpwwga9j0lomrmo5. It was uploaded on November 9th. I see a request for this asset that returned a 404. The reason for the 404 error response is that there was a double quote at the end of the request path prod/feature/suvarnabhumi-airport/np10jpwwga9j0lomrmo5.png"
    If you fix that small typo, the request will return the image as you are expecting.
    https://res.cloudinary.com/kalesa-dev/image/upload/v1699535109/prod/feature/suvarnabhumi-airport/np10jpwwga9j0lomrmo5.png

    Kind regards,

    Tia

    Helpful Links For You
    💬 Share questions, connect with other users in our Cloudinary Community forums and Discord server!
    🧑‍🎓 Join our Cloudinary Academy for free courses, workshops and other educational resources.
    📄 Read our documentation for in-depth details on Cloudinary product features and capabilities
    📰 Check out the Cloudinary blog for the latest company news and insights

  • kalesagobeyond
    kalesagobeyond Member Posts: 5
    edited November 2023
    Options

    As you can see from the JSON data saved on our database, the secure_url string is saved correctly, as-is it was received from the Cloudinary API response. Since it's a string, it's expected to have double quotation marks at the start and end.

        "secure_url": "https://res.cloudinary.com/kalesa-dev/image/upload/v1699535109/prod/feature/suvarnabhumi-airport/np10jpwwga9j0lomrmo5.png",

    As you can see, this data is valid as a JSON key/value pair as string, no extra quotation marks. Question is where is that extra quotation marks coming from? A

    P.S. Our application is still displaying 404 images randomly, sometimes it works, sometimes it's 404. We are nearing our production deployment and we to have this checked and fixed soon. Thank you ;)

  • Vdeub
    Vdeub Member, Cloudinary Staff Posts: 53
    Options

    Hi @kalesagobeyond,

    I am not sure the quote is the right issue here. If you look at the links on this thread you can see that the quote is added to the link so it might just be a formatting issue here and when we clicked on the link, we got the issue.

    Could you share live examples on your website?

    Thanks,

    Loic

  • kalesagobeyond
    kalesagobeyond Member Posts: 5
    Options

    Here's another example that was working totally fine before but now was displaying 404.

    Error from the BROWSER:

    Data from the DATABASE:

    "imageCover": {

                "url": "https://res.cloudinary.com/kalesa-dev/image/upload/v1699063498/prod/feature/wat-phantao/sznbhcah9zj1amqpwqlj.png",

                "owner": "Marco Angelo",

                "organization": "Kalesa image",

                "alt": "Kalesa image uploaded by Marco Angelo",

                "meta": {

                    "asset_id": "5cc1bfab95e56f10512363eba1e32231",

                    "public_id": "prod/feature/wat-phantao/sznbhcah9zj1amqpwqlj",

                    "version": 1699063498,

                    "version_id": "1def254d260c20e537ff3743fa6cd8bc",

                    "signature": "9b2966664146a61c168cd2afcc1f56ebcf15dfbd",

                    "width": 709,

                    "height": 851,

                    "format": "png",

                    "resource_type": "image",

                    "created_at": "2023-11-04T02:04:58Z",

                    "tags": [],

                    "bytes": 1433854,

                    "type": "upload",

                    "etag": "e292f94523cb20e74e12c81b3dac2c5e",

                    "placeholder": false,

                    "url": "http://res.cloudinary.com/kalesa-dev/image/upload/v1699063498/prod/feature/wat-phantao/sznbhcah9zj1amqpwqlj.png",

                    "secure_url": "https://res.cloudinary.com/kalesa-dev/image/upload/v1699063498/prod/feature/wat-phantao/sznbhcah9zj1amqpwqlj.png",

                    "folder": "prod/feature/wat-phantao",

                    "access_mode": "public",

                    "original_filename": "blob"

                }

            }

    Note:

    1. The 'meta' property is the exact of the JSON response we got from Cloudinary API 'upload'
    2. The 'url' property is a copy of the 'meta.secure_url' property where we use as the reference for loading the <img> tag for the webapp
  • Cloudinary Team
    Cloudinary Team Administrator, Cloudinary Staff Posts: 123 admin
    Options

    Hi @kalesagobeyond,

    I checked the logs and it seems like this image was deleted on November 9th 2023, at 13:41:4 UTC time.

    Hence this is returning a 404.

    Regards,
    Akshay

    Helpful Links For You
    💬 Share questions, connect with other users in our Cloudinary Community forums and Discord server!
    🧑‍🎓 Join our Cloudinary Academy for free courses, workshops and other educational resources.
    📄 Read our documentation for in-depth details on Cloudinary product features and capabilities
    📰 Check out the Cloudinary blog for the latest company news and insights