Struggling with Collage API, need help debugging.

raymondcamden · May 2023

Hi, I've been trying to make use of the Collage API, and I've run into a few issues.

First, I had code that worked once. I changed the size, and it didn't work. I deleted my original result, reverted the code, and ran it again, and it fails.

So I tried adding a notification_url to my call, and as far as I can tell, it's never run.

I end up not having any way to know what's going wrong. Here's the code I'm using. It basically reads an RSS feed, gets image URLs, and uses that for a collage.

const cloudinary = require('cloudinary').v2;
const Parser = require('rss-parser');
let parser = new Parser();

/* Cloudinary keys... */
const API_KEY = 'secret';
const API_SECRET = 'even more secret';
const CLOUD_NAME = 'raymondcamden';

async function getPosters(username) {
    
    let feed = await parser.parseURL(`https://letterboxd.com/${username}/rss/`);

    return feed.items.map(f => {
        return f.content.replace(/.*<img src="(.*?)"\/>.*/,'$1');
    });

}

async function makeCollage(username,images) {


    let assets = images.map(i => {
        return {'media':i }
    });


    let manifest = {
        "template":"grid",
        "width":800,
        "height":500,
        "columns":3,
        "rows":2,
        "assetDefaults": {
            "kind":"fetch",
            "crop":"fill",
            "gravity":"auto"
        },
        "spacing":10,
        "color":"black",
        "assets": assets
    };


    let params = {
        'manifest_json':JSON.stringify(manifest), 
        'public_id': `${username}_letterboxd`,
        'timestamp': Math.round((new Date).getTime()/1000),
        'notification_url':'https://webhook.site/aafcc8f8-6937-47f0-93a4-172ce1085072'
    }


    let signature = cloudinary.utils.api_sign_request(params, API_SECRET);


    let f = new FormData();
    f.append('api_key', API_KEY);
    f.append('timestamp', params.timestamp);
    f.append('signature', signature);
    f.append('public_id', params.public_id);
    f.append('resource_type', 'image');
    f.append('manifest_json', params.manifest_json);
    f.append('notification_url', params.notification_url);
    
    let resp = await fetch(`https://api.cloudinary.com/v1_1/${CLOUD_NAME}/image/create_collage`, {
        method:'post',
        body:f
    });
    return await resp.json();


}


(async () => {
    let films = (await getPosters('raymondcamden')).slice(0,6);
    let resp = await makeCollage('raymondcamden', films);
    console.log(`Resp from Cloudinary: ${JSON.stringify(resp)}`);

})();

When called, all I get is:

 {"status":"processing","public_id":"raymondcamden_letterboxd","batch_id":"3ff4068d36703fc969bc681e0ee27bb52b069b9613c5277b44cddfb7ead3d1cb84726e128dd4146c34530c90a0c8c8df"}

Any idea how to proceed?

DannyFromCloudinary · June 2023

https://community.cloudinary.com/discussion/comment/338#Comment_338

When we send the webhook, we use a POST because it includes a json payload. You should also see Cloudinary as the user-agent.

https://community.cloudinary.com/discussion/comment/339#Comment_339

At this moment in time, the webhook will only fire for successful requests, not failed ones. I can appreciate this isn't ideal and will feed this back to the relevant team. The same applies for overwrites - currently not implemented. The collage API is still fairly new, so I imagine these are both on the roadmap already, but giving the team a friendly nudge can't hurt :)

https://community.cloudinary.com/discussion/comment/340#Comment_340

If you have backups enabled on your account, you may not be able to use the same public ID as it's reserved by the deleted file in case it's restored. I would suggest making sure backups are disabled for these assets by including backup: false or alternatively, you could rename the asset before deleting it. Appending a timestamp to the public_id could work nicely.

Hope this helps!

-Danny

DannyFromCloudinary · June 2023

Hi Raymond.

I've managed to get this working on my own cloud just fine. I did make a few small tweaks to the code, but that was mainly just simplifying things a little, as you'll see if you run this in a diff checker against your code.

const cloudinary = require('cloudinary').v2;
const Parser = require('rss-parser');
let parser = new Parser();

const NOTIF_URL = "https://webhook.site/c48c4b36-32de-4f23-a943-796601156bfa"

async function getPosters(username) {
    let feed = await parser.parseURL(`https://letterboxd.com/${username}/rss/`);

    return feed.items.map(f => {
        return f.content.replace(/.*<img src="(.*?)"\/>.*/,'$1');
    });
}

async function makeCollage(username,images) {
    let assets = images.map(i => {
        return {'media':i }
    });

    let manifest = {
        "template":"grid",
        "width":800,
        "height":500,
        "columns":3,
        "rows":2,
        "assetDefaults": {
            "kind":"fetch",
            "crop":"fill",
            "gravity":"auto"
        },
        "spacing":10,
        "color":"black",
        "assets": assets
    };

    const timestamp = Math.round((new Date).getTime()/1000);

    let params = {
        'manifest_json':JSON.stringify(manifest), 
        'public_id': `230196/${username}_letterboxd_${timestamp}`,
        'timestamp': timestamp,
        'notification_url': NOTIF_URL
    }

    let signature = cloudinary.utils.api_sign_request(params, cloudinary.config().api_secret);

    let f = new FormData();
    f.append('api_key', cloudinary.config().api_key);
    f.append('timestamp', params.timestamp);
    f.append('signature', signature);
    f.append('public_id', params.public_id);
    f.append('resource_type', 'image');
    f.append('manifest_json', params.manifest_json);
    f.append('notification_url', params.notification_url);
    
    let resp = await fetch(`https://api.cloudinary.com/v1_1/${cloudinary.config().cloud_name}/image/create_collage`, {
        method:'post',
        body:f
    });
    return await resp.json();
}


(async () => {
    let films = (await getPosters('raymondcamden')).slice(0,6);
    let resp = await makeCollage('raymondcamden', films);
    console.log(`Resp from Cloudinary: ${JSON.stringify(resp)}`);
})();

The response when I submitted that was

Resp from Cloudinary: {"status":"processing","public_id":"230196/raymondcamden_letterboxd_1685619488","batch_id":"b461d97d6e754fba8da337b88cd3f3c11f0eec5f24612f3b4b7091a0192c78391397ac913b2c8bb1685309cda14a9352"}

If we look at https://webhook.site/#!/c48c4b36-32de-4f23-a943-796601156bfa/eb86f311-2dde-4292-915f-e9db31eebc4a/1, we see the request was made successfully and the resulting image URL was https://assets.danny-valentine.com/image/upload/v1685619489/230196/raymondcamden_letterboxd_1685619488.png

As far as I can tell, everything is working correctly, though I do see from https://webhook.site/#!/aafcc8f8-6937-47f0-93a4-172ce1085072/09692c14-7f82-4bf6-87e0-f83017236bf2/1 that all requests to your webhook have been GETs rather than POSTs.

What I would suggest is trying the code that I have provided, and if you encounter further issues, to raise a support query with your cloud name and approximate time you ran the code snippet in UTC and we can check our logs to see what's going on.

I hope this helps. Let us know how it goes :)

Thanks,

-Danny

raymondcamden · June 2023

I'm going to test this now, but query - you mention the webhook calls are GETs. Isn't that Cloudinary sending the webhook?

raymondcamden · June 2023

Ok, I'm seeing a POST now to my webhook, and your code works, and I think I see the issue. If I use the same public_id more than once, it doesn't over overwrite it. However - shouldn't the notification webhook give me an error of some sort? I also, randomly, tried adding overwrite:true to my call, and it didn't work. I notice the online demo destroys the existing collage first. So... I guess my questions are:

1) Why is the webhook never firing to give me an error?

2) Is there no way to pass a flag to the creation call to overwrite an existing collage?

raymondcamden · June 2023

And now I'm back to it not working again. I added a delete, the delete returns ok, but if I try to make a collage with public_id as just 'raymondcamden_letterboxd', it consistently fails, with no call to the webhook.

raymondcamden · June 2023

Thanks. I turned off auto backups, but also had to switch to a new public id (I just added _collage). Now I can resize, test, and confirm it's deleting/updating. Thank you again! I see you are on the team, so is this post the 'nudge' yall need. ;) Honestly, 99% of my time on this particular demo would have been gone if the error reporting was improved. The actual use of the API isn't too hard.

DannyFromCloudinary · June 2023

Glad to hear you've got things up and running!

As I say, I'll feed back your comments to the relevant team so things are improved in the future.

Let us know if there's anything else you need :)

Struggling with Collage API, need help debugging.

Best Answer

Answers

Categories