Struggling with Collage API, need help debugging.

Options
raymondcamden
raymondcamden Member Posts: 8
edited June 2023 in Developer APIs

Hi, I've been trying to make use of the Collage API, and I've run into a few issues.

First, I had code that worked once. I changed the size, and it didn't work. I deleted my original result, reverted the code, and ran it again, and it fails.

So I tried adding a notification_url to my call, and as far as I can tell, it's never run.

I end up not having any way to know what's going wrong. Here's the code I'm using. It basically reads an RSS feed, gets image URLs, and uses that for a collage.

const cloudinary = require('cloudinary').v2;
const Parser = require('rss-parser');
let parser = new Parser();

/* Cloudinary keys... */
const API_KEY = 'secret';
const API_SECRET = 'even more secret';
const CLOUD_NAME = 'raymondcamden';

async function getPosters(username) {
    
    let feed = await parser.parseURL(`https://letterboxd.com/${username}/rss/`);

    return feed.items.map(f => {
        return f.content.replace(/.*<img src="(.*?)"\/>.*/,'$1');
    });

}

async function makeCollage(username,images) {


    let assets = images.map(i => {
        return {'media':i }
    });


    let manifest = {
        "template":"grid",
        "width":800,
        "height":500,
        "columns":3,
        "rows":2,
        "assetDefaults": {
            "kind":"fetch",
            "crop":"fill",
            "gravity":"auto"
        },
        "spacing":10,
        "color":"black",
        "assets": assets
    };


    let params = {
        'manifest_json':JSON.stringify(manifest), 
        'public_id': `${username}_letterboxd`,
        'timestamp': Math.round((new Date).getTime()/1000),
        'notification_url':'https://webhook.site/aafcc8f8-6937-47f0-93a4-172ce1085072'
    }


    let signature = cloudinary.utils.api_sign_request(params, API_SECRET);


    let f = new FormData();
    f.append('api_key', API_KEY);
    f.append('timestamp', params.timestamp);
    f.append('signature', signature);
    f.append('public_id', params.public_id);
    f.append('resource_type', 'image');
    f.append('manifest_json', params.manifest_json);
    f.append('notification_url', params.notification_url);
    
    let resp = await fetch(`https://api.cloudinary.com/v1_1/${CLOUD_NAME}/image/create_collage`, {
        method:'post',
        body:f
    });
    return await resp.json();


}


(async () => {
    let films = (await getPosters('raymondcamden')).slice(0,6);
    let resp = await makeCollage('raymondcamden', films);
    console.log(`Resp from Cloudinary: ${JSON.stringify(resp)}`);

})();



When called, all I get is:

 {"status":"processing","public_id":"raymondcamden_letterboxd","batch_id":"3ff4068d36703fc969bc681e0ee27bb52b069b9613c5277b44cddfb7ead3d1cb84726e128dd4146c34530c90a0c8c8df"}

Any idea how to proceed?

Tagged:

Best Answer

  • DannyFromCloudinary
    DannyFromCloudinary Member, Cloudinary Staff Posts: 119
    Answer ✓
    Options

    When we send the webhook, we use a POST because it includes a json payload. You should also see Cloudinary as the user-agent.

    At this moment in time, the webhook will only fire for successful requests, not failed ones. I can appreciate this isn't ideal and will feed this back to the relevant team. The same applies for overwrites - currently not implemented. The collage API is still fairly new, so I imagine these are both on the roadmap already, but giving the team a friendly nudge can't hurt :)

    If you have backups enabled on your account, you may not be able to use the same public ID as it's reserved by the deleted file in case it's restored. I would suggest making sure backups are disabled for these assets by including backup: false or alternatively, you could rename the asset before deleting it. Appending a timestamp to the public_id could work nicely.

    Hope this helps!

    -Danny

Answers

  • DannyFromCloudinary
    DannyFromCloudinary Member, Cloudinary Staff Posts: 119
    Options

    Hi Raymond.

    I've managed to get this working on my own cloud just fine. I did make a few small tweaks to the code, but that was mainly just simplifying things a little, as you'll see if you run this in a diff checker against your code.

    const cloudinary = require('cloudinary').v2;
    const Parser = require('rss-parser');
    let parser = new Parser();
    
    const NOTIF_URL = "https://webhook.site/c48c4b36-32de-4f23-a943-796601156bfa"
    
    async function getPosters(username) {
        let feed = await parser.parseURL(`https://letterboxd.com/${username}/rss/`);
    
        return feed.items.map(f => {
            return f.content.replace(/.*<img src="(.*?)"\/>.*/,'$1');
        });
    }
    
    async function makeCollage(username,images) {
        let assets = images.map(i => {
            return {'media':i }
        });
    
        let manifest = {
            "template":"grid",
            "width":800,
            "height":500,
            "columns":3,
            "rows":2,
            "assetDefaults": {
                "kind":"fetch",
                "crop":"fill",
                "gravity":"auto"
            },
            "spacing":10,
            "color":"black",
            "assets": assets
        };
    
        const timestamp = Math.round((new Date).getTime()/1000);
    
        let params = {
            'manifest_json':JSON.stringify(manifest), 
            'public_id': `230196/${username}_letterboxd_${timestamp}`,
            'timestamp': timestamp,
            'notification_url': NOTIF_URL
        }
    
        let signature = cloudinary.utils.api_sign_request(params, cloudinary.config().api_secret);
    
        let f = new FormData();
        f.append('api_key', cloudinary.config().api_key);
        f.append('timestamp', params.timestamp);
        f.append('signature', signature);
        f.append('public_id', params.public_id);
        f.append('resource_type', 'image');
        f.append('manifest_json', params.manifest_json);
        f.append('notification_url', params.notification_url);
        
        let resp = await fetch(`https://api.cloudinary.com/v1_1/${cloudinary.config().cloud_name}/image/create_collage`, {
            method:'post',
            body:f
        });
        return await resp.json();
    }
    
    
    (async () => {
        let films = (await getPosters('raymondcamden')).slice(0,6);
        let resp = await makeCollage('raymondcamden', films);
        console.log(`Resp from Cloudinary: ${JSON.stringify(resp)}`);
    })();
    

    The response when I submitted that was

    Resp from Cloudinary: {"status":"processing","public_id":"230196/raymondcamden_letterboxd_1685619488","batch_id":"b461d97d6e754fba8da337b88cd3f3c11f0eec5f24612f3b4b7091a0192c78391397ac913b2c8bb1685309cda14a9352"}
    

    If we look at https://webhook.site/#!/c48c4b36-32de-4f23-a943-796601156bfa/eb86f311-2dde-4292-915f-e9db31eebc4a/1, we see the request was made successfully and the resulting image URL was https://assets.danny-valentine.com/image/upload/v1685619489/230196/raymondcamden_letterboxd_1685619488.png

    As far as I can tell, everything is working correctly, though I do see from https://webhook.site/#!/aafcc8f8-6937-47f0-93a4-172ce1085072/09692c14-7f82-4bf6-87e0-f83017236bf2/1 that all requests to your webhook have been GETs rather than POSTs.

    What I would suggest is trying the code that I have provided, and if you encounter further issues, to raise a support query with your cloud name and approximate time you ran the code snippet in UTC and we can check our logs to see what's going on.

    I hope this helps. Let us know how it goes :)

    Thanks,

    -Danny

  • raymondcamden
    raymondcamden Member Posts: 8
    Options

    I'm going to test this now, but query - you mention the webhook calls are GETs. Isn't that Cloudinary sending the webhook?

  • raymondcamden
    raymondcamden Member Posts: 8
    Options

    Ok, I'm seeing a POST now to my webhook, and your code works, and I think I see the issue. If I use the same public_id more than once, it doesn't over overwrite it. However - shouldn't the notification webhook give me an error of some sort? I also, randomly, tried adding overwrite:true to my call, and it didn't work. I notice the online demo destroys the existing collage first. So... I guess my questions are:

    1) Why is the webhook never firing to give me an error?

    2) Is there no way to pass a flag to the creation call to overwrite an existing collage?

  • raymondcamden
    raymondcamden Member Posts: 8
    Options

    And now I'm back to it not working again. I added a delete, the delete returns ok, but if I try to make a collage with public_id as just 'raymondcamden_letterboxd', it consistently fails, with no call to the webhook.

  • raymondcamden
    raymondcamden Member Posts: 8
    Options

    Thanks. I turned off auto backups, but also had to switch to a new public id (I just added _collage). Now I can resize, test, and confirm it's deleting/updating. Thank you again! I see you are on the team, so is this post the 'nudge' yall need. ;) Honestly, 99% of my time on this particular demo would have been gone if the error reporting was improved. The actual use of the API isn't too hard.

  • DannyFromCloudinary
    DannyFromCloudinary Member, Cloudinary Staff Posts: 119
    Options

    Glad to hear you've got things up and running!

    As I say, I'll feed back your comments to the relevant team so things are improved in the future.

    Let us know if there's anything else you need :)