Asynchronous execution for Google App Scripts (gas)

Async.gs

/*
* Async.gs
*
* Manages asyncronous execution via time-based triggers.
*
* Note that execution normally takes 30-60s due to scheduling of the trigger.
*
* @see https://developers.google.com/apps-script/reference/script/clock-trigger-builder.html
*/

var Async = Async || {};
var GLOBAL = this;

// Triggers asynchronous execution of a function (with arguments as extra parameters)
Async.call = function(handlerName) {
  return Async.apply(handlerName, Array.prototype.slice.call(arguments, 1));
};

// Triggers asynchronous execution of a function (with arguments as an array)
Async.apply = function(handlerName, args) {
  var trigger = ScriptApp
  .newTrigger('Async_handler')
  .timeBased()
  .after(1)
  .create();
  CacheService.getScriptCache().put(String(trigger.getUniqueId()), JSON.stringify({ handlerName: handlerName, args: args }));
  return { 
    triggerUid: trigger.getUniqueId(),
    source: String(trigger.getTriggerSource()), 
    eventType: String(trigger.getEventType()), 
    handlerName: handlerName,
    args: args
  };
};

// GENERIC HANDLING BELOW
// 
function Async_handler(e) {
  var triggerUid = e && e.triggerUid;
  var cache = CacheService.getScriptCache().get(triggerUid);
  if (cache) {
    try {
      var event = JSON.parse(cache);
      var handlerName = event && event.handlerName;
      var args = event && event.args;
      if (handlerName) {
        var context, fn = handlerName.split('.').reduce(function(parent, prop) {
          context = parent;
          return parent && parent[prop];
        }, GLOBAL);
        if (!fn || !fn.apply) throw "Handler `" + handlerName + "` does not exist! Exiting..";
        // Execute with arguments
        fn.apply(context, args || []);
      }
    } catch (e) {
      console.error(e);
    }
  }
  // Delete the trigger, it only needs to be executed once
  ScriptApp.getProjectTriggers().forEach(function(t) {
    if (t.getUniqueId() === triggerUid) {
      ScriptApp.deleteTrigger(t);
    }
  });
};

@sdesalas Hi! I tried to apply your async solution, but on first iteration an error pops up when calling the object's method. What do you think is the problem?

Error: [TypeError: folderSecond.createFile is not a function]

Code:
function uploadFilesFromDrive(url,filename,folderSecond){
const fileID = getIdFrom(url);
const getFile = DriveApp.getFileById(fileID);
const blob = getFile.getBlob().setName(filename).getAs('application/pdf');
const file = folderSecond.createFile(blob);
};

@ShadrinSpock see comment above

Only primitives will work as callback arguments, however its relatively straightforward to pass the ID and then recreate the Spreadsheet object inside the handler method.

https://gist.github.com/sdesalas/2972f8647897d5481fd8e01f03122805?permalink_comment_id=4428924#gistcomment-4428924

In your case you probably want the ID of the folder. Instead of the folder object.

@sdesalas Thank you so much for writing this! I've only just begun my journey as a developer and learned how to write asynchronous functions in Javascript today, only to find that it doesn't work in Google Apps Script. Your library worked like a charm for my purposes though. I managed to get the expected result on the first try, which is honestly incredibly surprising knowing me.

It seems so weird that you can write async function blah(), or return promises and write .then(), but Apps Script still runs it synchronously. Do you know why that is? What is the point of having the syntax if it doesn't work the way it was intended?

@mojoro https://stackoverflow.com/questions/31241396/is-google-apps-script-synchronous

Hi @sdesalas ,

We are trying to handle (to import data via Plaid API), large amount of data on the spreadsheet (google sheet).
The external API needs to be used in the while loop because response needs to be check the condition is true or false.
If the conditions meets true then the loop will continue until condition false.
And also the external API has limit restrictions(50 calls per minute).
We need to store all the response in a single variable like array collection then we need to format and manipulate them in the spreadsheets.

We have shared the code here for you.

//once trigger first time.
Async.call('updateSpreadSheet');

var responseData = [];

function updateSpreadSheet(){
  var collection = [];
  let response = fetchAPITransactions();
  if( response == true ){
   collection = fetchMoreAPITransactions();
  }
  if(collection.length > 0 ){
   manipulateDatatToSpreadsheet(collection);
  }
}

function manipulateDatatToSpreadsheet(){
 //format data and add/remove data on the spreadsheets.
}

function fetchMoreAPITransactions( response ){
 while( response == true ){
    responseData.push( response);
  break;
  }
  return responseData;
}
 
function fetchAPITransactions(){
 //call api end point and response. 50 calls per minute. 
 var response = responsedataofAPI;
 return response;
}

Is this approach correct for calling the Async function to execute the loop, the triggers are not called sequentially, this makes the data in the Spreadsheet also not in correct format. or not in sequence as per date order.
This process also runs for a very long time 45 to 60 minutes which is not practically feasible while retrieving the data.
What is the best approach in this case? We expect the data to be ordered by date in the spreadsheet and to fetch the data much quicker.

Hi @stallioninet,

I have not tested your approach, but I believe you don't need this library to achieve your desired result.

Google Apps Script has its own built-in API for making requests to external API's, and it will not execute code past the request if you are using their tools. With GAS, you don't need the traditional javascript async syntax to achieve some async results within standard functions. For example, every call you make to SpreadsheetApp or something similar is asynchronous, but the syntax doesn't require it. That is all handled under the hood.

Async.gs saves the function and the parameters you pass in at the time you call it and programmatically creates a time-based trigger that executes it as soon as possible. This is useful in many cases, such as when you need to execute things out of sequence, but you should not need it here. You would likely be better served by paginating through the data served by the API you are trying to contact and either writing it to the sheet as you receive it or storing it all in a 2-D array that is written after the pagination is complete.

Ensure that you use the built-in methods for interacting with external api's, run your code sequentially, and all should be well.