Meteor Async Guide

meteor-async.md

From Meteor's documentation:

In Meteor, your server code runs in a single thread per request, not in the asynchronous callback style typical of Node. We find the linear execution model a better fit for the typical server code in a Meteor application.

This guide serves as a mini-tour of tool, trix and patterns that can be used to run async code in Meteor.

Basic async

Often we want to use a 3rd party library without a synchronous interface or we want to use the asynchronous interface because we're going to run something that could take a very long time to complete and we want to use the unblocking technique (described below) to process additional request for the client while we wait. The basic pattern for this can be found all over Meteor's codebase and looks like this:

Meteor.methods({
  asyncJob: function(message) {
  
    // Set up a future
    var fut = new Future();

    // This should work for any async method
    setTimeout(function() {

      // Return the results
      fut.ret(message + " (delayed for 3 seconds)");

    }, 3 * 1000);

    // Wait for async to finish before returning
    // the result
    return fut.wait();
  }
});

Code

Parallel async

We also commonly need to run multiple async calls in parallel. This could be applied to any async method but here we're using Meteor's own Meteor.http.get's async method signature because it's commonly desirable to run a batch of http calls in parallel and wait for the result (or not wait for the result using the unblocking technique described below).

Meteor.methods({
  parallelAsyncJob: function(message) {

    // We're going to make http get calls to each url
    var urls = [
      'http://google.com',
      'http://news.ycombinator.com'
    ];

    // Keep track of each job in an array
    var futures = _.map(urls, function(url) {

      // Set up a future for the current job
      var future = new Future();

      // A callback so the job can signal completion
      var onComplete = future.resolver();

      /// Make async http call
      Meteor.http.get(url, function(error, result) {

        // Do whatever you need with the results here!
    
        // Inform the future that we're done with it
        onComplete(error, result);
      });

      // Return the future
      return future;
    });

    Future.wait(futures);
  }
});

Code

If you want to collect results from parallel async jobs you'll have to do a little more work:

Meteor.methods({
  parallelAsyncJob: function(message) {
    var urls = [
      'http://google.com',
      'http://news.ycombinator.com',
      'https://github.com'
    ];

    var futures = _.map(urls, function(url) {
      var future = new Future();
      var onComplete = future.resolver();
  
      /// Make async http call
      Meteor.http.get(url, function(error, result) {

        // Get the title if there was no error
        var title = (!error) && getTitle(result);
    
        onComplete(error, title);
      });
  
      return future;
    });

    // wait for all futures to finish
    Future.wait(futures);

    // and grab the results out.
    return _.invoke(futures, 'get'); 
  }
});

Code

Unblocking Meteor.methods

TODO

TODO

Accessing Meteor's full environment from async calls

TODO

Packaging async node modules

TODO

Abstractions

At this point I'm leaving this purposely unfinished because until I have opportunities to use these techniques more than once, in real apps, I don't want to guess what the right abstractions are. Hopefully people will write awesome smart packages for this stuff and we can find out together what will be most effective and then campaign for our favorites to be be included in core.

TODO

Cleanup, writing is terrible

Add all useful info mentioned by Matt here

Contributions

Thanks to Tom Coleman @tmeasday for his help with this!

Resources

http://stackoverflow.com/a/11510874

This is great. Any chance of this being documented?

The Future class seem to have been removed in Meteor 0.5.1 - any chance to get this back with a simple require?

require('fibers/future') is enough. I added a little howto here: https://gist.github.com/4130605

I think it would be nice to include the client side call as well for full working example - I had the client side callback arguments wrong for a while and it took me a while to spot it. Something like this should work:

Template.hello.events({

  'click input' : function () {

    //call the async function from the client
    Meteor.call("asyncJob", "async test", function(err, res){

      alert(res);

    });
  }
});

In Meteor 0.6.0, add:

var Future = Npm.require("fibers/future")

to the top of your file.

I made also a simple library to wrap common async functions into blocking ones.

Helpfull, thank you!

Just a heads up though, as of 5 days ago (Meteor 0.6.5) future.ret() is deprecated. You use it in your "Basic async" example.

I found out about it here: meteor/meteor#1311

Is there any async in meteor's roadmap?

Yes, if you need blocking methods for async calls you can use Async Utilities from npm smart package. If you need non-blocking methods you can use Q.js and promises. See my gist: https://gist.github.com/hipertracker/8064847

This gist is awesome but bro it's deprecated.

Stackoverflow topic for Meteor 0.8+ http://stackoverflow.com/questions/24743402/how-to-get-an-async-data-in-a-function-with-meteor

+1 On the updated documentation.

Here's an example of how I'm using this with GeoNear that isn't supported using just Meteor yet.

Meteor.methods({
  searchQuery: function(searchParams, callback) {
    // Look through skills and work experience.
    // Sort by distance (using $near forces this).
    // Limit of 100 is also enforced by $near.
    // TODO Incorporate recently active
    check(searchParams, Object);

    if (!this.userId) {
      throw new Meteor.Error('User is not logged in');
    }

    if (!searchParams.term) {
      throw new Meteor.Error('Search parameters did not contain `term`.');
    }

    var asyncDbLookup = function(callback) {
      _db.command({
        geoNear: 'users',
        near: [searchParams.lat, searchParams.lng],
        limit: 30,
        query: {
          $or: [
            { interests: searchParams.term },
            { skills: searchParams.term },
            { jobExperience: searchParams.term }
          ]
        }
      }, function(err, res) {
        var results = [];

        // Pluck only the fields we want.
        if (res.results && res.results.length > 0) {
          results = _.map(res.results, function(entry) {
            var mappedEntry = _.pick(entry.obj, '_id', 'pictureUrl', 'headline', 'firstName');
            mappedEntry.distance = entry.dis;

            return mappedEntry
          });
        }

        callback(null, results);
      });
    };

    var syncDbLookup = Meteor.wrapAsync(asyncDbLookup);
    return syncDbLookup();
  }
});

@Overload119 - excellent example, thank you!

@Overload119 Thank you very much! finally understood how Async works on Meteor!

The rest of the Javascript world use Promises which is effectively what you have here.
Thank you for bringing this to Meteor.

Is there a fresh up-to-date fork of this or is this still considered valid?

Hey, great article! Can you put up a few samples of how to use async/await syntax in Meteor server methods definition? :)

+1 for async/await