Node.js, Socket.io, and Force.com canvas and you

So I got back from Dreamforce a week ago, and my head hasn’t stopped spinning. So many cool technologies and possibilities, I’ve been coding all week playing with all this new stuff (canvas, node.js, socket.io, nforce, heroku, streaming api, etc). Cloudspokes conveniently also had a challenge asking us to write a force.com canvas application that used a node.js back end. I wanted to take this chance to see if I could put what I had learned into practice. Turns out, it’s not too hard once you actually get your dev environment set up, and get things actually uploading and all the ‘paper work’ done. I wanted to leverage the Salesforce.com streaming API, Node.Js, and Socket.io to build a real time data streaming app. I also wanted to use Force.com canvas to get rid of having to worry about authentication (honestly the best part about canvas, by a wide margin). You can see the end result here:

http://www.screencast.com/t/Qfen94pl (Note the animations don’t show too well in the video due to framerate issues with the video capture software).

You can also grab my source project from here

Demo Source

Getting Set Up

First off, huge word of warning. All this process is just what I’ve found to work from trial and error and reading a ton of shit. I have no idea if this is the recommended process or even a good way to do things. It did/does however work. This was my first ever node.js application, as well as my first time using canvas and only my second time using Heroku. So ya know, definitely not pro level here but it’s at least functional. Also the actual idea for this application was inspired by Kevin O’Hara (@kevinohara80) and Shoby Abdi (@shobyabdi) from their streaming API session in dreamforce. They are also the authors of the kick ass nforce library, without which this app would not be possible, so thanks guys!

So how can you get started doing the same? Well first of course get yourself setup with Heroku. Heroku is where we are going to store our project, it’s a free hosting platform where you can host node.js, python and java applications. So if you don’t have a Heroku account, go get one, it’s free.

You’ll also want to download the Heroku toolbelt. This is going to get us our tools for dealing with Heroku apps (heroku client), as well as testing our stuff locally (foreman), and working with git. You can grab it here. https://toolbelt.heroku.com/. On that page it walks you through creating your project. For a more in depth guide, check out https://devcenter.heroku.com/articles/nodejs. Get a node.js project created and move onto the next step.

So now I assume you have a basic node.js project on Heroku. Now to actually make it do stuff, we’ll need to install some libraries using NPM. Open a command prompt and navigate to your local project folder. Install express (npm install express), socket.io (npm install socket.io) and nforce (npm install nforce). This should add all the required libraries to your project and modify the package.json file that tells Heroku the shit it needs to include.

You’ll also need a winter 13 enabled salesforce org to start building canvas apps. So go sign up for one here (https://www.salesforce.com/form/signup/prerelease-winter13.jsp) and get signed up. Depending when you are reading this you may not need a prerelease org, winter 13 may just be standard issue. Whatever the case, you need at least winter 13 to create canvas apps. As soon as you get your org, you’ll also probably want to create a namespace. Only orgs with namespaces and publish canvas apps, which you may want to do later. Also having a namespace is just a good idea, so navigate to the setup->develop->packages section, and register one for yourself.

In your org, you’ll need to configure your push topic. This is the query that will provide the live streaming data to your application.Open a console or execute anonymous window, and run this:

PushTopic pushTopic = new PushTopic();
pushTopic.ApiVersion = 23.0;
pushTopic.Name = 'NewContacts';
pushtopic.Query = 'Select firstname, lastname, email, id from contact;
insert pushTopic;
System.debug('Created new PushTopic: '+ pushTopic.Id); 

This will create a live streaming push topic in your org of all the new incoming contacts. You could change the query to whatever you want of course but for the purpose of this example, lets keep it simple.

Next, you’ll want to configure your canvas application. In your org, go to setup->create->apps. There should be a section called connected apps. Create a new one. Give it all the information for your Heroku hosted application. Permissions and callbacks here are a bit un-needed (since canvas will be taking care of the auth for us via a signed request) but should be set properly anyway. The callback url can be just the url of your application on Heroku. Remember only https is accepted here, but that’s okay because Heroku supports https without you having to do anything. Pretty sweet. Set your canvas app url to the url of your Heroku application and set access method to post signed request. That means when your app is called by canvas, it’s going to be as a post request, and in the post body is going to be a encoded signed request that contains an oAuth key we can use to make calls on behalf of the user. Save your canvas application.

The actual code (there isn’t much of it)
So we have everything configured now, but no real code. Our app exists, but it doesn’t do shit. Lets make it do something cool. Open up node server file (it’s probably called like web.js, or maybe app.js if you followed the guide above. It’s going to be whatever file is specified in your Procfile in your project). Paste this code. You’ll need to modify the clientId and clientSecret values from your canvas application. They are the consumer key and consumer secret respectively. I honestly don’t know if you’d need to provide your client secret here since the app is already getting passed a valid oAuth token, but whatever, it can’t hurt.

var express = require('express');
var nforce = require('nforce');
var app = express.createServer() , io = require('socket.io').listen(app);

var port = process.env.PORT || 3000;
//configure static content route blah
app.configure(function(){
  app.use(express.methodOverride());
  app.use(express.bodyParser());
  app.use(express.static(__dirname + '/public'));
  app.use(express.errorHandler({
    dumpExceptions: true, 
    showStack: true
  }));
  app.use(app.router);
});

app.listen(port, function() {
  console.log('Listening on ' + port);
});

io.configure(function () { 
  io.set("transports", ["xhr-polling"]); 
  io.set("polling duration", 10); 
});

var oauth;

var org = nforce.createConnection({
      clientId: 'YOUR CAVANAS APPLICATION CONSUMER KEY',
      clientSecret: 'YOUR CANVAS APPLICATION CLIENT SECRET',
      redirectUri: 'http://localhost:' + port + '/oauth/_callback',
      apiVersion: 'v24.0',  // optional, defaults to v24.0
      environment: 'production'  // optional, sandbox or production, production default
});

//on post to the base url of our application
app.post('/', function(request, response){
    //get at the signed_request field in the post body
    var reqBody = request.body.signed_request;   

    //split the request body at any encountered period (the data has two sections, separated by a .)
    var requestSegments = reqBody.split('.'); 

    //the second part of the request segment is base64 encoded json. So decode it, and parse it to JSON
    //to get a javascript object with all the oAuth and user info we need. It actually contains a lot of 
    //data so feel free to do a console.log here and check out what's in it. Remember console.log statments 
    //in node run server side, so you'll need to check the server logs to see it, most likely using the eclipse plugin.   
    var requestContext = JSON.parse(new Buffer(requestSegments[1], 'base64').toString('ascii'));
    
    //create an object with the passed in oAuth data for nForce to use later to subscribe to the push topic
    oauth = new Object();
    oauth.access_token = requestContext.oauthToken;
    oauth.instance_url = requestContext.instanceUrl;
    
    //send the index file down to the client
    response.sendfile('index.html');

});


//when a new socket.io connection gets established
io.sockets.on('connection', function (socket) {
      
    try
    {
      //create connection to the NewContacts push topic.
      var str = org.stream('NewContacts', oauth);
    
      //on connection, log it.
      str.on('connect', function(){
        console.log('connected to pushtopic');
      });
    
      str.on('error', function(error) {
         socket.emit(error);
      });
    
      //as soon as our query has new data, emit it to any connected client using socket.emit.
      str.on('data', function(data) {
         socket.emit('news', data);
      });
    }
    catch(ex)
    {
        console.log(ex);
    }
    
});

Now you’ll also need the index.html file that the server will send to the client when it connects (as specified by the response.sendfile(‘index.html’); line). Create a file called index.html, and put this in there.

<!DOCTYPE html>
<html>
    <head>
        <!-- - does a commit work remotely as well? -->
        <title>New Contacts</title>
        <meta name="apple-mobile-web-app-capable" content="yes" />
        <meta name="apple-mobile-web-app-status-bar-style" content="black-translucent" />
        
        <link href='http://fonts.googleapis.com/css?family=Lato:400,700,400italic,700italic' rel='stylesheet' type='text/css'>
        
        <link rel="stylesheet" href="/reveal/css/main.css">
        <link rel="stylesheet" href="/reveal/css/theme/default.css" id="theme">    
        
        <script type="text/javascript" src="//ajax.googleapis.com/ajax/libs/jquery/1.4.2/jquery.min.js"></script>
    
        <script src="/socket.io/socket.io.js"></script>
    
        <script>
          $.noConflict();    
          var socket = io.connect();

          socket.on('news', function (data) {
            jQuery('.slides').append('<section><h2><a href="https://na2.salesforce.com/'+data.sobject.Id+'">'+data.sobject.FirstName+' '+data.sobject.LastName+'</a></h2><br>'+data.sobject.Email+'<br/></section>');
            
            Reveal.navigateNext();
          });



        </script>    
    </head>
    <body>

        <div class="state-background"></div>
        
        <div class="reveal">
            <div class="slides"> 
                <section>New Contacts Live Feed</section>
            </div>

            <!-- The navigational controls UI -->
            <aside class="controls">
                <a class="left" href="#">◄</a>
                <a class="right" href="#">►</a>
                <a class="up" href="#">▲</a>
                <a class="down" href="#">▼</a>
            </aside>

            <!-- Presentation progress bar -->
            <div class="progress"><span></span></div>                
        </div>


            
            <script src="/reveal/lib/js/head.min.js"></script>
            <script src="/reveal/js/reveal.min.js"></script>    
            <script>
                
                // Full list of configuration options available here:
                // https://github.com/hakimel/reveal.js#configuration
                Reveal.initialize({
                    controls: true,
                    progress: true,
                    history: true,
                    mouseWheel: true,
                    rollingLinks: true,
                    overview: true,
                    keyboard: true,
                    theme: Reveal.getQueryHash().theme || 'default', // available themes are in /css/theme
                    transition: Reveal.getQueryHash().transition || 'cube', // default/cube/page/concave/linear(2d)
    
                });

        
            
                    
            </script>            
    </body>
</html>

We are also going to need the CSS reveal framework to create the awesome slideshow. Grab it https://github.com/hakimel/reveal.js. In your Heroku project create a folder called public. In there create a folder called reveal. In that folder dump the css, js, lib and plugin folders from reveal. So it should be like root_folder->public->reveal->js->reveal.js for example. There is probably a more ‘git’ way to include the reveal library, but I don’t know what it is. So for now, moving folders around should work.

Now use git to push this all up to Heroku. I’d really highly recommend using the Heroku plugin for eclipse to make life easier. There is an install guide for it here https://devcenter.heroku.com/articles/getting-started-with-heroku-eclipse. However you do it, either from eclipse or command line, you gotta push your project up to Heroku. If you are using command line, I think it’s something like “git add .” then “git commit” then “git push heroku master” or something like that. Just use the damn eclipse plugin honestly (right click on your project and click team->commit, then right click on the root of your project and click team->push upstream).

If your app pushes successfully and doesn’t crash, it should run when called from Canvas now. Canvas will call your Heroku application using a post request. The post request contains the signed request data including an oAuth token. We use that oAuth token and store it in our node.js app for making subsequent api calls. Node.js returns the index.html file to the client. The client uses socket.io to connect back to the server. The server has a handler that says upon a new socket.io connection, create a connection to the push topic newContacts in salesforce, using the oAuth token we got before. When a new event comes from that connection, use Socket.IO to push it down to the client. The client handler says when a new socket.io event happens, create a new slide, and change to that slide. That’s it! It’s a ton of moving parts and about a million integration, but very little actual code.

Enjoy and have fun!

October 1, 2012 | Categories: Canvas, Heroku, Javascript, jQuery, Node.js, Salesforce, Socket.IO | Tags: canvas, Heroku, Node.js, real time, reveal, salesforce, socket.io, streaming | 2 Comments