Oh my god. It's full of code!

HTML5

Automatic Resize of Embedded/Inline Visualforce pages

I’m pretty sure at one time or another, most of us developers have made a sweet inline visualforce page that lives on the object detail page. Problem is since it’s all dynamic and cool we don’t know how ‘tall’ the content might end up being. We are basically stuck with having to guess at a decent height for the VF page and enable scrollbars, which kind of blows. Well not anymore (some exceptions apply)! Expanding on my previous post about using HTML5 postmessaging to move data between embedded visualforce pages and the standard salesforce domain I’ve come up with a first release of a ‘framework’ of sorts (term used very liberally). First an example. Lets check out what I’m talking about using my wordcloud app to demonstrate.

BEFORE

Scrollbars are Sad

 

AFTER
Woot woot no scrolling :D

So how did I accomplish this? The trick, as you may be able to guess by now is the HTML5 PostMessaging feature that I wrote about. By enabling communication between the frame (the visualforce page) and the parent (the detail page) it’s fairly simple to have the frame report it’s size to the parent, and have the parent adjust the height of the frame accordingly. Of course that isn’t everything my framework can do, oh no. It allows for passing of arbitrary commands/data between the two, so you can extend the functionality to pass any kind of commands/dom manipulations you might want between them. Of course the caveat is that this only works on newer browsers since postMessaging is fairly new. Also you’ll need to enable the sidebar on all pages, since that is where the proxy component lives. If you want to play with this install my alpha package from

https://login.salesforce.com/packaging/installPackage.apexp?p0=04ti0000000TY0J

Then include the home page component in the narrow/sidebar and enable the sidebar for all pages (setup->user interface->Show Custom Sidebar Components on All Pages). After that you just include these two components on any page you want to resize itself when embedded. The first component is the core piece that talks to the home page component. The second one is just some code that utilizes that framework to cause the iframe to resize. I’d put them at the bottom of your visualforce page.

<c:vfProxy />

<c:vfProxy_autoHeight />

I’m planning on possibly adding more ‘plugins’ for my framework to do other common handy things so if you have any ideas for potential plugins let me know.


Building a Better WordCloud

Hey all,

I know it’s been a while since my last post. Fun projects with cool end results have been rare and I’m not the type to post stuff just to fill space so I’ve kinda just been chillin recently. Though that changed when I was asked to take another whack at putting together a word cloud app, this time for Salesforce. You may or may not remember that I created a small word cloud app a while ago that didn’t have much anything to do with Salesforce and used a PHP back end that took free text replies from lime survey and created a real time word cloud. This time the task is a little different. I was asked to create a word cloud application that would take data from multiple records, and multiple fields, aggregate it all together and create a cloud from the resulting text blob. Real time updating was not requested, so I had some more flexibility in architecture. I also wanted to take this change to upgrade the actual display a bit since I wasn’t very pleased with my last attempt (it was functional, but a little clunky. Manual CSS rules specified formatting, etc).

Since I knew there must be a good word cloud generator out there I did a bit of searching and decided on the jQuery plugin ‘awesomeCloud’ which makes very stylish clouds without being overly complex. It’s licensing is pretty open (GPL v3) so it looked like a good fit. You can check out some of the sample word clouds in generates on the awesomeCloud demo page. It’s got some nice options for themeing, cloud shape, normalization and more.

So invocation is the word cloud is pretty easy, you just need to create a div with spans in it that have a data-weight attribute that specifies the frequency of that word. You then just call the plugin with the options you want on the container div. Like so

Example of creating a wordcloud (taken from the github page of awesomeCloud)

Example of creating a wordcloud (taken from the github page of awesomeCloud)

Easy enough? But now comes the tricky part, how do we get the data from the objects and fields count the word frequencies and then insert them into the DOM? That is where we come in. I decided that since this is going to be a pretty light weight application with most likely fairly small queries we could get away with using the ajax toolkit thus avoiding the extra complexities of Apex controllers. As usual I’m defaulting to using javascript where I can. You could of course modify the approach below to use Apex if it makes you happier. Whatever works for you. Anyway, lets get on with it.

I decided that since this code may be used in different places to do different things (maybe some inline pages, a stand alone VF page, a dashboard component maybe?), it made sense to make it into a component. Makes it easier to pass in configuration parameters as well, which I figured people would want to do. I’d recommend doing the same. First up lets go ahead of define the parameters we are going to allow to be passed into the component. Most of these are for configuring the word cloud itself, but there are a few others.

    <apex:attribute name="objectType" description="type of records to get data from" type="String" required="true"/>
    <apex:attribute name="records" description="records to get data from" type="String" required="false"/>
    <apex:attribute name="fields" description="fields on records to aggregate data from" type="String" required="true" default="Name"/> 

    <apex:attribute name="lowerbound" description="words below this frequency will not be dipslayed" type="String" required="false" default="0"/> 
    <apex:attribute name="skipwords" description="words not to display regardless of frequency" type="String" required="false" default="and,the,to,a,of,for,as,i,with,it,is,on,that,this,can,in,be,has,if"/> 

    <apex:attribute name="grid" description="word spacing; smaller is more tightly packed but takes longer" type="integer" required="false" default="8"/>    
    <apex:attribute name="factor" description="font resizing factor; default 0 means automatically fill the container" type="integer" required="false" default="0"/>
    <apex:attribute name="normalize" description="reduces outlier weights for a more attractive output" type="boolean" required="false" default="false"/> 

    <apex:attribute name="font" description=" font family, identical to CSS font-family attribute" type="string" required="false" default="Futura, Helvetica, sans-serif"/> 
    <apex:attribute name="shape" description="one of 'circle', 'square', 'diamond', 'triangle', 'triangle-forward', 'x', 'pentagon' or 'star'" type="string" required="false" default="circle"/> 

    <apex:attribute name="backgroundColor" description="background color" type="string" required="false" default="transparent"/>
    <apex:attribute name="colorTheme" description="dark or light" type="string" required="false" default="light"/>

    <apex:attribute name="width" description="how wide should the cloud be (css values such as a pixel or inch amount)?" type="string" required="false" default="600px"/>  
    <apex:attribute name="height" description="how tal; should the cloud be (css values such as a pixel or inch amount)?" type="string" required="false" default="400px"/>  
    <apex:attribute name="autoRefresh" description="should the wordcloud automatically refresh every so often?" type="boolean" required="false" default="false"/>

    <apex:attribute name="refreshInterval" description="how often shold the cloud refresh? In seconds" type="integer" required="false" default="5"/>

Of course you’ll need to include the required javascript libraries.

<script src="//code.jquery.com/jquery-latest.js"></script>
<script src="{!$Resource.jQueryWordCloud}"/>
<apex:includeScript value="/soap/ajax/15.0/connection.js"/>
<apex:includeScript value="/soap/ajax/15.0/apex.js"/>

Now we’ll need to start putting together the javascript functions to get the data and do the word frequnecy analysis. First up, lets create a javascript method that can build a dynamic query to get all the required data. Using the ajax toolkit it looks something like this

function getData(objectType,recordIds, fields)
{
    //build the basic SOQL query string
    var queryString = "Select "+fields+" from "+objectType;

    //if we are limiting the query by searching for specific object ids then add that condition
    if(recordIds.length > 0)
    {
        var whereStatment = "('" + recordIds.split("','") + "')";
        queryString += ' where id in ' + whereStatment;
    }
    //make sure to put a limit on there to stop big query errors.
    queryString += ' limit 2000';

    //run the query
    result = sforce.connection.query(queryString);

    //get the results
    records = result.getArray("records");

    //lets aggregate all the fetched data into one big string
    var wordArray = [];

    //we want to build an array that contains all the words in any of the requested fields. So we will
    //put all the desired fields into an array, iterate over all the records, then over all the fields
    //and stash all the data in another array.

    //fields is a comma separated string. Lets split it into an array so we can iterate over them easily 
    var fieldsArray = fields.split(',');

    //loop over all the records
    for (var i=0; i< records.length; i++)
    {
        var record = records[i];
        //loop over all the fields that we care about
        for(var j=0; j<fieldsArray.length; j++)
        {
            //get the value of this field from this record and add it to the array
            wordArray.push(record[fieldsArray[j]]);
        }
    }  
    //we will now pass in all the words from all the records fields into the getWordFrequnecy function. By calling join
    //on the array with a space character we get one big ass text chunk with all the words from all the records.

    var frequencyResult = getWordFrequency(wordArray.join(' '));

    //pass our frequnecy result data into the load cloud function
    loadCloud(frequencyResult); 
}

With that we can pass in an sObject type, an optional list of record ids (comma separated) and a list of fields to get data from on those records (also comma separated). Now we need to build the getWordFrequency function. That’s going to take a chunk of text and return an array of objects that contain a tag property, and a freq property. That data then gets fed into awesomeCloud to generate the cloud.

            //takes a string and counts the freqnecy of each word in it. Returns array of objects with tag and freq properties.
            //words should be space separated
            function getWordFrequency(wordString){

                //convert string to lower case, trims spaces, cleans up some special chars and splits it into an array using spaces and delimiter.
                var sWords = wordString.toLowerCase().trim().replace(/[,;.]/g,'').split(/[\s\/]+/g).sort();
                var iWordsCount = sWords.length; // count w/ duplicates

                // array of words to ignore
                var ignore = '{!skipwords}'.split(',');
                ignore = (function(){
                    var o = {}; // object prop checking > in array checking
                    var iCount = ignore.length;
                    for (var i=0;i<iCount;i++){
                        o[ignore[i]] = true;
                    }
                    return o;
                }());

                var counts = {}; // object for math
                for (var i=0; i<iWordsCount; i++) {
                    var sWord = sWords[i];
                    if (!ignore[sWord]) {
                        counts[sWord] = counts[sWord] || 0;
                        counts[sWord]++;
                    }
                }

                //get the lower bound as an integer. Lower bound controls the minimum frequnecy a word/tag can have for it to appear in the cloud
                var lowerBound = parseInt({!lowerbound},10);
                var arr = []; // an array of objects to return
                for (sWord in counts) {
                    if(counts[sWord] > lowerBound)
                    {
                        arr.push({
                            tag: sWord,
                            freq: counts[sWord]
                        });
                    }
                }

                /* Sorting code, not really required for this purpose but kept in case it is decided that we want it for some reason.
                // sort array by descending frequency | http://stackoverflow.com/a/8837505
                return arr.sort(function(a,b){
                    return (a.freq > b.freq) ? -1 : ((a.freq < b.freq) ? 1 : 0);
                });
                */

                return arr;

            }

Alright, so now we have all the data from all the records analyzed and the frequency of each word is known. Words we have decided to skip have been left out and those that don’t make the cut from the lower bound are also excluded hopefully leaving us only with actually interesting words. Now we have to pass in the that data to awesomeCloud along with our settings.

function loadCloud(data) 
{ 
    //wordcloud settings
    var settings = {
            size : {
                grid : {!grid},
                factor : {!factor},
                normalize: {!normalize}
            },
            color : {
                background: "{!backgroundColor}"
            },
            options : {
                color : "random-{!colorTheme}",
                rotationRatio : 0.5
            },
            font : "{!font}",
            shape : "{!shape}"
    } 

    //create array for tag spans
    var wordStringArray = [];

    //evaluate each array element, create a span with the data from the object
    $.each(data, function(i, val)
    {
        wordStringArray.push('<span data-weight="'+val.freq+'">'+val.tag+'</span>');
    });

    //join all the array elements into a string. I've heard to push stuff into array and join then push to the DOM is faster than
    //than modifying a string (since strings are unmutable) or modifying the DOM a ton, which makes sense.
    $( "#wordcloud" ).html(wordStringArray.join(''));

    //setup our word cloud
    $( "#wordcloud" ).awesomeCloud( settings );
}

Alright, so now we just need to invoke all this these crazy functions. We’ll do that with a functional call in the document onready to make sure everything is loaded before we start going all crazy modifying the DOM and such.

//login to salesforce API so we can run query
sforce.connection.sessionId = '{!$Api.Session_ID}';
$(document).ready(function() {
    //make an immediate call to getData with the info we need from the config params.
    getData('{!objectType}', '{!records}', '{!fields}');

    //if we are doing an auto refresh, rig that up now using the setInterval method
    if({!autoRefresh})
    {
        setInterval ( "getData('{!objectType}', '{!records}', '{!fields}')", parseInt({!refreshInterval},10) * 1000 );
    }
});

Finally we just need to create our HTML container for the wordcloud and setup the CSS style.

    <style>

    .wordcloud {
        /*border: 1px solid #036;*/
        height: {!height};
        margin: 0.5in auto;
        padding: 0;
        page-break-after: always;
        page-break-inside: avoid;
        width: {!width};

    }

    </style>
    <div id="container" title="wordcloud with the content of fields {!fields} for objects {!records}"> 

        <div id="wordcloud" class="wordcloud" ></div>
    </div>

Whew, alright so that’s it for our component. Now we just need a visualforce page to invoke it. That part is easy. Although our word cloud is capable of aggreating data from multiple objects, for this demo we’ll keep it simple. We will create a little inline visualforce page that can live on the account object. It will create a wordcloud from the description, name, type and ownership fields. To do that, create a visualforce page like this

<apex:page standardController="account">
    <!--- 
    WordCloud comes as a component that can be invoked from any visualforce page. You must pass it the object type to build the cloud for. The rest is optional.
    objectType: the type of sObject to get data from to power the word cloud
    fields: the records on the objects who's content will be used to create the cloud. Must be comma separated
    records: a list of ids which to query for. If none is provided all records of the objectType are queried
    skipwords: words that will not be included in the word cloud no matter how many times they appear. 
    lowerbound: the minimum number of times a word must appear in the text before it is displayed in the cloud.
    grid: word spacing; smaller is more tightly packed but takes longer
    factor: font resizing factor; default "0" means automatically fill the container
    normalize: reduces outlier weights for a more attractive output
    shape: shape of the cloud. Must be one of "circle", "square", "diamond", "triangle", "triangle-forward", "x", "pentagon" or "star"
    font: font family, identical to CSS font-family attribute
    width: width of the cloud. Can be a percent or pixel/inch amount.
    height: height of the cloud. Must be a pixel or inch amount.
    backgroundColor: a hexidecimal (#000000) or string color value. Use 'transparent' for no background
    colorTheme: theme for word colors. Use 'dark' or 'light'
    autoRefresh: automatically refresh the cloud after a specified interval?
    refreshInterval: interval (in seconds) after which the cloud is automatically refreshed
    --->
    <c:wordCloud objectType="account" 
                 records="{!account.id}" 
                 fields="Name,Type,Ownership,Description"
                 lowerbound="1"
                 skipwords="and,an,any,so,or,are,the,to,a,of,for,as,i,with,it,is,on,that,this,can,in,be,has,if"
                 grid="8"
                 factor="0"
                 normalize="false"
                 font="Futura, Helvetica, sans-serif"
                 shape="triangle"
                 width="100%"
                 height="400px" 
                 backgroundColor="black"
                 colorTheme="light" 
                 autoRefresh="false" 
                 refreshInterval="15" />
</apex:page>

Save that page and add it to the page layout of the account. Put a nice big blob of text in the description of an account and see what happens. For my example I figured it would be fitting to copy and paste text from the frequency analysis page of wikipedia. The result looks like this.

Result of WordCloud

Pretty cool eh? Anyway this was just something I threw together in an hour or two, hopefully it’s useful to someone out there. Let me know what ya think!


Displaying and Caching Salesforce Attachment Images in Sites

This time around we are going to be talking about images. How to store them, how to query for them, display them and cache them, in Salesforce, using javascript remoting. We’ll be building a simple application using jQuery, Salesforce and Apex to query for attachments, display them and cache them reduce load times and overhead.

Abstract:
First off, I’m having a bit of a hard time organizing all my thoughts on this topic. It’s kind of big, so please forgive if I skip around a bit. Feel free to ask for clarifications in the comments. So let’s say you are building an application to be hosted on Salesforce. Your application is going to need to publicly accessible (so you are going to be using sites) and the application is going to need to show images that may change frequently and hence would be configured by some non developer types. Your application is going to show all the products you have available, along with pictures of said products.

There is of course many ways you can go about storing your images and relating them to your products but the most straight forward option is to use the notes and attachments feature. That would allow users to easily manage the pictures related to each opportunity without having to go to some central picture repository, or building any additional relationships between objects or URLS. The problem of course is that attachments don’t have a publicly accessible URL to them. You can view them from within Salesforce but you don’t have any way to display them on a site. This could be an issue. Not so fast!

Images as Data
You know those images you uploaded to Salesforce via the attachments feature exist somewhere on Salesforce servers. We also know that Salesforce hates file storage and loves databases. It should come as little surprise that the attachments are actually stored in a table as blob data. That data can be queried for just like any other data. Another little know thing is that in HTML while the img tag normally has it’s src attribute set to a URL, it can in-fact accept base64 encoded image data by specified the data type (). Perhaps we can put all this information together into something useful. Yes, yes we can.

Getting The Image Data
So go ahead and get a visualforce page and controller set up. I’m calling mine productList and productListController respectively. Let’s get the code for our controller in place. Copy and paste this.

global class productListController
{

    //get all the products in the org along with their attachments.
    @remoteAction
    global static remoteObject getProducts()
    {
        remoteObject returnObj = new remoteObject();

        try
        {

            list<Product2> products = [select 
                                                Name,
                                                ProductCode,
                                               Description,
                                               Family,
                                               isActive,
                                               (SELECT Attachment.Name, Attachment.Id FROM Product2.Attachments)
                                               from product2
                                               where isActive = true];
            returnObj.sObjects = products;
        }
        catch(Exception e)
        {
            returnObj.success = false;
            returnObj.message = 'Error getting products';
            returnObj.data = 'Error Type: ' + e.getTypeName() + ' ' + e.getCause() + ' ' + ' on line: ' +e.getLineNumber(); 
        }

        return returnObj;       
    }

    //gets a single attachment (photo) by id. The data is returned as a base64 string that can be plugged into an html img tag to display the image.
    @RemoteAction
    global static remoteObject getAttachment(id attachmentId)
    {   
        remoteObject returnObj = new remoteObject();
        try
        {
            list<Attachment> docs = [select id, body from Attachment where id = :attachmentId limit 1]; 
            if(!docs.isEmpty())
            {
                returnObj.data = EncodingUtil.base64Encode(docs[0].body); 
            }    
        }
        catch(exception e)
        {
            returnObj.success = false;
            returnObj.message = e.getMessage();
            returnObj.data = 'Error Type: ' + e.getTypeName() + ' ' + e.getCause() + ' ' + ' on line: ' +e.getLineNumber();        
        } 
        return returnObj;    
    }   

    global class remoteObject
    {
        public boolean success = true;
        public string message = 'operation successful';
        public string data = null;
        public list<sObject> sObjects = new list<sObject>();
    }    
}

As you can see it’s a pretty simple little controller. We have one method that gets a listing of all the products and the Id’s of the associated attachments using a subquery. That prevents us from having to run another query to get the attachment Id’s. The second function takes a specific attachment id and will return an object with the base64 encoded version of the image. That’s what I was talking about earlier. You can query for an attachment and get it’s raw binary/blob data. Then you can base64 encode it for transfer from the controller back to the requesting page. With that you can get the image data out Salesforce and to your public application.

This does introduce another problem though. Caching. Normally images would be cached by the browser when they are loaded. It uses the filename to create a cached version of the image so next time your browser needs to load it it can just pull it off the hard drive instead of across the internet. The problem with base64 images is they can’t really be cached easily. By the time you have enough data to find it in the cache, you already loaded the whole thing, totally defeating the entire point of the cache. How can we fix this? Caching is too important to just skip in most applications, but yet we need to use base64 encoded images in our app.

Local Storage
With HTML5 we now have something called local storage. Basically it lets us store just about anything we want on the users computer for use at a later time. Basically cookies on steroids. Also where as cookies had to be small little text files, local storage gives us much more flexibility with size. We can leverage this to build own our cache.

Here is the game plan. We’ll run our query to find all the products. We’ll loop over each product we find and create a create an img tag that contains the ID of the image/attachment that needs to go there. After that, we’ll loop over each image tag and populate it with the image. We’ll check to see if we have a local storage item with the ID of the image/attachment. If so, we’ll load that data from the local cache. If not, we’ll make a remoting call to our Apex getAttachment method, and cache the results with local storage then load the data into the img tag. Here is what that looks like.

<apex:page controller="productListController">
    <head>
    <title>Product List</title>
    <script src="//ajax.googleapis.com/ajax/libs/jquery/1.7.2/jquery.js"></script>

    <script>
        $(document).ready(function() {
            getProducts(function(){
                $('.cacheable').each(function(index){
                    var img = $(this);
                     getImage($(this).attr('id'),function(imageId,imageData){
                         $(img).attr('src', 'data:image/png;base64,'+imageData);
                     });
                });               
            });            
        });  

        function getProducts(callback)
        {
                    Visualforce.remoting.Manager.invokeAction(
                        '{!$RemoteAction.productListController.getProducts}',
                        function(result, event)
                        {
                            if (event.status && (result.success == true || result.success == 'true')) 
                            {    
                               var html='';
                               for(var i = 0; i<result.sObjects.length;i++)
                               {
                                   var imageId = 'default image id here';
                                   if(result.sObjects[i].hasOwnProperty('Attachments'))
                                   {
                                        imageId = result.sObjects[i].Attachments[0].Id;
                                   }
                                   html += '<li><img class="cacheable"  id="'+imageId+'">'+result.sObjects[i].Name+'</li>';

                               }
                               $('#products').html(html);
                               callback();
                            } 
                            else
                            {
                                $("#responseErrors").html(event.message);
                            }
                        }, 
                        {escape: true});                   
        } 

        function getImage(imageId,callback)
        {
             var imageData;

              if ( localStorage.getItem(imageId))
              {   
                console.log('Getting image from local storage!');
                imageData = localStorage.getItem(imageId);
                callback(imageId,imageData);    
              }
              else 
              {
                   console.log('Getting image remote server!');
                    Visualforce.remoting.Manager.invokeAction(
                        '{!$RemoteAction.productListController.getAttachment}',
                        imageId,
                        function(result, event)
                        {
                            if (event.status && (result.success == true || result.success == 'true')) 
                            {    
                                 imageData = result.data;
                                 localStorage.setItem(imageId,imageData);      
                                 callback(imageId,imageData);    
                            } 
                            else
                            {
                                $("#responseErrors").html(event.message);
                            }
                        }, 
                        {escape: true});                   
              }      
        } 
    </script>
    </head>

    <body>
            <ul  id="products"></ul>
    </body>            
</apex:page>

So if you are familiar with jQuery and callbacks it’s pretty easy to make sense of what’s going on here. Once the DOM loads we are going to call the getProducts function. getProducts is going to use remoting to run the getProducts apex method. It will iterate over the results and create a list item for each product as well as that empty tag with the id attribute we talked about earlier. It also assigns the img tag the cacheable class so we can easily iterate over them once we are done. Once the looping and list building is complete, we call the callback functions. Since remoting requests are asyncronous we need to use callbacks when we only want to call one function when the other has completed first. Callbacks are a bit beyond the scope of this article, but just know that if we didn’t use them the get $(‘.cachable’).each() loop would run before the list had finished being populated.

So anyway getProducts finishes running and creating the list. Then comes the loop that uses jQuery to find any element that has the ‘cacheable’ class. For each element it finds, it calls the getImage() function on it, passing in the Id of that element. GetImage is where the cacheing magic happens. It will check to see if a local storage item exists with the id it gets passed. If so, it calls back with that content, if not, it queries Salesforce for an attachment with that id, creates a local storage element for it, and then again returns that content. The loop takes the returned content and sets the src tag of the img element with the base64 encoded data and boom! We have an image.

There you have it. Using Salesforce attachments to house images, using Apex and jQuery to query for them and display them, and HTML5 local storage to cache them. Pretty cool eh? I could write more, but I’m tired and I don’t feel like it. Hit me with questions if ya got em.