Oh my god. It's full of code!

Posts tagged “Google

Things I learned about Python and Google AppEngine

So this last cloudspokes challenge was particularily challenging for me, as it required the use of the python programming language, and the google appengine hosting platform. I am not sure if it’s just me, but I found solutions to seemingly simple problems hard to find, so I’ll outline a few things I found here in hopes to save you time.

1) With google appengine, use python 2.5. Nothing else. Don’t even think about it.

2) You have to enable appengine for your google apps account. Do so by by going to the users and groups link, then services. Find app engine and enable it. Otherwise you will get an unauthorized to use this application error.

3) Of course when using google appEngine you are not allowed access to the file system. So any file operations are not going to happen. You have to use the datastore or some other method.

4) When you use the GetDocumentListFeed() method to get data about your documents using the client library, the resource id returned (resourceId.text) is not just the resource ID. it is the type of file, and the resource ID. You have to split them to get each separate. For example.

feed = self.gd_client.GetDocumentListFeed()
for entry in feed.entry:
resource = entry.resourceId.text
doc_type = resource[:resource.find(‘:’)]
resource_Id = resource[resource.index(‘:’)+1:]

5) Not all files in google docs have extensions in the file names/titles, so don’t count on them being there. You have to figure out the file extension for yourself.

6) When calling a google API, error 401 Unauthorized may likely mean that there is something wrong with the header you have sent for the authorization. You don’t just send the auth token, you also send a short pre string. For example

docs_token = gd_client.GetClientLoginToken()
request.add_header(‘Authorization’, ‘GoogleLogin auth=’+docs_token)

7) The content type header of your HTTP request can change how the body content is sent. The correct content type for most requests against the google docs API is
request.add_header(‘Content-Type’, ‘application/atom+xml; charset=\”UTF-8\”‘)
You might be tempted to change it to text/xml since that seems like what you are sending, but don’t. You’ll get an error 415 media type unrecognized or some shit like that.

8) If you are debugging your post request, and you send it to a page that prints the content, your content may come back base64 encoded. This is fine. Don’t worry about it. Google knows what to do with the base64 shit. I spend a while trying to figure why this was happening, and it is because of the content type that you set in the headers. Yes changing the content type to text/xml makes the content come through as regular text but again you get that error 415. So don’t worry, your data is fine.

9) The one is probably obvious to most, but just in the spirit of clearing up confusion, you must include a content-length header. It must be the amount of characters in the body of your request. Nothing fancy, just run a len() on the body of your content and set that as the content-length header.

10) You must also include the header GData-Version, and it should be set to 3.0. Again, I know most of the pros probably know this, but it’s just one of those things that can slip by and screw you up.

11) Trim your XML content, and unicode escape it as well before inserting it into the body of your post request. Otherwise you could get some freak errors. Something like
body = body.encode(‘unicode-escape’).strip()
should do the trick.

12) HTTP status 201 means you finally did it and you’re request went through. However the urllib2 library still sees that as an error. It’s okay, just have a special handler set up for it. EX

            response = urllib2.urlopen(request)
         
        except urllib2.HTTPError, e:
            if e.code == 201:
                message = 'w00t success'
            else:
                message = 'There was an error processing your request. ' +e.read()
        except: # catch *all* exceptions
            e = sys.exc_info()[1]
            message = 'There was an error processing your request.'

13) For any other python n00bz like me, when creating your post request to send to google, urllib2 is the way to go. You can set your headers, and include your body content, everything you need to do. Here is an example of how I put together my post request for my most recent app.

 url = 'http://docs.google.com/feeds/default/private/archive'
            
            
            body = '''<entry xmlns="http://www.w3.org/2005/Atom" xmlns:docs="http://schemas.google.com/docs/2007">
                  <docs:archiveConversion source='application/vnd.google-apps.document' target='application/msword'/>
                  <docs:archiveConversion source='application/vnd.google-apps.spreadsheet' target='text/csv'/>
                  <docs:archiveConversion source='application/pdf' target='application/pdf'/>
                  <docs:archiveConversion source='application/vnd.google-apps.presentation' target='application/pdf'/>
                  <docs:archiveNotify>'''+username+'''</docs:archiveNotify>
                </entry>'''
            body = body.encode('unicode-escape').strip()
            

            request = urllib2.Request(url,body)
            request.add_header('GData-Version', '3.0')
            request.add_header('Authorization', 'GoogleLogin auth='+docs_token)
            request.add_header('Content-length', str(len(body)))
            request.add_header('Content-Type', 'application/atom+xml; charset=\"UTF-8\"')

        
            response = urllib2.urlopen(request)

So there you have it. Some handy tips for working with Python and AppEngine that should make you hate life a little less. Hope this helps someone out there!


Apex Google Powered Navigation Class

Hey all,
Expanding on what I was talking about yesterday I have packaged my google maps navigation tool into an apex class. Using this class you can easily get directions to and from any address using the tried and tested google maps system. Simply provide the origin, destination, desired data return format, and a toggle indicating if you want to expand abbreviations in the directions (like rd to road, nw to northwest, dr to drive, etc).

You have several options for how you want the data to be returned. You can either return the raw xml data from google (kml as they call it) get a nice JSON data structure, a semi-colon separated list, or a semi-colon separated list that is url formated. Invoking the class is super easy.

String directions = getDirections.getDirections('address 1','address 2','dataReturnType', expandAbbreviations);

One thing that I dislike about the current incarnation is that the XML parsing is a bit slow/inefficient. I am not very strong on my XML-fu so currently it must iterate over every XML element to find the desired data. It would be faster to tell it more accuratly where to look, but I don’t really know how. So if you are a smarty pants developer and know how to make the search faster, please feel free to let me know. I’ll be happy to update it and credit you for the fix.

With this tool, you could easily use it as a controller for a visualforce page, and create your own RESTful style webservice, or use it to help build other neat services. It’s a nice developer resource I think.

Download it here or from the projects page.

NOTE: To use this class you will have to add http://maps.google.com to your remote sites in your org. Just go to setup->security->remote sites and add the url.


Coldfusion, Angel.com, Google Maps Directions, and You!

Okay, so even the title is a mouthful, this post is probably going to be insane you are thinking. Well… maybe, but it’s cool stuff. So picture this. You are using Angel.com as an IVR provider. So people call in and talk to a phone machine for data. Now say you want to give directions over the phone. Say you want those directions to be dynamic, and given to the user step by step. So for example you are hosting an event. People pre-register for this event, and have provided their address, which you have stored in a database. Bob Johnson calls in (he has registered and provided his address before) and wants directions from his house to your event center. You might think you’d need a live person to do this. Blasphemy! Have Bob authenticate so we can find his address in the database. Feed that address into an Angel.com variable (if there were a better way to enter addresses over the phone, you wouldn’t even need to pre-register, but because entering data in phones sucks we kind of need their address to already exist somewhere we can get it). Once that variable is in Angel.com, pass it, along with the destination to this tool via URL arguments. This tool will then give step by step directions that the IVR can read aloud back to Bob. He even has the ability to replay each direction, and navigate backward and forward through the steps.

Just copy and paste this and run host it on a ColdFusion server somewhere. It’s ready to be called with all configs just being passed in the URL at runtime.

<cfparam name="url.start" type="string" default="1405+Olive+Ln+N,+Plymouth,+Hennepin,+Minnesota+55447">
<cfparam name="url.end" type="string" default="1111+Cambridge+St.+Hopkins,+MN+55343+(White+Castle)">
<cfparam name="url.stepID" type="integer" default="1">

<!---- The id of the page that calls this webservice in angel.com ---->
<cfparam name="url.thisPage" type="string" default="1">

<!---- the id of the page to go to if this thing errors for some reason --->
<cfparam name="url.failPage" type="string" default="2">

<!---- the id of the page to go when we are all done giving directions ---->
<cfparam name="url.nextPage" type="string" default="3">

<!---- the id of the page to go to if we just can't find directions or a route ---->
<cfparam name="url.reEnterInfoPage" type="string" default="4">

<!---- the id of the page to go to if the person decides they want to talk to a person ---->
<cfparam name="url.transferToCC" type="string" default="5">

<!---- the id of the page to go to if the person wants to hang up---->
<cfparam name="url.disconnectPage" type="string" default="6">

<cfparam name="XMLData" type="string" default="">
<cfparam name="text" type="string" default="">

<!--- Format the directions for sending to google --->
<cfset url.start = replacenocase(url.start, " ", "+")>
<cfset url.end = replacenocase(url.end, " ", "+")>

<cfhttp url="http://maps.google.com/" result="KMLData">
    <cfhttpparam name="saddr" value="#url.start#" type="url">
    <cfhttpparam name="daddr" value="#url.end#" type="url">
    <cfhttpparam name="output" value="kml" type="url">
</cfhttp> 


<cfoutput>
    <cftry>
        <cfset XMLData = xmlParse(KMLData.FileContent)>
        <cfset totalNumberOfSteps = arraylen(XMLData.kml.Document.XmlChildren)-4>
        <cfset text = XMLData.kml.Document.XmlChildren[stepID+3].XmlChildren[1].XmlText>
        <cfset text = text&" . .">
        <!--- Some extra text formatting for reading over the IVR. You can easily add more abbreviations here if there
              are some I forgot --->
        <cfset text = replacenocase(text, " LN ", " Lane ")>
        <cfset text = replacenocase(text, " BLVD ", " Boulevard ")>
        <cfset text = replacenocase(text, " RD ", " Road ")>
        <cfset text = replacenocase(text, " ST ", " Street ")>
        <cfset text = replacenocase(text, " Ave ", " Avenue ")>
        <cfset text = replacenocase(text, " NW ", " North West ")>
        <cfset text = replacenocase(text, " NE ", " North East ")>
        <cfset text = replacenocase(text, " SE ", " South East ")>
        <cfset text = replacenocase(text, " SW ",  "South West")>            
        <cfset text = replacenocase(text, " N ", " North ")>
        <cfset text = replacenocase(text, " E ", " East ")>
        <cfset text = replacenocase(text, " W ", " West ")>
        <cfset text = replacenocase(text, " S ",  "South ")>

        
        <cfsavecontent variable="PromptMessage">
            #text#
            <cfif stepID LT totalNumberOfSteps>
                Press 1 for the next direction. Press 2 to repeat this direction. Press 3 to hear the previous direction.
                <cfelse>
                    You have reached your destination.
                    Press 1 to disconnect. Press 2 to repeat this direction. Press 3 to hear the previous direction.
            </cfif>    
        </cfsavecontent>
    
        <cfif stepID LT totalNumberOfSteps>
            <cfsavecontent variable="XMLLinks">
                <LINK dtmf="1" returnValue="#stepID+1#" destination="#url.thisPage#" />
                <LINK dtmf="2" returnValue="#stepID#" destination="#url.thisPage#" />
                <LINK dtmf="3" returnValue="#stepID-1#" destination="#url.thisPage#" />
            </cfsavecontent>
            
            <cfelse>
                <cfsavecontent variable="XMLLinks">
                    <LINK dtmf="1" returnValue="#url.nextPage#" destination="#url.thisPage#" />
                    <LINK dtmf="2" returnValue="#stepID#" destination="#url.thisPage#" />
                    <LINK dtmf="3" returnValue="#stepID-1#" destination="#url.thisPage#" />
                </cfsavecontent>            
        </cfif>
        
        
        <cfset counter = 1>
        <cfset VariablesObject[counter] = structnew()>
        <cfset VariablesObject[counter]["Name"] = "totalDirections">
        <cfset VariablesObject[counter]["Value"] = totalNumberOfSteps>

         

        <cfcatch type="any">
            <cfset promptMessage = "Sorry we couldn't find a route with the information supplied. Press 1 to try a different direction method. Press 2 to disconnect, or press 3 to be transferred to customer care.">
            <cfsavecontent variable="XMLLinks">
                <LINK dtmf="1" returnValue="#reEnterInfoPage#" destination="#failPage#" />
                <LINK dtmf="2" returnValue="#disconnectPage#" destination="#url.thisPage#" />
                <LINK dtmf="3" returnValue="#transferToCC#" destination="#url.thisPage#" />
            </cfsavecontent>        
            

            <cfset VariablesObject[1] = structnew()>
            <cfset VariablesObject[1]["Name"] = "ErrorType">
            <cfset VariablesObject[1]["Value"] = cfcatch.Type>        

            <cfset VariablesObject[2] = structnew()>
            <cfset VariablesObject[2]["Name"] = "ErrorMessage">
            <cfset VariablesObject[2]["Value"] = cfcatch.Message>

            <cfset VariablesObject[3] = structnew()>
            <cfset VariablesObject[3]["Name"] = "ErrorDetails">
            <cfset VariablesObject[3]["Value"] = cfcatch.Detail>                                    
        </cfcatch>
    </cftry>    
    
    <cfset ReturnObject = printQuestionReturnVariables('stepID',PromptMessage,XMLLinks,url.failPage,VariablesObject)>
#trim(ReturnObject)#
</cfoutput>


<cffunction name="printQuestionReturnVariables" access="remote" hint="Print Question Data For Angel IVR with returnable variables">
    <cfargument name="varName" default="none" type="string">
    <cfargument name="promptMessage" default="none" type="string">
    <cfargument name="linkMessage" default="none" type="string">
    <cfargument name="failPage" default="failPagePlaceholder" type="string">
    
    <!--- This is an array of structures, with keys "name" and "value" --->
    <!--- EX variables[1].Name = Gender --->
    <!--- EX variables[1].Value = Male --->
    <cfargument name="variablesToInclude" default="" type="any" required="no">
    
    <!--- create, scope and set the loop counter variable used below --->
    <cfset var i = 0>
    
    <cfoutput>
        <cfsavecontent variable="ReturnMessage">
            <ANGELXML>
                <QUESTION var="#ucase(varname)#">
                    <PLAY>
                        <PROMPT type="text">
                            #promptMessage#
                        </PROMPT>
                    </PLAY>
                    <RESPONSE>
                        <KEYWORD>
                            #linkMessage#
                        </KEYWORD>
                    </RESPONSE>
    
                    <ERROR_STRATEGY type="nomatch" reprompt="true">
                        <PROMPT type="text"> Sorry I did not get that. </PROMPT>
                        <PROMPT type="text"> I still did not get that. </PROMPT>
                        <PROMPT type="text"> Since I am having so much trouble; please hold while I transfer you to a customer representative who can better serve you. </PROMPT>
                        <GOTO destination="/25" />
                    </ERROR_STRATEGY>
                    <ERROR_STRATEGY type="noinput" reprompt="true">
                        <PROMPT type="text"> Sorry I did not get that. </PROMPT>
                        <PROMPT type="text"> I still did not get that. </PROMPT>
                        <PROMPT type="text"> Since I am having so much trouble; please hold while I transfer you to a customer representative who can better serve you. </PROMPT>
                        <GOTO destination="/25" />
                    </ERROR_STRATEGY>
                </QUESTION>        
                <cfif IsArray(arguments.variablesToInclude)>
                    <VARIABLES>
                        <cfloop from="1" to="#arraylen(arguments.variablesToInclude)#" index="i">
                            <cftry>
                                <cfif structkeyexists(arguments.variablesToInclude[i],'Name') and  structkeyexists(arguments.variablesToInclude[i],'value')>
                                    <VAR name="#ucase(arguments.variablesToInclude[i]['Name'])#" value="#arguments.variablesToInclude[i]['Value']#" />
                                </cfif>
                                <cfcatch type="any">
                                    <cfset ErrorData = structnew()>
                                    <cfset ErrorData.Error = cfcatch>
                                    <cfset ErrorData.Arguments = arguments>
                                    <cfset ErrorData.form = form>
                                    
                                    <!--- You might wanna email this data to yourself or something --->
                                 
                                </cfcatch>    
                            </cftry>                            
                        </cfloop>

                    </VARIABLES>
                </cfif>    
                
            </ANGELXML>
        </cfsavecontent>
    </cfoutput>
    
    <cfreturn ReturnMessage>
</cffunction>

So really what’s happening here is that the page gets called with some URL variables. Those variables are used to construct a google maps http request. That request actually prints out XML. We take the XML and clean it up a little bit and format it. Then print it off in a nice Angel XML package so it can be read by the system. The page just provides one step at a time, and uses a recursive style setup to just continually give directions until there are no more.

If you just want to use the step by step direction giving, you can of course easily remove all the Angel XML junk and just access the #return# variable and do whatever you like with it. This could easily be adapted to an online direction giver, or for Twilio, SMS, whatever. For now it is ColdFusion based, but I may try and convert it to an Apex class in the not too distant future. Depending on how the user is interacting with this, you could even remove the need to have the address pre stored. You could for example have a dedicated number where users just text their current address and you text them back step by step directions to your office or whatever. Really the sky is the limit here.

Here is a sample to see how it works. The demo has some extra stuff to make it more “person usable” instead of stripped down to be consumed by a computer.
Example of ColdFusion/Google Maps/Angel.com dynamic directions

Anyway, hope you guys think this is cool. It was a lot of fun to write!