Does Google Wave herald the arrival of natural language interaction with computers?

I've been spending some time recently thinking about Google Wave, and how it can be useful as a method of communicating and working with multiple participants at the same time, which is what Wave is for, but with a robot as one of those participants.  Wave provides an easy way to incorporate a computer participant in a conversation with people, getting it to receive all updates to the wave and provide its own input.  By doing this it makes performing workflow like tasks much easier, and brings the computer system into the conversation with the humans, rather than having the humans interact with a computer system.

In my example, everybody has a "manager" to which their leave application must be sent before it can be approved.  The bot would detect when you were finished with your application and automatically add in your manager for approval.  This works great, but it needs to know who your manager is.  This information is stored in the HR system, which has its own user interface and logins and whatnot.  If you change manager, somebody with admin privileges on the HR system would need to go in and change your record in that UI.  This breaks the paradigm of bringing the computer into the conversation with humans.

For my demo, I was planning on writing a little user interface for my bot.  It was going to be a web application which allowed any user to go in and play with their manager, so that they could try out the different permutations of the workflow.  Then I got thinking: Why should somebody need to leave the conversation in wave to make changes to the leave system?  Why should they have to interact with yet another user interface?  I already had put in some basic capability for it to tell you your leave balance if you asked it, so why not extend that even further?

Imagine the following scenario:

Bob is having a discussion with Jane about planning for the next release of their product on google wave.  They have a query about upcoming leave for the staff because they might need to cancel leave in order to meed deadlines.  In order to do this, they have to get a list of users on the project, log onto the HR system, and perform queries on each of those users.  They went to the HR systems world, rather than bringing the HR system into their conversation.

Wouldn't it be cool if Bob could talk to the HR system instead?  he could add the HR bot to the wave, and say something like:

Hey HRBot, what is the upcoming leave for the following users:

• Mary 
• John
• Simon

The bot could then parse the question, send the request to the HR system web services, and provide a response directly in the wave so that both Bob and Jane can see the results.  Any leave requests would be listed as links to the request waves that submitted those requests, so Bob could quickly check to see how important that leave is or ask the users if it is okay to cancel that leave.

People have been trying to get natural language systems going for decades, and its still really really hard.  When I was a postgraduate student, I entered the Loebner Prize competition with a colleague of mine, Jason Hutchens.  The purpose of the Loebner Prize is to encourage people to write a bot that is capable of fooling humans that they are talking to a person instead of a computer.  This is impossibly difficult of course, and pretty silly really as Marvin Minsky has pointed out, but its a good bit of fun to write a chatterbot.  The bot that comes closest to being human wins an annual prize of $2000.  Natural language processing was Jason's area of research and he had won the prize before so we decided we'd have a crack.

We lost (curse you Ellen Degeneres!), but sold the software to a german company that wanted to put chatterbot robots on company's web sites as level-0 helpdesk for a sum considerably more than we would have won as prizemoney.  In the end, they had difficulty in selling the software and I don't think it ever really went anywhere.   The range of language that the bot was expected to deal with was very broad which makes parsing it exponentially more difficult, plus websites those days weren't really sophisticated enough to support that level of interaction.

These days, I think there is more of a case for using natural language parsing bots within narrow contexts (to keep the complexity down).  Salesforce have an interesting demo which shows the sort of thing that I am talking about within Wave.  They talk about customer interaction, but I think it will be more useful within an organisation.

In the future, I think we'll see more and more of these bots participating in our business practices.  They can enrich the information we type (for example, by putting in contextual links into what we type, bringing up supporting reports and figures when we are performing tasks, that sort of thing), plus they can become active participants in the procedures while still giving us control over what is going on.



P.S. I feel terrible that I have used cancelling leave as an example here.  I would never advocate cancelling leave as a valid project management technique.  Its a terrible thing to do to your staff, and you should never do it.  Its only used as an example.

Who plays the part of transformation in mashups?

In the last week, two people have independently told me about an Australian government sponsored conference to create interesting mashup applications from government data.  I love the idea, and I'm really glad that the government believes that its data should be freely available.  I think most app providers are realising the power of providing open access to their data to drive adoption now.  In my opinion however, independent transformation of data between web applications is still missing as a generic tool to mashup creators.

Generally, in enterprise as in mash-up applications, the source data is not in the correct format to be directly consumed by the final application.  As an example, I am writing an iPhone application which takes heart rate monitor recordings of your exercise and stores them as a Google spreadsheet.  The reason I chose to store the information in an online spreadsheet instead of a bespoke database service is that google already provide all of the tools to make the available in the spreadsheet easily available as XML for others to consume.  It does this using the Atom protocol, which is great, but hardly easy to consume.

Traditionally, a mash-up is seen as the combination of data from one or more external sources with a javascript driven user interface.  The data flow looks something like this.

This is great, however it induces a great amount of coupling between the components.  The mashup provider needs to communicate directly with the data sources, transform them into its own native format, then consume it.  There's no opportunity to substitute in a different datasource if it becomes available, or easily fix things if the source data format changes slightly.  In enterprise applications, this has long been recognised as an issue and ESBs were developed as a way of handling this.  When an ESB is used correctly, the source data (or application) is abstracted from the destination by a transformation process, usually performed by XSLT.

I think that the same approach should be used for mash-up style applications.  The big advantage this brings is that it releases the data from the application (and the user interface).  More importantly, it allows the application itself to fetch data from different sources.  It is no longer limited to the sources that the programmer's put in.  A sufficiently talented user can take any datasource, transform it into the correct format that the application expects, and then get the application to point at that transformed source.

For this to work, there is a need for a generic XSLT service, that can take a data feed and an XSL style sheet and produce the desired output.  W3C provide a service which does exactly this.  Unfortunately the bandwidth requirements of any large enterprise use would crush their server so they have put limitations on the service to restrict it to personal use.  This is a shame.  I've written a very similar service for bleep, but it is run as a free Google App Engine app which has quite severe resource limitations of its own.   I reckon Google should release a transformation service of its own.  It would be very useful in many of its apps.  There's no way to make advertising revenue off it though :-/

Its not really within the average person's skill set to write a transform.  Many software engineers do not know how to do it properly.  In the future, I'd like to hope to think a web app will be written which brings it more within the reach of normal internet consumers.

To bring this back to the govhack conference I mentioned at the beginning of this post, I think that its good that the government wants people to make mashups, but in some ways they are a little misguided.  Its not just about the applications.  Most people will just put some data up on a google map, which is hardly innovative.  Instead, what I would like to see people taking the source data and transforming it and correlating it against other data sources to produce new data sets.  Then its possible for any number of people to take that data and visualise it in cool and impressive ways.

For Bleep, some of my random thoughts on data transformation and visualisation have been gathered at this page

Using Google Wave for Workflow tasks

I've been thinking over the last few days about what Google Wave could be used for. Obviously it can be used as a document collaboration and review platform. It can also be used as a multi-user chat program, although there are probably other existing programs that are just as good for managing that. Some people have claimed that it isn't anything revolutionary. In and of itself, this is true, as it just takes concepts already available in email, instant messaging and collaborative documents and puts them together. What I'm more interested in now however is what we can do with it now that it has brought those technologies together.

In a blog post, Daniel tenner quite succinctly outlines where Google Wave will be useful, and primarily its going to be used by people working together. Why couldn't we use it to do workflow related stuff too, especially when there is an automated component to it? I decided to look at how we could do leave forms. The first question I asked was "why would you want to do leave forms in wave?". There are already web applications for doing workflow. The problem that I see with that is that they are still fairly rigid in their operation. Whenver your manager is on leave or you want to go slightly outside the pre-defined workflow thats there the procedure becomes very brittle and you need to go to a user with superuser access to get things done. In addition to this, all notifications that you have new forms to attend to or that the status has changed go out by email. Email shouldn't be a notification system. We just use it that way because its familiar, and it is the tool that we spend most of our time in.

Assuming that google wave becomes popular enough that we use it each day, wouldn't it be nice if the workflow/collaboration tool entered into our messaging tool. That way we wouldn't need to log into a separate tool to manage things. It would be there in front of us, and allow us to deal with the situation immedaitely.

To test this theory, and also to play with Google Wave bots, I have written an extension to Google wave, which gives users the option to create a Leave application in google wave. Interestingly enough, I found it easiest to document the procedure directly in Wave, which would make it very easy to introduce new users to our procedures. The procedure wave is available at Leave Application Procedure. One nice thing about this is that if HR needs to change the procedure, It automatically pops back up in people's inbox so that they see the changes. There's no need for notifications to be sent out, as the change to the document (to which everyone is subscribed) automatically gets distributed. Likewise, if a staff member has a question on the procedure, it can be asked directly in the wave itself (privately if necessary) to provide context to the conversation.

When a user creates a new leave form wave, it automatically includes a bot which is responsible for progressing the wave through the workflow. This is done by a series of buttons (or actions) at the bottom of the document which take the standard approval route (Draft -> Submitted -> Approved -> Processed). It is flexible however, as anyone that needs to deviate from the process, say to get additional approval from another team leader because he is to another team, can simply add the other team leader to the wave and have a chat. All of the context associated with the process are kept with the wave, and it can easily be searched later on to see what happens.

Its also possible for the bot to take a greater role in the process itself. It can check leave balances (assuming that the leave system is available to it), add leave to the company leave calendar, and any number of other integration tasks because it is the thing that is managing the workflow itself. Its very flexible, easy to change, and completely under the control of the organisation. One thing I played with was getting the bot to understand (as best as bots can) natural language. If you wanted to query your leave balance for example you could start a wave with the bot and ask it "What is my leave balance". It would could then look up your balance and reply. This would be able to free up HR staff from having to perform mundane tasks. Obviously bots have got a long way to go before they can understand our language properly, but if queries conform to simple grammar rules then it should work.

If anyone would like to have a play with what I have written, let me (brucejcooper@googlewave.com) know and I will add you to the procedure wave, which will allow you to install the extension and create a dummy leave application. It puts me in as the approver for everyone (except for me, for whom it uses another bloke) as their manager. Only the manager can approve/reject a timesheet. If you say anything to the bot, it replies with a message about your leave balance (which is bogus. the bot isn't connected to our HR system).

What does everyone think? Is this a good idea for managing workflow? Until we have broader access for people to get on Wave it will be difficult to tell, but I think it is a good use.

P.S. I wrote this post in Wave originally, but had to copy and paste it here because the embedding API doesn't allow anonymous viewing yet.  Google Wave is great, but it is still definitely a beta product.

UPDATE: I've created a screencast demonstration of how the flow will work.  Remember this is intended as a proof of concept rather than the full thing, but it gets the point across.

Interesting article on app pricing

Gizmodo have an interesting article on the price of iPhone applications, how they have dragged the consumer's expectation of app pricing down, and how this might not be a good idea in the future.

I can certainly say that I won't be expecting to make much money out of Bleep. Its taken longer than expected to develop, and I don't think I'm going to sell lots of copies. How anyone can make a business out of developing these things is beyond me... I suppose we'll see :)

Another Heart Rate monitor device/app for the iPhone

There's another company that is producing a heart rate monitor device for the iPhone. Its outlined at fastcompany.com, and looks great. Its exactly the sort of thing that would render Bleep irrelevant. Sadly, its not going to be made into a product at this stage :(

New blog format for the crimson cactus

When I started Crimson Cactus, I started up a blog for it. It seemed to make sense, and I could keep the company posts separate from holiday pictures and musings and whatnot that way. As its turned out, I find that I want to cross-post. That is, I want to be able to post to both of my blogs.

There are a couple of ways of doing this, but I think it just goes to show that I'm doing it wrong. What I really want is to continue posting to my personal blog, but tag those posts that are of interest to the crimson cactus, and then have my web page pick up that tag. This is exactly what I've done now, so now all of the old posts have been imported into my personal blog, and thats what you'll be seeing from now on.

For those of you that follow via RSS/Atom, The new link to use is http://feeds2.feedburner.com/brucecooper/Pqee

Fitbit review on engadget

Engadget have released a review of the fitbit networked pedometer. I remember first seeing this about 12 months ago, when it had just launched at techcrunch 50. I like the idea of the device, but it is yet another thing to carry around, and yet another thing to charge.

It makes me thing of the belt valet computers in David Marusek's Counting Heads (Thanks for the reccomendation @doctorow). One day, not too far away, we will have computers that we carry, strapped to our person (perhaps in belt form), that can handle all of the biometric sensing that we want. It will be able to count our steps, work out our heartbeat, and record everything that we hear and see for future reference.

I'm really excited about the prospect, even if it does open up a lot of privacy concerns. It will be important that we as individuals retain control of the information that is being collected. I'm pretty sure that google having access to all of my heartbeat (which is basically an EKG) information would be a bad idea. David's sequel book, Mind over Ship shows us a very good example of the mis-use of complete information on people.

Costumes from fancy dress party

Melissa and I went to a Royalty themed fancy dress party last night, so we went as Louis XVI and Marie Antoinette. Here's a couple of photos.