Category: calendar

WebFinger, OAuth and Freebusy lookups

One of the more frustrating aspects of calendaring systems is that the freebusy lookups are all proprietary.  Meeting invitations can be sent from one system to another (assuming you know a time to meet).  However, it is not possible to lookup when someone from Lotuslive, someone from gmail and someone from Yahoo are all available to meet.  In the corporate space this type of scheduling is invaluable.

The format for looking up someone’s freebusy time is included in a standard that was completed in 1998, but they punted on all the hard stuff.  The hard bit, as I have mentioned before, is working out where someone’s freebusy is stored on the web and then authenticating with that store in a manner that can be verified. WebFinger and OAuth are now putting the complete round trip within spitting distance.

Below I’ll propose an approach to scheduling a meeting with my mom (who uses gmail) from LotusLive (which I use). I will be rob@robubu.com (but using LotusLive for my calendaring service) and my mom is mom@gmail.com.  We’ll also assume that my mom has told google that it can share her calendar free time data with any one in her contact list and that I am in her contact list.

  1. I head into my calendar service (on lotuslive), click on Add Event and type mom@gmail.com into the invitees list.
  2. LotusLive now uses WebFinger to lookup the different api services that google provides for access to my mom’s data along with the corresponding URL for the service.  The details on how this works are outlined here on Eran’s blog. At the end of this, LotusLive gets back a XRD document that looks something like the following.

    <?xml version='1.0' encoding='UTF-8'?>
    <XRD xmlns='http://docs.oasis-open.org/ns/xri/xrd-1.0'>
        <Subject>acct:mom@gmail.com</Subject>
        <Alias>http://www.google.com/profiles/mom</Alias>
        <Link rel='http://portablecontacts.net/spec/1.0'
              href='http://google.com/api/people/' />
        <Link rel='http://ietf.org/icalendar/freebusy'
              href='https://google.com/api/calendar/mom/freebusy/' />
    </XRD>

    From this LotusLive can now determine that my mom’s freebusy endpoint is at https://google.com/api/calendar/mom/freebusy/.  It concludes this by looking for the link with a rel attribute of http://ietf.org/icalendar/freebusy

  3. If my mom had made her freetime calendar data public then LotusLive can simply retrieve the data from the URL, but to add to the complexity let’s assume that it requires authentication i.e. LotusLive needs to prove to Google that it has rob@robubu.com at the browser and then Google checks that rob@robubu.com is in my mom’s contact list. We’ll do something here very similar to what signed fetches do in opensocial i.e. lotuslive will use OAuth to assert that it has rob@robubu.com at the browser. What we’ll end up with is a url that looks something like

    https://google.com/api/calendar/mom/freebusy/
    ?opensocial_viewer_id=rob@robyates.com
    &xoauth_public_key=https://lotuslive.com/keys/publickey.crt
    &ampoauth_signature_method=RSA-SHA1
    &oauth_signature=djr9rjt0jd78jf88%26jjd99%2524tj88uiths3

    LotusLive has here claimed that it has rob@robubu.com at the browser and using OAuth has signed the request with a private key.  It has also indicated where the public key is to validate the signature.

  4. Google receives the request, retrieves the public key and verifies the signature.  If it trusts signatures and keys from LotusLive (verifiable by retrieving certs from an https url with a lotuslive.com domain) then it is done at this point. However that is a fairly large amount of trust to place on LotusLive as LotusLive could assert on behalf of any identity. Google really needs to check that LotusLive can assert rob@robubu.com’s identity.  Here we’ll use webfinger again.
  5. Google now does a WebFinger lookup on rob@robubu.com and gets an XRD document such as the one below

    <?xml version='1.0' encoding='UTF-8'?>
    <XRD xmlns='http://docs.oasis-open.org/ns/xri/xrd-1.0'>
        <Subject>acct:rob@robubu.com</Subject>
        <Link rel='IDP' href='    https://lotuslive.com' />
    </XRD>

    Google now sees that lotuslive.com is a valid Identity Provider for rob@robubu.com and so accepts the assertion.

  6. Google checks that rob@robubu.com is in my mom’s list of contacts and as I am returns her freebusy.
  7. Finally, LotusLive gets a response from Google outlining my mom’s free time and displays it in a nice calendar.  I can choose a time that she is free and send her an invite.

I know this is not perfect and I know there are probably a fair amount of changes that are needed, but I wanted to jot down something that, I think, is fairly close to a workable solution.  Am very interested in other’s thoughts.

p.s. WebFinger on email addresses does provide a means of discovering valid email addresses, but no where near as much as this does.  The fight against spam can’t center on not making email addresses discoverable.

Freebusy and Yadis

So for a while now, we have been trying to figure out a standard means to lookup someone’s freetime from their calendar (called freebusy in the calendaring world).  We knew we wanted a REST service that could be passed url parameters, we even had a demo one up on the net.

The problem, though, had always been how to figure out the url for the service given the user’s e-mail address. Openid, DIX, yadis et al always showed promise, but it always felt clunky to have to first translate an e-mail identifier for the user into an http url based identifier for the user and then ask for an attribute for the calendar service etc. etc. A recent proposal to the yadis mailing list, however, showed the way.  Simply resolve the e-mail address to its domain and use the yadis protocol (section 6) on that to discover a freebusy service for all the members of the domain.

So, for example, let’s say that the e-mail address of someone who I want to schedule a meeting with is rob@robubu.com.  Use the yadis protocol on http://robubu.com, i.e. retrieve the page at http://robubu.com and dereference the "X-XRDS-Location" meta tag in the html head to get back a yadis document (that looks something like this).

<?xml version="1.0" encoding="UTF-8"?>
<xrds:XRDS xmlns:xrds="xri://$xrds" xmlns="xri://$xrd*($v*2.0)">
  <XRD>
   <Service>
    <Type>http://ietf.org/cal/freebusy/1.0</Type>
    <URI>http://robubu.com/calendar/freebusy.php</URI>
   </Service>
  </XRD>
</xrds:XRDS>

Then extract the endpoint for the freebusy service by looking for the URI that corresponds to a service of type "http://ietf.org/cal/freebusy/1.0". Finally, construct the url request that returns the freebusy time in iCalendar format for a given period e.g. http://robubu.com/calendar/freebusy.php?email=rob@robubu.com&start=20070101&end=20071212. Done.

I’ve also hacked a little on webcalendar and got an end point up and running.  The calendar in html is here, but if you want to schedule a meeting with me via yadis and the freebusy api then my e-mail address (for the purposes of the demo) is rob@robubu.com.

Finally, I do want to use the URI Template approach for the URI, but I’ll leave that for another post.

CalAtom – draft

So here is the first rough draft of the CalAtom spec.

We’ve made a few changes since the earlier posts on CalAtom, probably the biggest is the fact that CalAtom clients and servers MUST support xcal representations of calendar data.

I expect this to change a fair bit before formally posting the spec (I, at least need to spell check it), but this provides a general idea as to how it is all going to work. Feedback and anyone interested in implementing, helping out with the spec etc. are all very welcome. Please contact me or just respond to this blog post.

I know already that it is out of date as it is dependent on two specs that weren’t publically available when it was written. One now resides here and another one I believe is being published later today, I will update this post once that happens.

Freebusy Demo

As a follow up to the Freebusy Lookup posting I wanted to point out a demo that Dan Gurney has put together on top of Domino.  This approach to free busy is similar to the one outlined in the Freebusy Lookup post.  Here are the details for how to get yourself setup with a Calendar and the try out the freebusy lookups. Feedback is very very welcome. 

Please note that the demo server will only be available for the next few weeks,

From Dan…..

Good news!

IBM will be hosting a "Freetime" servlet running on a Domino server. You’ll be able to create your own mail files and create/update/remove calendar entries.

You can create your own demo account at any time.

To create a demo account:
1. Go to http://www.notes.net
2. Click the menu: Lotus – Live Demos
3. Click the link: Lotus Domino Web Access
4. Go thru the legal stuff, give your account a user name and password, and voila … you will have a Domino mail file. Domino Web Access is (hopefully) a straight forward web application that allows you full control over your mail database and its contents.

I have a demo account that you can schedule meetings with. I’ll set the account to auto-respond to all requests:
Spider_Man@showcase2.notes.net

Once you have your account, you can then get free-time information via a <free-busy-request/> POST (there is currently no way for Domino to enable the "REPORT" verb) or a simple GET request.

Here is the URL for any account on this test server for POST or GET:
https://showcase2.notes.net/servlet/Freetime/<email>

where <email> is your RFC-821 email address. For instance, my demo account can be reached like this:

https://showcase2.notes.net/servlet/Freetime/Spider_Man@showcase2.notes.net

You can pass dates as optional parameters for a GET request:
https://showcase2.notes.net/servlet/Freetime/Spider_Man@showcase2.notes.net?start-min=2006-07-12&start-max=2006-07-14

Start-min and start-max are ISO-8601 dates. You can use partial or full ISO-8601 dates (I am showing my age here … there is probably another name for the CalDAV date format now).

Set the "ACCEPT" header to "text/icalendar" if you wish to receive iCalendar instead of xCalendar.

Freebusy Lookups

I attended part of the CalConnect roundtable in Cambridge this week, had some great discussions and also got to see the group’s reaction to CalAtom. It was met with very mixed reactions and we definitely left a little bruised but I think there is some real potential and interest there.

Anyway this post wasn’t intended to be about CalAtom. Instead it is on freebusy lookups, i.e. how do I find out when you are available to meet. It was pointed out to me that one of the first interoperability aspects that any calendaring standard faces is to figure out when you are free so that I can schedule a meeting with you. If that can be accomplished then passing around iCal files in email messages can get us a first stab at interoperability. I can see when you are free, send you a meeting invitation, you can accept it and I get told you accepted. Not too shabby, although admittedly a long long way from having this work with any calendaring client server combo. It’s surprising that an existing standard for this doesn’t exist yet, but having researched some implementations I presently cannot find one. Please post responses if you know of one that I have missed.

I then looked in detail at both Google’s approach and CalDav. I first looked at what google had done. They provide a “projection” for free-busy. You can get your free-busy back as a feed, here’s a snippet.

<snip/>.....I removed the feed header and shortened the urls in one or two places as well
<entry>
 <id>http://www.google.com/4ca650bc087d/free-busy/4g0ncm350mc5pv8487fd8megmo</id>
 <published>2006-04-17T15:00:00.000Z</published>
 <updated>2006-05-02T18:23:59.000Z</updated>
 <category term="http://schemas.google.com/g/2005#event"/>
 <link href="http://www.google.com/calendar/0eWF0ZXNAZ21haWwuY29t"/>
 <link rel="self" href="http://www.google.com/calendar/fee8487fd8megmo"/>
 <gd:when startTime="2006-04-17T08:00:00.000-07:00"
   endTime="2006-04-17T10:00:00.000-07:00"/>
</entry>
<entry>
 <id>http://www.google.com/calendar/feeds/</id>
 <published>2006-05-25T22:00:00.000Z</published>
 <updated>2006-04-28T20:05:27.000Z</updated>
 <category term="http://schemas.google.com/g/2005#event"/>
 <link href="http://www.google.com/calen1haWwuY29t"/>
 <link rel="self"  href="http://www.google50eotvfa8qgronj9oje08"/>
 <gd:when startTime="2006-05-25T15:00:00.000-07:00" 
  endTime="2006-05-25T16:00:00.000-07:00"/>
</entry>

Now I am a BIG BIG fan of ATOM, but there is a time and a place. On the plus side this was really easy to do and with any old xml parser I can make sense of it. I should also have been able to use standard atom parsers with it, but…. Well, first off this is not valid atom. Any atom parser will barf at this as it doesn’t provide a title for the entries. It is also extremely verbose containing information that is not needed by clients that are only interested in your free time. Contrast it with a similar ical representation.

BEGIN:VFREEBUSY
ORGANIZER:jsmith@host.com
FREEBUSY:19980314T233000Z/19980315T003000Z
FREEBUSY:19980316T153000Z/19980316T163000Z
END:VFREEBUSY

hmmm, not a great start, so next up was CalDav. The relevant section is 7.9 and if we ignore some complexity due to depth and a few other things and instead head straight onto the example, then this seems much more reasonable, very straightforward. On deeper examination though, the first thing that stands out is the use of the REPORT method. Ignoring very valid firewall concerns and ploughing straight into using it I first went in search of a java library to use. The standard library of choice for the java programmer when using http is the apache commons httpclient. Uh oh!, so no luck there. So off to google which eventually leads me to the Slide WebDav client and a solution, yeh!. It’s slightly concerning though to see that it has not been updated since 2004 and more worrisome that it still depends on a version of httpclient that was end of life’d on 27th Feb 2006. But let’s assume that this works, onto the next bit if trickery, how to parse the response. Writing a custom parser seems wrong, someone must have done this, so off to google again and I find iCal4J, yippeeee! now all I have to do is learn how to use it and ensure that I can usher this licence past my employer’s legal department and I am flying. What’s that? you want to also do this in php and ruby? and you don’t know of any libraries available for those languages?

This is too hard, too hard. There are plenty of examples of complicated http based api’s and ALL of them, ALL of them are easier to consume than this, and why is this? It’s because they use standard http verbs that all the language http utilities support and that they represent their data in a format that is accessible by parsers that all the languages support.

Contrast this with the following approach (that I will submit for discussion to the CalDav mailing list).

GET /bernard/work/freebusy?start=20060104T140000Z&end= 20060105T220000Z
Host: cal.example.com

response

HTTP/1.1 200 OK
Date: Fri, 11 Nov 2006 09:32:12 GMT
Content-Type: application/calendar+xml
Content-Length: xxxx

<vcalendar>
 <version>2.0</version>
 <prodid>-//hacksw/handcal//NONSGML 1.0//EN</prodid>
 <vfreebusy>
  <dtstart> 20060104T140000Z </dtstart>
  <dtend> 20060105T220000Z </dtend>
  <freebusy fbtype="BUSY-TENTATIVE">20060104T150000Z/PT1H</freebusy>
  <freebusy>20060104T190000Z/PT1H</freebusy>
 </vfreebusy>
</vcalendar>

As a consumer I can now use HttpClient and the bundled xml parser in java, that I have used a thousand times before, done.

This has certainly been an interesting exercise for me. One of the claims I had leveled at CalDav, in an attempt to justify CalAtom, is that CalDav was just too hard. I stand by that claim. For a product group to incorporate just the freetime lookup it seems they have to convince management that it is ok to depend on a end of life’d jar file, that it really isn’t too hard to write or learn a custom parser for some proprietary data format and that as other programming languages gain popularity that there will, of course, be the base utilities in those languages to also use these features. I think that this is presently a very hard sell.

CalDav may indeed lead to a protocol that allows calendaring systems to interoperate with each other, but I wonder whether that’s enough. To be successful doesn’t it also need to interoperate with emerging web applications? and, more importantly, be consumable in a similar way? In sampling many of the current approaches there are clear patterns emerging, namely the use of http 1.1 methods and xml/json data packets. These patterns are strongly supported in all the major languages and will continue to be for the foreseeable future (this is where it’s presently at).

With the majority of http applications heading down a different path, are the custom parsers, end of life’d jars and special firewall compensations, that CalDav requires, really going to be tolerated? I’m not sure, but all this seems to beg the question “what’s so special about calendaring?” 30Boxes, google, evdb, yahoo nothing?