Easy Augmented Reality with ICEmobile

If science fiction is any indicator, future computer user interfaces will include “augmented reality”, where the user’s view is enhanced by application data. Augmented Reality applications are now very easy to develop with ICEmobile using the new <mobi:augmentedReality> component and the ICEmobile client.

In visual terms, Augmented Reality will take essentially two forms (auditory and haptic augmentations being yet another topic). The basic process is to overlay virtual objects on the user’s view based on either detected objects or patterns in the view or the known position and orientation of the view. We’ll leave advanced image recognition for a future version of ICEmobile (or maybe the self-aware machines as above), so for now the technique is to allow your application to place text and image labels in space and overlay them on the camera view using knowledge of the camera’s position (from GPS) and orientation (from compass and gyroscope). (Future versions will allow the overlay of three dimensional virtual objects rather than just labels, but labels are a good starting point.)

So, it’s very easy to add the Augmented Reality feature to your application: just determine the latitudes and longitudes of a set of image labels, and make them available to the component or JSP tag for ICEmobile to do the rest:

<mobi:augmentedReality value="#{bean.location}">
   <mobi:selectLocations value="#{bean.locationList}"
     var="place"
     locationLabel="#{place.name}"
     locationLat="#{place.latitude}"
     locationLon="#{place.longitude}"
     locationAlt="#{place.altitude}"
     locationIcon="#{place.icon}"
   />
</mobi:augmentedReality>

This works just like <h:selectOneMenu> (except in 3D) where the value is the label of the item selected by the user. (This allows the user to choose one of the items from their view.)

The ICEmobile client knows the user’s current location and orientation, so is able to render the labels in space as expected.

At this moment, the ICEmobile client supports this on the iPhone, but the JSF component is not yet complete. For now, a very simple JSP tag is available in the trunk code:

<mobi:augmentedReality id="selection"
    style="float:right" params="${reality}"/>

(This is not the final form of the JSP tag either, since here we are relying on application code to format the list of locations for the client — “params” will be replaced with a “locations” attribute and EL-specified attributes as above.)

What can you do with this? It’s really just a different way to display mapping data with selection, so the possibilities are tremendous: help tourists identify mountain peaks or find points of interest; help people find shops, restaurants, or each other (by updating location data dynamically); play geocaching games; collaborate on location-dependent tasks (“I’m taking the backhoe to the north lot”); and others. At the moment, the real limitation stems from the accuracy of the location data. If we had meter-resolution, we could build very interesting office, warehouse, and construction applications (“where did you install that printer?”). If we had centimeter-resolution, the possibilities are even more interesting (“where is that conduit?” “turn that light on”).

In other words, the next step is for every wifi base station to provide differential GPS.

Leave a Reply

Your email address will not be published. Required fields are marked *

5 × three =