Augmented Reality Shopping

Augmented Reality Forms is new (but experimental) and is coming up in ICEmobile 1.3. There are a lot of potential applications, but one that appeals to nearly everyone is an Augmented Reality Shopping Cart. You can see it in action here:

There are two sides to the application: the web view, and the augmented reality view, but with ICEmobile you develop both of them in the a similar, familiar way. In this case we developed the web view with ICEmobile and ICEfaces, but could have used JSP or HTML5:


There are only ever four items in the demo shopping cart, so the item references are hand-coded. In a real shopping cart you would pull these from a collection, but this gives you the idea:

    <mobi:augmentedReality buttonLabel="View Augmented Catalog"
                value="#{realityBean.markers}" var="marker"
                markerModel="#{marker.model}" />


   <mobi:outputListItem styleClass="aritem">
       <h:graphicImage library="images" style="width:40px"
       <h:graphicImage library="images" style="width:20px"
       <h:graphicImage library="images" style="width:20px"

The augmented reality form looks like this to the user:


but is developed just like a web page, albeit with different form controls (toggles and buttons):

<targets database="mobileshowcase.xml">
  <target name="page3" >
    <toggle name="icecar" value="off" 
        on="circlecheckon.obj" off="circlecheckoff.obj" 
        scale="1.5" location="170.0, 174.0, 190.0, 164.0" />
    <toggle name="iceskate" value="off" 
        on="circlecheckon.obj" off="circlecheckoff.obj" 
        scale="1.5" location="-294.0, -23.0, -274.0, -33.0" />
    <toggle name="icesailer" value="off" 
        on="circlecheckon.obj" off="circlecheckoff.obj" 
        scale="1.5" location="-149.0, -72.0, -129.0, -82.0" />
    <toggle name="icebreaker" value="off" 
        on="circlecheckon.obj" off="circlecheckoff.obj" 
        scale="1.5" location="220.0, -29.0, 240.0, -39.0" />
    <button name="buy" model="buybutton.obj" 
        scale="2.0" location="10.0, -140.0, 20.0, -150.0" />

The catalog page is recognized by the Vuforia image target configured in “mobileshowcase.xml”, obtained from the Vuforia target manager. Each catalog item is represented as a toggle with two states (“on” and “off”) at the specified image coordinates. We use the same 3D model in each case, but could have had unique models for every item. (However, this would have increased the graphic design effort required for the catalog page; it depends on the experience you want to create. An intermediate approach is to use the same simple “3D” rectangle model, but change the texture map to uniquely reflect the catalog item — just using a different texture map on the same rectangle model is how the toggles and the buttons were created.) There is also a “buy” button that completes the interaction and sets the augmentedReality component value to the URL-encoded representation of the form, such as “buy=buy&icecar=on&iceskate=off”. The AR Form data is then decoded in the backing bean to be displayed and added up in the web shopping cart.
(Isn’t the Augmented Reality view really part of the web now?)

There are so many interesting directions to take this. The first step is to refine the AR Form language. Should it be more or less like HTML? What controls should be added? Should it be standardized? Should there be ICEfaces tags for AR forms? (Yes.) After that we will need tools specifically for designing AR forms. Without question, this is the beginning of an entirely new industry.

Leave a Reply

Your email address will not be published. Required fields are marked *

5 × three =