Spark Minute » Wireless

What you can and can’t do (but soon will do) with augmented reality

David Spark — Tue, 11 May 2010 21:25:46 +0000

Last week I attended a fantastic presentation by Gene Becker (@genebecker) of LightningLaboratories, an augmented reality research and consulting firm. I’ve been fascinated by augmented reality (AR), and after attending Becker’s presentation at the Web 2.0 Expo in San Francisco, I felt I got the full lowdown of the current state of AR, the challenges, and its hopeful future. Much of this post is a summary of what Becker discussed in his presentation. Here are his slides.

Experience Design for Mobile Augmented Reality

View more presentations from Lightning Laboratories.

What is augmented reality?

As Becker explains, “Augmented reality is the display of computer graphics and media, overlaid and registered on real-world environments, interactively in real time.” That simple explanation requires a lot of wizards behind the curtain. There’s a confluence of multiple technologies and disciplines to make AR appear as a seamless experience. AR is comprised of computer graphics, machine vision, sensing and sensor fusion, geographic information systems, mobile systems, ubiquitous computing, and the web, explained Becker.

While we can realize a certain level of AR today, we’re constantly striving for improvement in these technologies and greater synergy.

History of virtual reality

Becker gave a brief history of augmented reality, covering the work of Steve Mann and his wearable computer in the early 1990s, and the Touring Machine from 1997. Becker claims that modern AR has been around for 15 years. But back then to achieve augmented reality the user had to carry a backpack full of gear. Today, those capabilities that filled a backpack – your orientation, direction, view of the world, and display over the world – can now fit in your iPhone, Android phone, or Symbian device.

Uses for augmented reality

For each of AR’s uses, Becker poses a series of questions that a developer has to contend with when trying to effectively achieve and cope with that level of augmented reality.

Visual search or querying the natural world: You can look at a building and augment it with printed text, or maybe call up a historical photo. Question is how do you go about recognizing things in the world? How do you then translate what you know of the real world and make it interesting?

Augmented Reality ID or Social Augmented Reality – If we can identify people in a room, we can understand more about them and their relations. There’s already an example of this with TAT, a new AR technology that IDs the people in the room through facial recognition. Once you positively identify a person, you can access their social account, see that person’s friends (do you have any in common?) and see personal public information.

Problem is augmented reality touches upon some new social and privacy issues. Becker points out that ARI ID is the collapse of social nuance and the end of personal privacy. Someone can stand in front of you and query your social profile while they’re sitting there. It’s more than just seeing you right here in the moment, but now I have a window into your entire background. Social Augmented Reality redefines what it means to be public.

While having your personal information completely public wherever you go may creep some people out, most are slowly becoming accustomed to this type of recognition. Many embrace it. They’ve chosen to be public online, so it’s only natural that they would want their public information to be portable with them. The question is how will others use that information.

We’ve been creeping to the point of accepting this kind of behavior. Already people are accustomed to having their photo taken from a mobile device, uploaded to Facebook, and then tagged with their name. That’s a manual tagging. Soon it will be done automatically. And once you can make that connection, it opens the path to a plethora of other personal data and connections. Becker argues that we’ll need this technology to be built into glasses before it can be more socially acceptable. 3D movies and TV are trying to move us towards this natural concept. Becker predicts that we’re five to ten years away from Terminator-like vision capability.

What’s available today

Layar – Utilizing your GPS and compass, this system knows where you are and what direction you’re pointing. The majority of iPhone-enabled AR applications work like Layar. For the Layar application they look at the Latitude and longitude coordinates of you and where you’re looking. The dots on houses allow you to call up the address and also the price.

Problem with this application is its use is not realistic. People would use this product for a moment, but not continuously. You can’t expect someone to walk around with their phone out in front of them. Inclusion of this technology in glasses is necessary.

Metaio – This technology lets you see 3D-augmented items on top of physical items. In one example with Lego, you can hold up a box to the camera and it will show you on top, in virtual 3D, what the item inside will look like, once you make it.

Becker stressed that AR is not just a new technology, but rather a new medium of creative expression that expands human experience. It’s a mashup of all sorts of information on top of our physical world: Web, images, sound, movies, games, search engines, databases, advertising, and social media connected to people, places, and things. AR can reveal invisible stories all around us.

What makes a compelling AR experience?

Becker explained that there are three dimensions that create a compelling augmented reality experience. A true AR experience allows for flow across all these issues. Becker talked at great length that success in AR requires complete flow.

When people have a good experience with AR, they talk about these three aspects:

Challenge/Expression – physical/mental challenge, self-expression, and creativity
Drama/Sensation – physical stimulation, mental/emotional stimulation, and imagination
Social – competing with others, bonding, and cooperation

All the tools out now are only good at solving one maybe two of these aspects, said Becker.

The reality of AR experiences today

Here’s Becker’s estimation of what does and doesn’t work with AR today.

What works

Physical engagement
Visually captivating
Emotional “wow” factor
Invisible stories, revealed

What doesn’t

Awkward use models
No flow opportunities – Just point situations. Not continuous.
No social dimension
Point experiences only

Mobile AR issues and limitations

Technical limitations

Sensor accuracy
Indoor positioning
Line of sight
Device constraints – Eats up battery life
Lack of interoperable standards – No cooperation on what is a “point.” It’s a lot more than just latitude and longitude.
Relative positioning – Things that we’ve come to expect from an experience on the screen, don’t necessarily work in AR.
Object recognition and tracking
Graphics issues

Use model issues

Unnatural physical position
Ambiguous social signals – What are you doing that you’re pointing this at me?
Cognitive model – What am I seeing here? How do I interpret this?

Authoring Limitations

Platform specific
Low level tools, complex workflow
Focus on point interactions. No support for stories or interesting interactivity. How do we create a flow?

An example of a compelling AR experience

Becker showed us this concept video from HP that shows how a great AR experience could play itself out.

The video addresses all the issues of what makes a compelling experience.

Challenge/Expression – Running to get to your destination. Finding your way through the city.
Drama/Sensation – There’s a storyline. How should you get there? Will you find the locations in time?
Social – Racing and competing against other people.

Mobile AR experience design practices

Becker summarized what you need to be thinking about when developing an AR experience.

Understand limitations and design around them. You’re looking create flow and physical drama.
- Design for inaccuracies in the system.
- Emphasize audio for immersion, continuity, and story.
- Use “magic lens” position sparingly. Don’t need to be looking through it all the time.
- Fail gracefully and transparently.
Design for challenge, expression, and flow.
- Focus on experience, not technology. Have your experience through the device, not of the device.
- Goals should be challenging.
- Set clear goals, expectations, and rules.
- Encourage focus and concentration. Limit distractions and gratuitous things. Don’t overload the individual with ads.
- Make activities rewarding.
- Provide direct and immediate feedback.
Design for physical sensation and emotional drama.
- Scout the locations, know the environment and context deeply, sights, sounds, smells, textures, weather, rhythms, and moods. It’s like filming on location.
- Design for “magic moments” – mental immersion, physical/virtual coincidence.
- Emphasize sound design for immersion, continuity.
- Develop a cross-media story.
Design for social bonding, sharing, and competition.
- Game mechanics drive competitive engagement – Create multi-player competitions, scoring, and leveling systems
- Social media integration to share broadly/vicariously.
- Tight and loose multi-person activities encourage bonding and shared experience.

Bring an interdisciplinary media production mindset

Take all jobs you might have with film, gaming, and web development, mash them together and you’ll have the team for your AR project. To be successful in AR, you’ll need to reach from multiple creative disciplines.

Photo credit: CC Estaban Trigos

VIDEO: Siftables: Toy blocks that are aware of each other

David Spark — Sat, 14 Feb 2009 21:02:14 +0000

This is a video from the recent TED conference in Monterey. Siftables are these blocks with computer chips, software, a screen, and near field communications. The combination of all allows them to be aware of each other and interact learning math, playing a Scrabble-like game, and connect with a program playing on TV. My description alone can’t do it justice. Watch the video, they’re pretty darn cool.

This news item is for the Spark Minute week of 2/16/09 which can be heard daily on Green 960 and 910 KNEW in San Francisco, CA.

Airplanes and BART are no longer an oasis for no Internet access

David Spark — Sun, 01 Feb 2009 01:35:13 +0000

I actually sometimes like to commute. It gives me a chance to unplug and do all the things I like to do when I’m not online, such as sleep, think, read, listen to podcasts, or watch movies on my iPod.

Unfortunately, the days of commuting being an oasis from connectivity are now over. More and more airlines are offering Internet access in flight and now BART has signed a 20-year deal to deliver Wi-Fi on its trains. BART expects to have all the routes and stations online by the end of 2011.

This sucks. As we become more and more connected we’re running out of excuses as to why we didn’t respond to someone’s message. The good ‘ole “I didn’t get your message” doesn’t work if you’ve got a BlackBerry or iPhone with its ubiquitous connectivity on email, Twitter, and Facebook.

Luckily technology is not perfect, and with the volume of information we’re all processing and filtering we’ve become overloaded. So no matter how ubiquitous wireless and communications technologies become, we can still rely on these two standard excuses:

Internet access wasn’t working

I’ve been so overloaded. I didn’t see your message. Sorry.

This news item is for the Spark Minute week of 2/2/09 which can be heard daily on Green 960 and 910 KNEW in San Francisco, CA.

VIDEO: Enterprise 2.0 Conf: Schmoozing on the eve of Enterprise 2.0

David Spark — Tue, 10 Jun 2008 02:28:52 +0000

After the “Evening in the cloud” session I pulled out my Nokia N82, loaded up the Qik streaming video service, and roamed the crowd asking what they thought of the session and the Enterprise 2.0 conference which will officially launch full sessions tomorrow.

Make sure you check out the summary of all coverage from the Enterprise 2.0 Conference 2008 in Boston.
In the twenty minute video you’ll meet the following people plus a lot more whose business cards I never collected:

Jeff Keltner of Google Apps
Steve Golab and his colleagues of FG Squared Interactive Marketing
Val Pishavazadeh of a start up he won’t talk about
Sam Eder of Small World Labs

Make sure you check out the summary of all coverage from the Enterprise 2.0 Conference 2008 in Boston.

This post is cross-posted from the Enterprise 2.0 Blog.

VIDEO: KQED-15th Anniversary of the Web on "This Week in Northern California"

David Spark — Sat, 03 May 2008 21:44:15 +0000

Last night I appeared on KQED’s This Week in Northern California talking about the birth of the Web. Fifteen years ago Tim Berners-Lee convinced his bosses at CERN to make his design of the Web open and free to everyone. It was a critical decision as we all realize now. Billions upon billions have been made as a result of that decision. San Francisco and Silicon Valley are the hubs of innovation in the Web. What can we expect in the next 15 years? Plan on the development of the semantic Web (Web 3.0), wireless ubiquity, and mobile content following us wherever we go.

VIDEO: Wi-Fi should be built onto every mobile device

David Spark — Thu, 20 Mar 2008 14:43:51 +0000

Ran into Matthew Snyder of ADObjects last night at a pre-party for the Under the Radar conference. We got to talking about Wi-Fi on handsets and how he thinks it’s going to be ubiquitous. I don’t think carriers are going to allow it. That was our debate.

Make sure you also catch my video with Amy Dalton of Topix. We talk about using Topix for really local news.

No related posts.

Nokia gives me three seconds of fame

David Spark — Thu, 20 Dec 2007 05:10:55 +0000

A couple months ago Nokia interviewed me at the radio station (Green 960) for a promo video about the release of its Internet tablet device, the Nokia N810. After interviewing me for an hour, they pulled out a choice three seconds (:50 to :53). Doesn’t sound like a lot, but I’m actually surprised the number of my friends that contacted me saying they had seen me in the video. Don’t blink, you may miss me.

8 Cool mobile technologies from CTIA 2007, San Francisco

David Spark — Wed, 24 Oct 2007 20:53:01 +0000

I got to spend a little time this week at the trade show floor of CTIA plus attended a few press events. Here’s a wrap up of some of the cool stuff I saw. Note, this is not exhaustive. I only got a chance to see a few items, but these are the items I thought were worth a mention.

Listen to the Spark Minute. John Scott and David Spark from Green 960 in San Francisco, CA talk about some cool mobile technology Spark saw at the CTIA conference (Run time: 6:06).

Download the MP3 (right-click, select Save Target As…)

Talkster – Free mobile-to-mobile phone calls ~~anywhere in the world~~ to 29 countries. All you have to do is listen to a ten second advertising message on each end and go through this funky handshaking process of the recipient receiving a pre-call and then hanging up. Then the calling party receives an SMS with a local number that you can then call and be connected to the international party. Their direct competitor is Jajah which is also very cool (See “How to make cheap to free phone calls”) but Jajah can’t connect everywhere in the world for free ~~like Talkster claims~~. Talkster also has a Facebook application. I think I’m the only one who doesn’t these days.

Whrrl (from Pelago) – What happens when you mash social networking, Yelp recommendations, mapping, and location based services delivered to your mobile phone? Whrrl is trying to be Yelp, but only recommendations from your trusted sphere of friends. A compatible service to this would be Trusted Opinion (see “Web 2.0 companies from Israel“) which shows an impressive visual mind map of your friends’ recommendations.

Solio – Portable hybrid solar power charger for mobile phones and other electronic devices. They’ve got three models, and this is their newest and smallest one but doesn’t hold nearly as much juice as the other two models.

iVisionMobile – Creators of iMessenger and iPayText. iMessenger offers an extremely simply tool for launching an SMS campaign using a shared shortcode. Truly takes only a few minutes to do. Great for small phone lists like real estate brokers that want to send listings to their 20 clients. First 100 messages are free and then after that costs range from 3 to 10 cents depending on quantity. The other product allows you to sell items using the mobile phone. Mobile ticketing is not new. What’s new is how cheap this is. No hugely expensive terminal to purchase. The item they have rents for $49/month.

Mobiledata Now – Traditionally this CTIA conference has been focused on mobile entertainment. And at every show I attend they keep telling us that mobile will be coming to the enterprise. Mobile has come to the enterprise but all everyone in a corporate setting uses their smartphones for is email. There are many players trying to port over corporate databases to the mobile device. Mobiledata Now is offering a simpler solution that operates through SMS. Instead of doing a massive configuration and having to outfit everyone in the office only with Treos, BlackBerry’s or Windows Mobile Devices you can set up a query system to your backend databases via SMS on any mobile device, smart or not. For example, you could check the stock on an item from a customer site using a simple SMS query.

Edioma – English as a second language (ESL) on your mobile phone. Series of games for you to learn English. Sounds simple and it is. But the ESL market is enormous and I’m amazed nobody is trying to capitalize on it. These guys are.

LG’s Venus and Voyager – There’s already been plenty of press on these two phones. I got a chance to see them. They’re both cool. The Voyager has the internal and external screen, mobile broadcast MediaFLO technology, and full keypad inside. The Venus has a series of soft buttons on the lower half of the screen.

Bluepulse – Mobile social networking. Whoop ‘de doo you say. And generally you’d be right, but this one actually hyper-simplifies the mobile social networking concept by putting all messaging (text, pictures, videos) into a single inbox and letting you reach friends via username, email address, and phone number. The reason they’ve been able to garner 3 million users in 10 months since launch is because to actually read a message from a friend you have to register with Bluepulse. What’s also impressive is this is fully a Web mobile application. No download necessary (anymore). The company claims it hasn’t found a Web-enabled phone that it can’t work on.

Google really knows how to mess up a wireless spectrum auction for incumbent Telcos

David Spark — Fri, 03 Aug 2007 05:18:25 +0000

The FCC’s auction of the 700 MHz spectrum was going to go like any other. The top Telco’s and cable companies would be the only ones that could afford to participate. They’d make the bids, snatch up the available airwaves, and if they needed the space they bought, they’d use it otherwise they’d sit on it for “future development.” Good luck shareholders.

Listen to the Spark Minute (John Scott and David Spark from Green 960 in San Francisco, CA) talk about the FCC’s 700 MHz auction and Google’s desire to participate (Run time: 7:08).

Download the MP3 (right-click, select Save Target As…)

The 700 MHz auction is turning out to be a different game. The spectrum being made available for auction (what’s being vacated as TV stations move over to digital, assuming we really do achieve the cut off date in 2009 or it gets pushed back even further as it has historically) could turn out to be a great channel to deliver wireless broadband. It’s a single low power channel that can go through walls and blankets the entire country.

Watch August 3rd, 2007 appearance on KQED’s “This Week in Northern California” with Belva Davis talking about the FCC 700 MHz auction and Google’s involvement.

To ruin the Telco and cable companies’ monopolized fun, Google has decided to participate in this reindeer game. Google publicly announced they’re willing to bid up to $4.6 billion for a piece of the 700 MHz spectrum on the condition that the FCC requires all participants adhere to four points of openness. Well, the FCC agreed to device and application portability. But they’re not allowing open wholesale access to the networks for resellers, or allowing for geolocation interconnection among the networks.

This story is jam packed with arm chair quarterbacks with their theories on whats going to happen with the Telco’s dominating the wireless airwaves and what Google’s intentions are. I have to extend a huge thank you to Scott Slater of the Personal Broadband Industry Association for giving me an enormous brain dump today at the AlwaysOn Conference at Stanford. He’s got a really grand view of the future of wireless broadband Internet communications that nobody else in the media seems to see.

So here are the issues and arguments at stake:

Deregulated ubiquitous wireless broadband Internet access - An open broadband Internet network that works across the country in which any IP-based device could intercommunicate with any other device. Such a network could deliver a level of innovation in devices and applications unfettered by a controlling network metering access, devices, and applications. Slater argues that the bid for the 700 MHz isn’t about a 3rd broadband network, but rather connecting everything to everything. Without a network intermediary, a truly open 700 MHz network could change the way businesses operate.
Google wants to sell ads on all devices - The more people that use the Web, the more applications that are made available (including Google’s), and the more sites that host Google ads ultimately results in Google making more money. So it would stand to reason that Google wants as many people using devices with Web browsers for people to access applications and content. CEO Eric Schmidt admits that Google ads on mobile phones will be more than twice as profitable as Web ads because of hyper-personalization and targeting.
$4.6 billion might not be enough – It’s what Google has offered and it’s the public reserve price for the available spectrum for device and application portability. The Telco’s and cable companies have more money than Google.
Google is going to prove its theory regardless – Whether Google purchases some of the 700 MHz spectrum or they fail, they’re going to create a ubiquitous wireless broadband network. They’re already doing it in Mountain View, CA where they’ve deployed a municipal Wi-Fi network. They have plans to do it in San Francisco, and maybe they’ll do it in a broader form with the Sprint WiMax network for which they’ve already created a partnership to host their applications and data. Whether they win in the 700 MHz spectrum or not, they’re going to create zones of blanketed connectivity that operate fully on IP proving their theory of ubiquitous wireless broadband communications across devices. It will be a model that should scare Telco’s and get the shareholders complaining.
It’s more than phone portability – The FCC Chairman and commissioners are patting themselves on the back for myopic accomplishments. Yes, the advent of phone number portability was great. And now the new rule for the 700 MHz auction requires handset and application portability. That’s great too. But that’s what Europe already has. Everyone buys unlocked phones and can switch to any network they want. So we’re innovating at the speed of Europe? Commissioner Deborah Taylor Tate said, “My hope is that we have created an incubator for the next killer app, the next platform or the next cool device.” The next killer app, platform, or device would happen no matter what you do. Look at the trades. Walk a trade show floor. It’s happening now in spite of the domination controls you’ve given Telco’s.
Google’s ad network isn’t vertically integrated with wireless devices, yet – Google wants its Web applications to be used anywhere. Its competitors are better positioned to distribute ads on mobile devices on mobile networks. Microsoft has Windows Mobile and its newly purchased ad network aQuantive, plus a few others: Screentonic and AdECN. Yahoo! and AT&T Wireless have already been distributing portals and applications over mobile devices. Google is late to the game, but it’s furiously developing mobile applications (some available now like Google Maps and Gmail) and the Google phone which they hope could work on multiple networks and not be locked to just one network like the iPhone (that won’t last for long).
The incumbents create hooks to keep customers using their technology – Everything that Qualcomm and the carriers create: the chip, the network, the phones, the applications, and the yearly contracts, are all designed to hold onto the customer. You have to pick our phones and chips to use our technology and we decide what applications you can use.
Wireless carriers aren’t offering broadband – For broadband Internet access, Slater says there’s a duopoly, cable and DSL, and they’re deregulated. He doesn’t see wireless as a competitor in the broadband Internet space. Because he defines broadband as consistent 1 Mbps. When I did work for Sprint, the company had no problem advertising that they had “broadband” access if the network hit 100 Kbps.
Telco’s have to continue to increase value of their proprietary technology - The Telco’s have told their shareholders that they’re going to buy this spectrum. They’ll need to purchase it and do something with it so as to create continued value. Slater things they’ll just buy it and warehouse it.
Google domination or altruism? - Is Google being altruistic or are they trying to control the pipes and all the ad content that goes on it? Or is it something in between? Lance Ulanoff of PC Magazine argues that such a purchase by Google would put it in a dominant position. Even if they do win the auction, it’s going to be very difficult to get its network up to speed. He points to Qualcomm’s difficulties trying to get its mobile broadcast video network, MediaFLO, to work on the UHF spectrum. Slater agrees and doesn’t think the network will be built out overnight, but he believes Google will give away the spectrum or partner with companies that can build out the network. They don’t want to build or run a network themselves. Google will simply make more money if there are more people, more devices, and more applications on the network.

Some are arguing that the 700 MHz spectrum may be the last chance for unfettered deregulated wireless broadband. This could be why Google has entered the race. Given America’s history of existing networks, the U.S. isn’t up to speed versus the rest of the world in broadband deployment and coverage. In fact, it ranks 17th in the world. The reason the wireless fight is so important is because access has shifted to individuals, and it’s not about connecting buildings to the last mile. Slater argues that if the 700 MHz spectrum was controlled by local TV stations, it should return to the localities and not be gobbled up to the few national incumbents.

UPDATE August 26th, 2008: Google is going to the masses now in hopes that the community of users will see the value of free airwaves rather than telephone companies bidding and holding onto spectrum.

Microsoft's surface computer

David Spark — Thu, 31 May 2007 23:27:50 +0000

Microsoft just announced this new cool coffee table computer. A multi-touch device that actually interacts with devices you put on top of it. So for example, drop a digital camera and it will take bring photos out onto the table. Want to put the photo on your mobile phone? Just drop your phone on the table and drag the image of the photo to where your mobile phone is on the table and it’s automatically uploaded. It’s pretty darn cool. Check out this video from Popular Mechanics.

Disclosure: Microsoft is a client of Spark Media Solutions.