Live Texturing of Augmented Reality Characters from Colored Drawings

From the abstract – Coloring books capture the imagination of children and provide them with one of their earliest opportunities for creative expression. However, given the proliferation and popularity of digital devices, real-world activities like coloring can seem unexciting, and children become less engaged in them. Augmented reality holds unique potential to impact this situation by providing a bridge between real-world activities and digital enhancements. In this paper, we present an augmented reality coloring book App in which children color characters in a printed coloring book and inspect their work using a mobile device. The drawing is detected and tracked, and the video stream is augmented with an animated 3-D version of the character that is textured according to the child’s coloring. This is possible thanks to several novel technical contributions. We present a texturing process that applies the captured texture from a 2-D colored drawing to both the visible and occluded regions of a 3-D character in real time. We develop a deformable surface tracking method designed for colored drawings that uses a new outlier rejection algorithm for real-time tracking and surface deformation recovery. We present a content creation pipeline to efficiently create the 2-D and 3-D content. And, finally, we validate our work with two user studies that examine the quality of our texturing algorithm and the overall App experience.

Read the full paper.

The Fjord Fido project

Diabetes is a life threatening condition affecting almost 400 million people around the globe today.

In the past years, innovation within digital and the health and fitness industry has provided tremendous opportunities to collect and analyse data affecting our bodies. Opportunities that could create great possibilities for the management of diabetes.

Jonas Höglund, service design director at Fjord Stockholm and father of a son with diabetes, saw the potential. At a global Accenture Innovation event he presented an idea of diabetes care through digital and big data. A speech that became the starting point for the Fjord Fido project.

The Interface Theory of Perception

The Interface Theory of Perception

Perception is a product of evolution. Our perceptual systems, like our limbs and livers, have been shaped by natural selection. The effects of selection on perception can be studied using evolutionary games and genetic algorithms. To this end, we define and classify perceptual strategies and allow them to compete in evolutionary games in a variety of worlds with a variety of fitness functions. We find that veridical perceptions—strategies tuned to the true structure of the world—are routinely dominated by nonveridical strategies tuned to fitness. Veridical perceptions escape extinction only if fitness varies monotonically with truth. Thus, a perceptual strategy favored by selection is best thought of not as a window on truth but as akin to a windows interface of a PC. Just as the color and shape of an icon for a text file do not entail that the text file itself has a color or shape, so also our perceptions of space-time and objects do not entail (by the Invention of Space-Time Theorem) that objective reality has the structure of space-time and objects. An interface serves to guide useful actions, not to resemble truth. Indeed, an interface hides the truth; for someone editing a paper or photo, seeing transistors and firmware is an irrelevant hindrance. For the perceptions of H. sapiens, space-time is the desktop and physical objects are the icons. Our perceptions of space-time and objects have been shaped by natural selection to hide the truth and guide adaptive behaviors. Perception is an adaptive interface.

Download the full paper.

Zeitmaschine: the uncertainty of the future onto the past

Zeitmaschine is an interactive video installation that projects the uncertainty of the future onto the past. It deals with the fluidity of time and the way we construct our world through what we see. It makes the recent past explorable in a fluid and playful way. But although the perceivable time continuum is coherent, events might not be. The machine adapts to the space it is set up in and behaves much like a human brain: Memories are selective, sometimes altered, invented or forgotten.

The installation had a first public test run at the 48h Neukoelln Arts Festival 2014 at Ke//er L/A/B in Berlin. Although only planned to be on show for three days, thanks to its many fans and Ke//er, the installation did run for almost a month.

Retrieved on Cycling 74 website.

Nevermind: a video game powered by your biometrics

Nevermind is a biofeedback-enhanced adventure horror game that takes you into the dark and twisted world of the subconscious.

As you explore surreal labyrinths and solve the puzzles of the mind, a biofeedback sensor will monitor how scared or stressed you become moment-to-moment. If you let your fears get the best of you, the game will become harder. If you’re able to calm yourself in the face of terror, the game will be more forgiving.

Nevermind strives to create a haunting gameplay experience that also teaches you how to be more aware of your internal responses to stressful situations. If you can learn to control your anxiety within the disturbing realm of Nevermind, just imagine what you can do when it comes to those inevitable stressful moments in the real world.

See the project page.

Camera Restricta. A disobedient tool for taking unique photographs

Camera Restricta by Philipp Schmitt is a brilliant example of speculative design. According to the designer, “algorithms are already looking through the viewfinder alongside with you: they adjust settings, scan faces and take a photo when you smile. What if your grin wasn’t the only thing they cared about?

Camera Restricta is a speculative design of a new kind of camera. It locates itself via GPS and searches online for photos that have been geotagged nearby. If the camera decides that too many photos have been taken at your location, it retracts the shutter and blocks the viewfinder. You can’t take any more pictures here”.

Read more about the project.


Designing the experience of Elite: Dangerous

Designing the experience of Elite: Dangerous

In this interview, Louise McLennan – Senior UX Designer at Frontier Developments – discuss the design challenges of a video game such as Elite: Dangerous.

About the game – Frontier Developments released Elite: Dangerous in December 2014, after a successful Kickstarter campaign. For this game, they further developed the concepts of trading and combat that were part of the original version of Elite, which Acornsoft released in 1984. However, Elite: Dangerous offers both a massively multiplayer, persistent world of online play, as well as single-player gaming. The game’s user experience evolves many of the same designs and concepts that the original game introduced, and many of its current players are part of the older demographic who played the original game.

Read the interview on UXMatters.

A conversation-based approach to Interaction Design

A conversation-based approach to Interaction Design

In this articile, Hugh Dubberly and Paul Pangaro discuss the potential of a new approach to interaction design based on a conversation model. Starting from Claude Shannon’ Model of Communication, they shed the light on

“a fundamental limit of nearly all human-to-computer interaction: Our input gestures can only activate an existing interface command (select a message) from the preprogrammed set. While we can automate sequences of existing commands, we can’t ask for something novel. If our software application does anything novel, we file a bug report!”

Continue reading…