Live Texturing of Augmented Reality Characters from Colored Drawings

From the abstract – Coloring books capture the imagination of children and provide them with one of their earliest opportunities for creative expression. However, given the proliferation and popularity of digital devices, real-world activities like coloring can seem unexciting, and children become less engaged in them. Augmented reality holds unique potential to impact this situation by providing a bridge between real-world activities and digital enhancements. In this paper, we present an augmented reality coloring book App in which children color characters in a printed coloring book and inspect their work using a mobile device. The drawing is detected and tracked, and the video stream is augmented with an animated 3-D version of the character that is textured according to the child’s coloring. This is possible thanks to several novel technical contributions. We present a texturing process that applies the captured texture from a 2-D colored drawing to both the visible and occluded regions of a 3-D character in real time. We develop a deformable surface tracking method designed for colored drawings that uses a new outlier rejection algorithm for real-time tracking and surface deformation recovery. We present a content creation pipeline to efficiently create the 2-D and 3-D content. And, finally, we validate our work with two user studies that examine the quality of our texturing algorithm and the overall App experience.

Read the full paper.

The Fjord Fido project

Diabetes is a life threatening condition affecting almost 400 million people around the globe today.

In the past years, innovation within digital and the health and fitness industry has provided tremendous opportunities to collect and analyse data affecting our bodies. Opportunities that could create great possibilities for the management of diabetes.

Jonas Höglund, service design director at Fjord Stockholm and father of a son with diabetes, saw the potential. At a global Accenture Innovation event he presented an idea of diabetes care through digital and big data. A speech that became the starting point for the Fjord Fido project.

The Interface Theory of Perception

Perception is a product of evolution. Our perceptual systems, like our limbs and livers, have been shaped by natural selection. The effects of selection on perception can be studied using evolutionary games and genetic algorithms. To this end, we define and classify perceptual strategies and allow them to compete in evolutionary games in a variety of worlds with a variety of fitness functions. We find that veridical perceptions—strategies tuned to the true structure of the world—are routinely dominated by nonveridical strategies tuned to fitness. Veridical perceptions escape extinction only if fitness varies monotonically with truth. Thus, a perceptual strategy favored by selection is best thought of not as a window on truth but as akin to a windows interface of a PC. Just as the color and shape of an icon for a text file do not entail that the text file itself has a color or shape, so also our perceptions of space-time and objects do not entail (by the Invention of Space-Time Theorem) that objective reality has the structure of space-time and objects. An interface serves to guide useful actions, not to resemble truth. Indeed, an interface hides the truth; for someone editing a paper or photo, seeing transistors and firmware is an irrelevant hindrance. For the perceptions of H. sapiens, space-time is the desktop and physical objects are the icons. Our perceptions of space-time and objects have been shaped by natural selection to hide the truth and guide adaptive behaviors. Perception is an adaptive interface.

Download the full paper.

Zeitmaschine: the uncertainty of the future onto the past

Zeitmaschine is an interactive video installation that projects the uncertainty of the future onto the past. It deals with the fluidity of time and the way we construct our world through what we see. It makes the recent past explorable in a fluid and playful way. But although the perceivable time continuum is coherent, events might not be. The machine adapts to the space it is set up in and behaves much like a human brain: Memories are selective, sometimes altered, invented or forgotten.

The installation had a first public test run at the 48h Neukoelln Arts Festival 2014 at Ke//er L/A/B in Berlin. Although only planned to be on show for three days, thanks to its many fans and Ke//er, the installation did run for almost a month.

Retrieved on Cycling 74 website.

Nevermind: a video game powered by your biometrics

Nevermind is a biofeedback-enhanced adventure horror game that takes you into the dark and twisted world of the subconscious.

As you explore surreal labyrinths and solve the puzzles of the mind, a biofeedback sensor will monitor how scared or stressed you become moment-to-moment. If you let your fears get the best of you, the game will become harder. If you’re able to calm yourself in the face of terror, the game will be more forgiving.

Nevermind strives to create a haunting gameplay experience that also teaches you how to be more aware of your internal responses to stressful situations. If you can learn to control your anxiety within the disturbing realm of Nevermind, just imagine what you can do when it comes to those inevitable stressful moments in the real world.

See the project page.

Camera Restricta. A disobedient tool for taking unique photographs

Camera Restricta by Philipp Schmitt is a brilliant example of speculative design. According to the designer, “algorithms are already looking through the viewfinder alongside with you: they adjust settings, scan faces and take a photo when you smile. What if your grin wasn’t the only thing they cared about?

Camera Restricta is a speculative design of a new kind of camera. It locates itself via GPS and searches online for photos that have been geotagged nearby. If the camera decides that too many photos have been taken at your location, it retracts the shutter and blocks the viewfinder. You can’t take any more pictures here”.

Read more about the project.

 

A conversation-based approach to Interaction Design

In this articile, Hugh Dubberly and Paul Pangaro discuss the potential of a new approach to interaction design based on a conversation model. Starting from Claude Shannon’ Model of Communication, they shed the light on

“a fundamental limit of nearly all human-to-computer interaction: Our input gestures can only activate an existing interface command (select a message) from the preprogrammed set. While we can automate sequences of existing commands, we can’t ask for something novel. If our software application does anything novel, we file a bug report!”

Continue reading…

Syllabus: Interaction & Service Design Concepts Principles, Perspectives, and Practices

Prof. Molly Wright Steenson (Carnegie Mellon School of Design) has published the syllabus for her Interaction & Service Design class (Fall 2015).  According to the introduction:

What is interaction, what is design, where did these notions come from, and where are they going? Our seminar explores principles, practices, and practices that undergird interaction and service design and beyond. We will explore the underlying principles of design, examine themes from a variety of perspectives, and consider the effects of both on different practices. Through this grounding, you will return to questions of what kind of designer you are and wish to be, what you believe in, and how that will extend to your research and practice.

Continue reading…