[Paper] Human-level control through deep reinforcement learning

Google Deepmind‘s scientists have built a software that can play video games as a human being, or even better. By exploiting the theory of reinforcement learning, Google’s software is able to improve its performance after hours of play. As stated in the abstract:

The theory of reinforcement learning provides a normative account, deeply rooted in psychological and neuroscientific  perspectives on animal behaviour, of how agents may optimize their control of an environment. To use reinforcement learning successfully in situations approaching real-world complexity, however, agents are confronted with a difficult task: they must derive efficient representations of the environment from high-dimensional sensory inputs, and use these to generalize past experience to new situations. Remarkably, humans and other animals seem to solve this problem through a harmonious combination of reinforcement learning and hierarchical sensory processing systems, the former evidenced by a wealth of neural data revealing notable parallels between the phasic signals emitted by dopaminergic neurons and temporal difference reinforcement learning algorithms. While reinforcement learning agents have achieved some successes in a variety of domains, their applicability has previously been limited to domains in which useful features can be handcrafted, or to domains with fully observed, low-dimensional state spaces. Here we use recent advances in training deep neural networks to develop a novel artificial agent, termed a deep Q-network, that can learn successful policies directly from high-dimensional sensory inputs using end-to-end reinforcement learning. We tested this agent on the challenging domain of classic Atari 2600 games. We demonstrate that the deep Q-network agent, receiving only the pixels and the game score as inputs, was able to surpass the performance of all previous algorithms and achieve a level comparable to that of a professional human games tester across a set of 49 games, using the same algorithm, network architecture and hyperparameters. This work bridges the divide between high-dimensional sensory inputs and actions, resulting in the first artificial agent that is capable of learning to excel at a diverse array of challenging tasks.

Read the full article on Nature.

AI + IoT: should we be scared of the conscious web?

AI + IoT: should we be scared of the conscious web?

Stephen Hawking, Bill Gates and Elon Musk have already expressed more than a simple concern about AI and the threat it may represent for the future of human race. Should we be scared?  In an article published on The Guardian, Stephen Balkam writes:

What are these wise souls afraid of? AI is broadly described as the ability of computer systems to ape or mimic human intelligent behavior. This could be anything from recognizing speech, to visual perception, making decisions and translating languages. Examples run from Deep Blue who beat chess champion Garry Kasparov to supercomputer Watson who outguessed the world’s best Jeopardy player. Fictionally, we have Her, Spike Jonze’s movie that depicts the protagonist, played by Joaquin Phoenix, falling in love with his operating system, seductively voiced by Scarlet Johansson. And coming soon, Chappie stars a stolen police robot who is reprogrammed to make conscious choices and to feel emotions. An important component of AI, and a key element in the fears it engenders, is the ability of machines to take action on their own without human intervention. This could take the form of a computer reprogramming itself in the face of an obstacle or restriction. In other words, to think for itself and to take action accordingly.

[…]

Running parallel to the extraordinary advances in the field of AI is the even bigger development of what is loosely called, the internet of things (IoT). This can be broadly described as the emergence of countless objects, animals and even people with uniquely identifiable, embedded devices that are wirelessly connected to the internet. These ‘nodes’ can send or receive information without the need for human intervention. There are estimates that there will be 50 billion connected devices by 2020. Current examples of these smart devices include Nest thermostats, wifi-enabled washing machines and the increasingly connected cars with their built-in sensors that can avoid accidents and even park for you. The US Federal Trade Commission is sufficiently concerned about the security and privacy implications of the Internet of Things, and has conducted a public workshop and released a report urging companies to adopt best practices and “bake in” procedures to minimise data collection and to ensure consumer trust in the new networked environment.

[…]

So what happens when these millions of embedded devices connect to artificially intelligent machines? What does AI + IoT = ? Will it mean the end of civilisation as we know it? Will our self-programming computers send out hostile orders to the chips we’ve added to our everyday objects? Or is this just another disruptive moment, similar to the harnessing of steam or the splitting of the atom? An important step in our own evolution as a species, but nothing to be too concerned about?

Read the full article.