AI for Robotics

Post by 
February 16, 2021
No additional tags.

uch has been written about the emergence of robotics, chief among which has been their impact on society, the economy, and the future of work. Robots are already being used for many purposes, including serving as remote-controlled agents in dangerous environments, manufacturing processes, and delivery tasks. 

Advancements in Artificial Intelligence (AI) allow machines to function increasingly independently of this remote control. The AI stack can be broken down into two stages: a sensing stage to extract information about the robot’s environment and internal state, and a decision-making stage which uses this information to determine the robot’s next actions. For more information about the sensing stage, please check out our Computer Vision Selects which delves into using imagery to help computers understand the world around us. For this decision-making layer, Reinforcement Learning has been gaining strong momentum as a method to teach robots to solve problems in an unsupervised manner. 

In this week’s ACM Selects, we provide a snapshot of current advancements in AI that impact robotics. Our selections briefly cover relevant articles regarding Reinforcement Learning and user experience design for robotics, as well as discussions on applications being used today. 

While in this article we implicitly define a ‘robot’ as a single or few actuated devices able to sense and interact with its surroundings, the rise of the internet of things and advances of multi-agent systems may soon expand our understanding of “a robot” to a system potentially encompassing vast numbers of sensors and actuated parts. We will discuss one such IOT application, creating city-sized robots, in an upcoming ACM Selects on Smart Cities.

We invite you to consider participating in ACM’s activities on these topics, be it through our professional community,​ global policy​ activities​, ongoing work in ​professional ethics​, and/or through our ​chapters, ​SIGs​, ​local meetups and/or ​conferences​.

We value your feedback and look forward to your guidance on how we can continue to improve ACM Selects together. Your suggestions and opinions on how we can do better are welcome via email through

Reinforcement Learning

"Reinforcement learning is a way of not needing labels, or labeling automatically by who’s winning or losing—by the rewards, so you can learn to play the Go game by playing the moves and winning or losing, and no one has to tell you if that was a good move or a bad move because you can figure it out for yourself; it led to a win, so it was a good move.”
--Richard Sutton, Distinguished Research Scientist at DeepMind

Reinforcement renaissance 

First published in Communications of the ACM, Vol. 59, No. 8, July 2016.

The article describes the emerging interest in reinforcement learning (RL) in combination with deep learning, and its application to games and robotics, especially after Deepmind's success on complex games such as Go. It has unique perspectives from different experts in the field and provides a great overview of how they think about the problems, challenges and limitations, and the future work involved to reach human-like capabilities.
Learn more ]

Reinforcement learning in Robotics 

First published in Medium, July 2016.

Víctor Mayoral Vilches, CTO at Alias Robotics, describes that RL works for complex problems where there is no easily programmable solution e.g. game playing and robotic control. He mentions via examples how similar a robot learning via RL and discovering optimal behavior through  trial-and-error is similar to humans' way of learning. Later, challenges about "curse of dimensionality" are being discussed, and how we can use the effective representations, prior knowledge and approximations to make robot RL yield good results in real world scenarios.

Learn more ]

“Deep learning is the greatest thing since sliced bread, but it quickly becomes limited by the data. If we can use reinforcement learning to automatically generate data, even if the data is more weakly labeled than having humans go in and label everything, there can be much more of it because we can generate it automatically, so these two together really fit well."
--Richard Sutton, Distinguished Research Scientist at DeepMind

The Ingredients of Real World Robotic Reinforcement Learning

First published in Berkeley Artificial Intelligence Research, April 2020.

Abhishek Gupta, a PhD student in EECS at UC Berkeley, shows a way of training robots directly with RL in the real world as opposed to the typical simulation to real-world transfer paradigm. He describes ways of addressing constraints of the RL in the real world  such as "no reset mechanism", "non-availability of a low dimensional Markovian state", "no reward function", and demonstrates successful results on simplistic robotic tasks.
[ Learn more ]

An Introduction to Deep Reinforcement Learning

First published by Now Publishers, 2019. Available on arxiv.

A great resource for people who are further interested to learn more about Deep reinforcement learning.
Learn more ]

Applications and Human-Robot Interactions

Healthcare robotics 

First published in Communications of the ACM, Vol. 60, No. 11, October 2017.

A fantastic review article on the usage of Robotics in Healthcare by UCSD Professor Laurel D. Riek. The article provides great insights into how healthcare robotics could help the over 20% of the world’s population facing physical, cognitive, or sensory impairments,  and fill the gaps in an overburdened healthcare system.

The structure of this article is also a framework for any product design workflow: 1. First, build a profile for the potential beneficiaries of the technology, settings in which it will be used, and tasks it should perform. 2. Create a taxonomy of different types of solutions (in this case healthcare robots), and 3. List the challenges of the field and how to mitigate them. The article is filled with lessons in user-centric design to create technology that is actually beneficial and useful and is a highly recommended read.
Learn more ]

A Roadmap for US Robotics: 2020 Edition

First published as the US National Robotics Roadmap by the Computing Community Consortium, September 2020.

A comprehensive report on the state of Robotics applications in the United States from its leading research universities. A thorough coverage of societal drivers, research efforts, technology contexts, and the legal and societal contexts of the field.
Learn more ]

A Primer for Conducting Experiments in Human–Robot Interaction 

First published in ACM Transactions on Human-Robot Interaction, October 2020.

A comprehensive tutorial on conducting studies with human participants, from a Human-Robot Interaction research perspective. As we start to work with, drive alongside, and be cared for by robots, it is increasingly important to understand Human-Robot interactions. Example studies in improving human-robot collaboration include understanding proper timings for interactions, how to create interactions for multiple people with a robot controlling multiple devices, and how pedestrians interact with autonomous vehicles. Through this tutorial, we hope that young practitioners will gain 1. A critical thinking for determining the methodological rigor, reproducibility, and impact of the studies they read about in academic papers and popular media, and 2. An authoritative and comprehensive toolset for conducting their own research with human participants.
Learn more ]

THere's More

Recommended Selects

See all selects
Getting Started Series

Getting Started with Internet of Things: IoT Applications

This Selects finalizes with an example application domain of Industrial Internet ofThings (IIoT), and a source to delve into state-of-the-art IoT research trends.
Getting Started Series

Getting Started with Internet of Things: Computing and Communication

The selection includes easy to read articles describing and motivating the IoT, and later deep dives into the major aspects of IoT such as communication protocols, edge-to-cloud continuum, AI and data analytics, and security/privacy.
Computing in Practice Series

Trustworthy AI in Healthcare #02

AI needs to be trustworthy. Trustworthiness means that healthcare organizations, doctors, and patients should be able to rely on the AI solution as being lawful, ethical, and robust.

Help guide ACM Selects!

Let us know how we can improve your ACM Selects experiences, what topics you would like us to cover in the future, whether you would like to contribute and/or subscribe to our newsletter by emailing

We never share your info. View our Privacy Policy
Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.
continue learning with the acm digital library!
explore ACM DL