AI in Industrial Automation Summit schedule

The Current State of Industrial Robotics

Results in the fields of deep and reinforcement learning have ignited immense interest in the possible applications for robotics. In particular, research into learning based approaches to control as well as scene understanding techniques hint at the possibility of new applications and new ways of programming robots. However, most research has been geared toward toy problems, and there continues to be a gap between the most advanced papers and the reality of deployed industrial robots. I will discuss the most recent advances in deep and reinforcement learning for robotics, the current state of industrial robotics, and how Osaro is working to bridge the gap.

Hariharan Ananthanarayanan is a Robotics Engineer and an enthusiast who currently works as a Motion planning Engineer at Osaro, a San Francisco based machine learning company building products powered by Deep Reinforcement Learning. Hariharan has close to ten years of experience in the automation and material handling industry since when he started working for an Automated Guided Vehicle manufacturer in 2006. Hariharan’s expertise lies in the Kinematics of robotic arms particularly in motion planning and controlling of robotic manipulators in industrial environment. He also possesses excellent product development skills which he leveraged in building the first functioning prototype of Obi, an independent feeding device for people with disabilities. His critical contribution to the product and active participation in the company, enabled the founders of Desin LLC to launch their product Obi, successfully in 2016. Hariharan currently focusses on integrating machine learning techniques to control of robotic arms for more intuitive and reliable performance. Hariharan strongly believes that the advancements in machine learning can be leveraged to achieve human like capabilities in manipulation. Hariharan possess a Bachelors Degree in Mechanical Engineering from India (2003), a Master’s in Mechanical Engineering with focus on robotics from University of Tennessee (2005) and a Ph.D in motion planning of industrial manipulators from University of Dayton (2015).

Machine Learning in the Wild: Scaling Automation Applications from Prototype to Plant Floor

Prototyping machine learning solutions off-line can now be done quickly and efficiently. But what considerations must be taken to ensure that a quickly created prototype can be effectively deployed on the plant floor, in production? When should simpler modeling techniques be favored over more powerful, but also more complex modeling techniques? What should be considered when choosing to deploy a solution on existing PLCs, versus leveraging a separate processing unit? This talk will walk through these considerations in the context of PepsiCo’s potato chip manufacturing process, and answer how some of these questions and considerations have been handled at PepsiCo. In addition, the presentation will discuss how traditionally non-technology companies face a different set of challenges when starting to build machine learning applications at scale.

Shahmeer Mirza is a Senior Research and Development Engineer at PepsiCo. He is focused on leveraging machine learning to develop next generation automation for Industry 4.0 applications. During his time with PepsiCo he has developed robotics, computer vision, sensor, and soft sensor applications for automation using machine learning. Shahmeer has also worked with the strategy and marketing departments to develop predictive AI applications. He holds a B.S. in Chemical and Biomolecular Engineering from Georgia Tech and is currently pursuing his M.S. in Computer Science at Georgia Tech, as well. In his free time he enjoys traveling, scuba diving, playing guitar, and writing.

Machine Learning through analogies formalized with tools from Category Theory

Industrial applications relying on Machine Learning (ML) to control robotic systems need to learn with few trials. The number of experiments in this context is limited by either the available time or budget. Such a goal can be attained by using knowledge transfer through analogies, both of which can be mathematically formalized with tools from Category Theory. This leads to a new ML approach that transposes accumulated knowledge to new configurations, while still relying on existing ML tools. Illustrations of this innovative solution are presented on a cyber-physical system, a slot-car game. This method also proves to be versatile enough to be used on simulated systems as well, as it is demonstrated on the Atari 2600 games.

Lionel Cordesses holds a Ph.D. in Computer Vision and Control. After developing autonomous vehicles in the late 90s, he joined the Renault group in France where he created and led the Engine Control team. Then, after six years as a Project Manager in the field of Electric Vehicles, he crossed the Atlantic to build and lead Renault’s Artificial Intelligence team in Sunnyvale, CA. Lionel is also an Independent Expert in Signal Processing. He loves to hear "we know it cannot be done", as this presents an exciting challenge to find a unique solution and ultimately create new patents (34 and counting)

Combining Semantic and Geometric Scene Understanding: From Robot Manipulation to Planetary Science

Renaud Detry is a research scientist at NASA JPL, and a visiting researcher at ULiege/Belgium and KTH/Stockholm. Detry earned a Master's degree in computer engineering and a Ph.D. in robot learning from ULiege in 2006 and 2010. Shortly thereafter he earned two Starting Grants from the Swedish and Belgian national research institutes. He served as a postdoc at KTH and ULiege between 2011 and 2015, before joining the Robotics and Mobility Section at JPL in 2016. His research interests are in perception for manipulation, robot grasping, computer vision and machine learning. At JPL, Detry is involved in the Mars Sample Return technology development program, and he conducts research in robot autonomy for mobility and opportunistic science on Mars, Europa and Enceladus.

How Machine Vision Fails: Adversarial Attacks and Other Problems

In recent years, we’ve seen remarkable computer vision successes using neural networks. OCR allows automated mail routing. Facial recognition identifies suspects on security video and our friends on social media. Scene segmentation and object detection and classification are used in autonomous vehicle navigation, medical diagnosis and robotic manufacturing.

But we’ve also seen notable failures. A black person is misclassified as a gorilla in Google photos, recalling centuries of racial slurs. In one fatal crash, a Tesla can’t see cross traffic. In another, an Uber misclassifies a pedestrian as an unknown object, then as a vehicle and then as a bicycle, with varying expectations of future travel path. Snow on the road might be mistaken for lane markings. Worn markings and asphalt smoothly transitioning to dirt might not be seen at all. A stop sign can be changed to a speed limit sign with a few pieces of tape or a bit of graffiti. A person can make her face disappear or turn a toy turtle into a rifle. A single pixel change can make an image unrecognizable.

How fragile are the machine vision systems you rely on, what are the ways they will fail and what can you do about it?

An AI policy researcher, lawyer and computer vision scientist, Gretchen Greene advises government leaders on AI strategy, use and policy and works with Cambridge startups on everything from autonomous vehicles to QR codes in wearables. Greene has been a U.S. Departments of Defense, Energy and Homeland Security mathematician, has published in machine learning, science and policy journals and has been interviewed by the Economist, Forbes China and the BBC. An affiliate researcher at MIT’s Media Lab, Greene has a CPhil and MS in math from UCLA and a JD from Yale.

One robot doesn't fit all; lessons from the field

As sophisticated and cool as today's army of autonomous robots and self-driving cars seem, the reality is that they are a lot more akin to managing an army of 10yr olds spread out across a city. They are almost always backed by a team of human robot operators remotely watching over them while trying to teach them right from wrong. How do you manage all of that? How do you choose when to intervene and take action or to stay back and let the robots learn on their own? In a lot of ways, the robotics industry today is similar to where PCs were in the 80s- they're specific purpose and the technology is just maturing now, but at the same time, robots and AI are popping up all around you in your daily lives. Yet being so seamless is not an accident; theres actually a lot that goes into designing them to set the right expectations for being accepted by people naturally while still being useful for business purposes. We will cover some of the learnings, stories and (sometimes funny) adaptations we've had to make in order to make robots fit in frictionlessly with you.

Thavidu Ranatunga is CTO of Fellow Robots. Fellow has had robots deployed live in front of customers for almost 4 years in more than 7 different retailers across 2 countries. The robots help customers find products in stores and help store associates with inventory management in the store. Formerly with Microsoft, Groupon and IBM, Thavidu now leads the technology side of Fellow Robots, revolutionizing the future of the Retail experience. He has been programming since he was 8 years old and has a wealth of experience in a wide variety of software development areas, encompassing everything from web technologies to embedded systems. However, his strongest passion is for all things Machine Learning, Computer Vision and Robotics. Thavidu graduated from the Australian National University with a Bachelor of Software Engineering majoring in Mechatronics. He also ranked top 10 in the Australian Informatics Olympiad.

Performance Metrics for AI in Manufacturing HRI

Recent years have witnessed the birth of a new era in industrial robotics in which collaborative systems, designed to work safety beside the human workforce, are integrated into historically manual processes. Such technologies represent a relatively low-risk gateway solution to transitioning facilities and operations to a state of partial automation, but retain many of the characteristics of their non-collaborative predecessors. Specifically, collaborative robots largely remain difficult to program, integrate, and maintain. Although the skills of the labor force are expected to increase in the coming years, the collaborative capabilities of next-generation industrial robots must evolve to bridge the technology gap, leading to more effective human-robot teaming in manufacturing application. It is expected advances in artificial intelligence will play a significant role in aiding this transition, but prior to adoption such technologies must be validated and hardened for the industrial environment. In this talk, Dr. Jeremy Marvel from the U.S. National Institute of Standards and Technology (NIST) will discuss work in assessing and assuring artificial intelligence technology for human-robot interaction through measurement science. Dr. Marvel will present an overview of the current technology landscape, and provide some initial metrology results from NIST’s ongoing Performance of Collaborative Robot Systems project. Focal topics include artificial intelligence for collaborative robot safety, robotic system integration, and situation awareness for human-robot teaming.

Jeremy A. Marvel is a research scientist and project leader at the U.S. National Institute of Standards and Technology (NIST), a non-regulatory branch of the U.S. Department of Commerce. Prior to NIST, Dr. Marvel was a research scientist at the Institute for Research in Engineering and Applied Physics at the University of Maryland, College Park, MD. He joined the Intelligent Systems Division at NIST in 2012, and has over thirteen years of robotics research experience in both industry and government. His research expertise include intelligent and adaptive solutions for robot applications, with particular attention paid to human-robot and robot-robot collaborations, multirobot coordination, industrial robot safety, machine learning, perception, and automated parameter optimization. Dr. Marvel currently leads a team of scientists and engineers in metrology efforts at NIST toward collaborative robot performance, and developing tools to enable small and medium-sized enterprises to effectively deploy robot solutions. Dr. Marvel received the Bachelor’s degree in computer science from Boston University in 2003, the Master’s degree in computer science from Brandeis University in 2005, and the Ph.D. degree in computer engineering from Case Western Reserve University in 2010.

Better Earth-moving Machine Control through Artificial Intelligence

Virtual product development iterates new designs in simulation before any prototypes are built, thus reducing cost until the design is optimal. When these simulations are run by a computer, the machines must be operated in a human-like way so that the results can be trusted. This is the function of the virtual “operator model”. Historically, these models have been difficult to create since the simulation environment is very dynamic – simple logical rules fail to achieve the right behavior and advanced operator models require advanced control theory expertise.

Using reinforcement learning, a program can learn to operate an earth-moving machine by itself. Over time, the agent learns to improve its behavior and converge on actions that are optimal. In this way, operations that are difficult to simulate can be solved with little to no human input. Machine simulation examples that have been demonstrated with this approach include excavator bucket-leveling, wheel loader bucket-lifting, and truck path-following. Special attention has been given towards finding policies that produce smooth actions and not oscillatory control. These same techniques can help development of on-machine operator-assist features and autonomous control.

Benj enjoys creating software solutions at the place where physical systems intersect applied science and analytics. Having received a BS in Mechanical Engineering from the University of Illinois Urbana-Champaign, Benj has worked at Caterpillar for over 16 years. He spent his early career as a developer of engineering data analysis software specializing in signal processing and is now an analytics technical specialist. He is currently leading the use of machine learning and deep learning technologies to make better Caterpillar products and solutions. He lives in Dunlap, Illinois, with his wife and three girls.

Autoencoder Based Image Segmentation for Precision Agriculture

The use of herbicide in agriculture has skyrocketed in the past few decades. This trend has largely been caused by new genetically modified, herbicide resistant crops. Combating the ecological side-effects of chemical overspray as well as easing the economic burden of costly herbicides is where Blue River Technology (BRT) comes in.

Blue River’s flagship product is See & Spray. An intelligent machine that utilizes deep learning to automatically detect and classify crops and weeds on-the-fly and uses precision sprayers to selectively spray weeds, saving vast quantities of chemicals in the process.

This presentation will cover pixelwise semantic segmentation of rear camera image data collected in the field by See & Spray machines and how that information feeds into an in-field calibration for the fluid sprayer system. On the fly detection of spray allows for a closed feedback loop control system where a GPU accelerated semantic autoencoder model works in tandem with the mechanical sprayer system to achieve precision farming.

Andrei is a senior research scientist at Blue River Technology. He is focused on deep learning and computer vision for perception for smarter agricultural machines. Previously, Andrei worked on processing Airbus’ satellite imagery, drones for Lockheed Martin and text localization and semantic understanding of text for Singapore’s Agency for Science Technology and Research. In his spare time Andrei enjoys skiing and hiking.

Machine Learning through analogies formalized with tools from Category Theory

Industrial applications relying on Machine Learning (ML) to control robotic systems need to learn with few trials. The number of experiments in this context is limited by either the available time or budget. Such a goal can be attained by using knowledge transfer through analogies, both of which can be mathematically formalized with tools from Category Theory. This leads to a new ML approach that transposes accumulated knowledge to new configurations, while still relying on existing ML tools. Illustrations of this innovative solution are presented on a cyber-physical system, a slot-car game. This method also proves to be versatile enough to be used on simulated systems as well, as it is demonstrated on the Atari 2600 games.

Lionel Cordesses holds a Ph.D. in Computer Vision and Control. After developing autonomous vehicles in the late 90s, he joined the Renault group in France where he created and led the Engine Control team. Then, after six years as a Project Manager in the field of Electric Vehicles, he crossed the Atlantic to build and lead Renault’s Artificial Intelligence team in Sunnyvale, CA. Lionel is also an Independent Expert in Signal Processing. He loves to hear "we know it cannot be done", as this presents an exciting challenge to find a unique solution and ultimately create new patents (34 and counting)

Samiron Ray is a Principal at Comet Labs, an early-stage VC fund for AI and robotics startups that are transforming traditional industries. Comet Labs has made over 30 investments to date in companies like Abundant Robotics, Airmap, Cobalt Robotics, Doc.ai, and 3Scan. Samiron is a graduate of Harvard Law School and Duke University and has previously worked in frontier technology seed investing and technology law.

How Machine Vision Fails: Adversarial Attacks and Other Problems

In recent years, we’ve seen remarkable computer vision successes using neural networks. OCR allows automated mail routing. Facial recognition identifies suspects on security video and our friends on social media. Scene segmentation and object detection and classification are used in autonomous vehicle navigation, medical diagnosis and robotic manufacturing.

But we’ve also seen notable failures. A black person is misclassified as a gorilla in Google photos, recalling centuries of racial slurs. In one fatal crash, a Tesla can’t see cross traffic. In another, an Uber misclassifies a pedestrian as an unknown object, then as a vehicle and then as a bicycle, with varying expectations of future travel path. Snow on the road might be mistaken for lane markings. Worn markings and asphalt smoothly transitioning to dirt might not be seen at all. A stop sign can be changed to a speed limit sign with a few pieces of tape or a bit of graffiti. A person can make her face disappear or turn a toy turtle into a rifle. A single pixel change can make an image unrecognizable.

How fragile are the machine vision systems you rely on, what are the ways they will fail and what can you do about it?

An AI policy researcher, lawyer and computer vision scientist, Gretchen Greene advises government leaders on AI strategy, use and policy and works with Cambridge startups on everything from autonomous vehicles to QR codes in wearables. Greene has been a U.S. Departments of Defense, Energy and Homeland Security mathematician, has published in machine learning, science and policy journals and has been interviewed by the Economist, Forbes China and the BBC. An affiliate researcher at MIT’s Media Lab, Greene has a CPhil and MS in math from UCLA and a JD from Yale.

Michael Hayes is the Sr. Manager of Government Affairs at the Consumer Technology Association. Hayes is focused on advancing pro-innovation public policy. He leads CTA’s federal and state efforts on emerging technology issues including artificial intelligence and the sharing economy. He is also responsible for leading federal policy initiatives related to patent litigation reform and high-skilled immigration. Prior to CTA he worked in the U.S. House of Representatives.

Abhishek Gupta is the founder of Montreal AI Ethics Institute. His research focuses on applied technical and policy methods to address ethical, safety and inclusivity concerns in using AI in different domains. Abhishek comes from a strong technical background, working as a Software Engineer, Machine Learning at Microsoft in Montreal.

He is also the founder of the AI Ethics community in Montreal that has more than 1350 members from diverse backgrounds who do a deep dive into AI ethics and offer public consultations to initiatives like the Montreal Declaration for Responsible AI. His work has been featured by the United Nations, Oxford, Stanford Social Innovation Review, World Economic Forum and he travels frequently across North America and Europe to help governments, industry and academia understand AI and how they can incorporate ethical, safe and inclusive development processes within their work. More information can be found on https://atg-abhishek.github.io

Semantic Understanding for Robot Perception

Advancements in robotic navigation and fetch and delivery tasks rest to a large extent on robust, efficient and scalable semantic understanding of the surrounding environment. Deep learning fueled rapid progress in computer vision in object category recognition, localization and semantic segmentation, exploiting large amounts of labelled data and using mostly static images. I will talk about challenges and opportunities in tackling these problems in indoors and outdoors environments relevant to robotics applications. These include methods for semantic segmentation and 3D structure recovery using deep convolutional neural networks (CNNs), localization and mapping of large scale environments, training object instance detectors using synthetically generated training data and 3D object pose recovery. The applicability of the techniques for autonomous driving, service robotics, manipulation and navigation will be discussed.

Jana Kosecka is a Professor at the Department of Computer Science, George Mason University and currently a Visiting Research scientist at Google. She is the recipient of David Marr's prize in Computer Vision and received the National Science Foundation CAREER Award. Jana is an Associate Editor of IEEE Robotics and Automation Letters, Member of the Editorial Board of International Journal of Computer Vision and Associate Editor of IEEE Transactions on Pattern Analysis and Machine Intelligence. She has numerous publications in refereed journals and conferences and is a co-author of a monograph titled Invitation to 3D vision: From Images to Geometric Models. Her general research interests are in Computer Vision and Robotics. In particular she is interested 'seeing' systems engaged in autonomous tasks, acquisition of static and dynamic models of environments by means of visual sensing, object recognition and human-computer interaction.

Autoencoder Based Image Segmentation for Precision Agriculture

The use of herbicide in agriculture has skyrocketed in the past few decades. This trend has largely been caused by new genetically modified, herbicide resistant crops. Combating the ecological side-effects of chemical overspray as well as easing the economic burden of costly herbicides is where Blue River Technology (BRT) comes in.

Blue River’s flagship product is See & Spray. An intelligent machine that utilizes deep learning to automatically detect and classify crops and weeds on-the-fly and uses precision sprayers to selectively spray weeds, saving vast quantities of chemicals in the process.

This presentation will cover pixelwise semantic segmentation of rear camera image data collected in the field by See & Spray machines and how that information feeds into an in-field calibration for the fluid sprayer system. On the fly detection of spray allows for a closed feedback loop control system where a GPU accelerated semantic autoencoder model works in tandem with the mechanical sprayer system to achieve precision farming.

Andrei is a senior research scientist at Blue River Technology. He is focused on deep learning and computer vision for perception for smarter agricultural machines. Previously, Andrei worked on processing Airbus’ satellite imagery, drones for Lockheed Martin and text localization and semantic understanding of text for Singapore’s Agency for Science Technology and Research. In his spare time Andrei enjoys skiing and hiking.

Object Identification and Defect Recognition for Manufacturing and Logistics

Real-time object identification and defects recognition are fundamental building blocks of “Industry 4.0” and “Logistics 4.0”, the latest phases of automation in manufacturing and logistics. An intuitive solution for both problems consists in firstly constructing three-dimensional scale models of all the items in the considered inventory and then analyzing all such models by means of 3D Convolutional Neural Networks. Within this session, an overview of the system architecture considered for tackling this problem is provided, with a specific focus on the 3D convolutional Neural Network which has been trained and deployed. Experimental results are also provided.

Carlo Dal Mutto is a computer vision and machine learning engineer interested in the application of deep learning techniques to 3D data. He has received a Ph.D (Dottorato di ricerca) in Information Engineering from University of Padova, Italy in 2012. Since 2011 he has been working with Aquifi Inc. as advisor, computer vision architect, R&D lead and currently as CTO. He is inventor of several patents, he has been invited speaker at major technical conferences, and he has co-authored research papers, two book chapters and two books on 3D data acquisition and processing: Time-of-Flight Cameras and Microsoft KinectTM, Springer, 2011; Time-of-Flight and Structured Light Depth Cameras: Technology and Applications, Springer, 2016. He has served as a reviewer and TPC member for CVPR, ECCV, ICCV, 3DPVT, 3DIMPVT, ICME, IJCV, IEEE SPL, and Springer MVAP.

Abhishek Gupta is the founder of Montreal AI Ethics Institute. His research focuses on applied technical and policy methods to address ethical, safety and inclusivity concerns in using AI in different domains. Abhishek comes from a strong technical background, working as a Software Engineer, Machine Learning at Microsoft in Montreal.

He is also the founder of the AI Ethics community in Montreal that has more than 1350 members from diverse backgrounds who do a deep dive into AI ethics and offer public consultations to initiatives like the Montreal Declaration for Responsible AI. His work has been featured by the United Nations, Oxford, Stanford Social Innovation Review, World Economic Forum and he travels frequently across North America and Europe to help governments, industry and academia understand AI and how they can incorporate ethical, safe and inclusive development processes within their work. More information can be found on https://atg-abhishek.github.io

Abhishek Gupta is the founder of Montreal AI Ethics Institute. His research focuses on applied technical and policy methods to address ethical, safety and inclusivity concerns in using AI in different domains. Abhishek comes from a strong technical background, working as a Software Engineer, Machine Learning at Microsoft in Montreal.

He is also the founder of the AI Ethics community in Montreal that has more than 1350 members from diverse backgrounds who do a deep dive into AI ethics and offer public consultations to initiatives like the Montreal Declaration for Responsible AI. His work has been featured by the United Nations, Oxford, Stanford Social Innovation Review, World Economic Forum and he travels frequently across North America and Europe to help governments, industry and academia understand AI and how they can incorporate ethical, safe and inclusive development processes within their work. More information can be found on https://atg-abhishek.github.io

Andrew J. Grotto is a Research Fellow at the Hoover Institution and a William J. Perry International Security Fellow at the Center for International Security and Cooperation, both at Stanford University. His research and teaching center on the national security and international economic dimensions of America’s global leadership in information technology innovation.

Before coming to Stanford, Grotto was the Senior Director for Cybersecurity Policy at the White House in both the Obama and Trump Administrations, where he was responsible for coordinating the development and execution of cyber policies relating to critical infrastructure, Federal networks and consumers, as well as broader technology policies with a nexus to cybersecurity, such as artificial intelligence and encryption. Grotto previously served as Senior Advisor for Technology Policy to Commerce Secretary Penny Pritzker; a member of the professional staff of the Senate Select Committee on Intelligence; and as a Senior National Security Analyst at the Center for American Progress.

Roel Schouwenberg is the Intelligence & Research Director at Celsus Advisory Group. He has over twenty years of experience in the information security space. His extensive experience in threat intelligence and security research first exposed him to defensive and adversarial ML and AI over a decade ago. Roel has a special interest in destructive and targeted cyber attacks against physical infrastructure.

AI in Industrial Applications: Challenges, Solutions and Future Directions

Applications of AI in industrial applications are in many ways different from their use in consumer applications, in terms of availability of data, accessibility of data, meaning of data and explainability of data. The presentation then discusses solutions to these thorny challenges, by combining machine learning with existing domain knowhow in the form of knowledge graph. The practices involves using various machine learning practice to convert existing domain knowhow to knowledge graph and to grasp the meaning of data, building standard information models to enable interoperability across domains, and giving explainable answer by combine the power of machine learning and knowledge graph. Finally, the presentation outlooks how combining symbolic and statistic AI will maximize the benefits to industrial applications in general..

Dan Yu is an Innovator at Siemens Corporate Technology since June 2015. Prior to joining Siemens US, he founded Siemens Innovation Center in Wuxi, China in 2012, where Siemens innovates with local partners. He joined Siemens Research (Corporate Technology) China after his study in Tsinghua University and Munich University of Technology. Since then he has been doing technology innovation for a wide spectrum of industrial application areas including industrial manufacturing, intelligent traffic, smart building and logistics with various Internet of Things technologies. Because of the significant innovation contribution, he was awarded “Siemens Inventor of the Year 2010”. Among the 70 patents he authored/coauthored many have already become products or part of Siemens products.