Special Forum

Tuesday, October 11, 13:45-16:05, Grand Ballroom

Organizer: Daniel Lee, University of Pennsylvania

This forum will explore the impact of AI and Deep Learning methods in robotics. Keynote speakers from prominent academic and industrial institutions in North America, Asia, and Europe will present their latest research and discuss potential future applications and implications during the ensuing panel discussion.

  Sergey Levine, UC Berkeley

The problem of building an autonomous robot has traditionally been viewed as one of integration: connecting together modular components, each one designed to handle some portion of the perception and decision making process. For example, a vision system might be connected to a planner that might in turn provide commands to a low-level controller that drives the robot's motors. In this talk, I will discuss how ideas from deep learning can allow us to build robotic control mechanisms that combine both perception and control into a single system. This system can then be trained end-to-end on the task at hand. I will show how this end-to-end approach actually simplifies the perception and control problems, by allowing the perception and control mechanisms to adapt to one another and to the task. I will also present some recent work on scaling up deep robotic learning on a cluster consisting of multiple robotic arms, and demonstrate results for learning grasping strategies that involve continuous feedback and hand-eye coordination using deep convolutional neural networks.


Professor Sergey Levine is an assistant professor at UC Berkeley. His research focuses on robotics and machine learning. In his PhD thesis, he developed a novel guided policy search algorithm for learning complex neural network control policies, which was later applied to enable a range of robotic tasks, including end-to-end training of policies for perception and control. He has also developed algorithms for learning from demonstration, inverse reinforcement learning, efficient training of stochastic neural networks, computer vision, and data-driven character animation.

  Yuanqing Lin, Director, Baidu Institute of Deep Learning

Thanks to big data, deep learning algorithms and high-performance computing, artificial intelligence (AI) technologies have gained dramatic developments in the past a few years. At Baidu, AI is the (next) big thing. This talk will present our strategy and practice. Aiming to achieve artificial general intelligence one day, we are taking two directions simultaneously: 1) developing common fundamental technologies ranging from deep learning platform, basic computer vision technologies such as image classification, object detection/tracking, image/video segmentation, etc.; 2) building ultimate-level AI technologies in vertical domains such as food recognition, car/pedestrian detection, etc. The talk will present some case studies with strong emphasis on forming the close loop of algorithms, applications, users and data. These advances in deep learning and computer vision offer up new possibilities not only for mobile/internet apps but also for robotics.


Dr. Yuanqing Lin is now the Director of the Baidu Institute of Deep Learning. Dr. Lin received his Ph.D. degree in Electrical Engineering from University of Pennsylvania in 2008. After that, he joined NEC Labs America as a Research Staff Member, working on feature learning and large-scale classification. In 2010, he led the NEC-UIUC team and won the No.1 place in ImageNet Large Scale Visual Recognition Challenge. In April 2012, he became the head of the Media Analytics Department at NEC Labs America. His team was focusing on two major research directions, large-scale fine-grained image recognition and 3D visual sensing for autonomous driving. In November 2015, he joined Baidu, and his research interests are in machine learning, computer vision and robotics. He served as the Area Chair for NIPS 2015.

  Jianwei Zhang, University of Hamburg

In a dynamic and changing world, a robust and effective robot system must have adaptive behaviors, incrementally learnable skills and a high-level conceptual understanding of the world it inhabits, as well as planning capabilities for autonomous operations. Future intelligent control systems will benefit from the recent research on neurocognitive models in processing multisensory data, exploiting synergy, integrating high-level knowledge and learning, etc. I will first introduce crossmodal integration methods for intelligent service robots. Then I will present our investigation and experiments on synergy technique which uses fewer parameters to govern the high DOF of multifinger robot movement. The third part of my talk will demonstrate how an intelligent system like a robot can evolve its model as a result of learning from experiences; and how such a model allows a robot to better understand new situations by integration of knowledge, planning and learning. I will show some integrated results of operational mobile robot platforms with grasping facilities in a restaurant service scenario.


Jianwei Zhang is professor and head of TAMS, Department of Informatics, University of Hamburg, Germany. He received both his Bachelor of Engineering (1986, with distinction) and Master of Engineering (1989) at the Department of Computer Science of Tsinghua University, Beijing, China, his PhD (1994) at the Institute of Real-Time Computer Systems and Robotics, Department of Computer Science, University of Karlsruhe, Germany, and Habilitation (2000) at the Faculty of Technology, University of Bielefeld, Germany. His research interests are service robotics, sensor fusion, service robotics and multimodal machine learning, cognitive computing of Industry4.0, etc. In these areas he has published about 300 journal and conference papers, technical reports, four book chapters and three research monographs. He holds 37 patents on service robot components and systems. He is the coordinator of the DFG/NSFC Transregional Collaborative Research Centre SFB/TRR169 "Crossmodal Learning" and several EU robotics projects.

He has received several awards, including the IEEE ROMAN Best Paper Award in 2002, the IEEE AIM Best Paper Award 2008, the IEEE ROBIO Best Conference Paper Award 2013 and ROBIO Best Paper on Biomimetics 2014. He is the General Chairs of IEEE MFI 2012 and IEEE/RSJ IROS 2015, IEEE Robotics and Automation Society AdCom (2013-2015). Jianwei Zhang is life-long Academician of Academy of Sciences in Hamburg.
13:45-13:55: Welcome and Forum Introduction
----------- Daniel Lee
13:55-14:30: Deep Robotic Learning
----------- Sergey Levine (UC Berkeley)
14:30-15:05: Artificial Intelligence, the (Next) Big Thing
----------- Yuanqing Lin (Baidu)
15:05-15:40: Crossmodal, Experience-Based Learning of Service Robot Systems
----------- Jianwei Zhang (U. Hamburg)
15:40-16:05: Panel Discussion
----------- Manuela Veloso, Sergey Levine, Yuanqing Lin, Jianwei Zhang  
Wednesday, October 12, 13:30-15:50, Grand Ballroom

Organizer: Hyunchul Shim, KAIST

Recently, there have been great advances in autonomous systems that are deployed in real world to perform complex tasks owing to the advances in various aspects of robotics from sensing, decision-making to control. Especially, mobile robots have shown great progress in not only on the development, but on the physical applications and even legislation for day-to-day applications. In this series of talks, the speakers in this field presents the latest results on autonomous systems in three perspectives: the decision making algorithms, application to real systems, and the transition to real world in UAVs. In the first talk, Prof. Daniel Lee explains a number of estimation, mapping, planning and control on various systems. Prof. Jianda Han introduces his work on polar exploration using autonomous robots. Prof. Shim introduces the latest advances in unmanned aerial vehicles, which is now considered for full integration in civil airspace. This session is concluded with a panel discussion with the invited speakers with active discussions on the topics The After the detailed introduction of these areas by the speakers, a panel discussion is held to discuss more deeply in the topic, along with the questions and comments from the participating audiences.
  Daniel Lee, University of Pennsylvania

Current AI systems for perception and action incorporate a number of techniques: Bayesian state estimation, probabilistic mapping, trajectory planning, and feedback control. I will describe and demonstrate some of these methods on various autonomous systems including wheeled, legged, and flying robots. In order to model variability due to pose, illumination, and background changes, low-dimensional manifold representations have been used for learning in these systems. But how well can such manifolds be processed by neural networks? I will show how notions of linear separability and VC dimension can be generalized from input points to manifolds. This analysis provides theoretical predictions for the capacity and generalization ability of invariant classifiers, and better understanding of the performance of deep neural networks. .


Daniel Lee is the UPS Foundation Chair Professor in the School of Engineering and Applied Science at the University of Pennsylvania. He received his B.A. summa cum laude in Physics from Harvard University and his Ph.D. in Condensed Matter Physics from the Massachusetts Institute of Technology in 1995. Before coming to Penn, he was a researcher at AT&T and Lucent Bell Laboratories in the Theoretical Physics and Biological Computation departments. He is a Fellow of the IEEE and AAAI and has received the National Science Foundation CAREER award and the University of Pennsylvania Lindback award for distinguished teaching. He was also a fellow of the Hebrew University Institute of Advanced Studies in Jerusalem, an affiliate of the Korea Advanced Institute of Science and Technology, and organized the US-Japan National Academy of Engineering Frontiers of Engineering symposium.

As director of the GRASP Laboratory and co-director of the CMU-Penn University Transportation Center, his group focuses on understanding general computational principles in biological systems, and on applying that knowledge to build autonomous systems.

  Jianda Han, Shenyang Institute of Automation, Chinese Academy of Sciences

Exploration of the polar region is of great importance for scientific research on global climate change and the evolution of the earth. However, the tough environment, e.g., low temperature, strong wind, complex terrains, makes human exploration difficult and dangerous. In 2011, the China National High Technology Research and Development (863) Program and National Antarctic Research Expedition (CHINARE) jointly launched a project to design ground mobile and flying robots to conduct large-scale scientific explorations on Antarctica. In this talk, the corresponding techniques, especially the polar rover's prototypes and autonomous control, as well as the preliminary field test results on Antarctica are introduced. The challenging open problems are also summarized as conclusion..


Jianda Han received his PhD on Electrical Engineering from Harbin Institute of Technology in 1998. Currently he is a professor and vice director of the State Key Laboratory of Robotics, Shenyang Institute of Automation, Chinese Academy of Sciences. His research interests include nonlinear estimation and control, control for the autonomy of robots, and robotic system integrations and applications. He currently also serves as a member of the Expert Panel of the Intelligent Robot Division, the National 863 Program of China.

  Hyunchul Shim, KAIST

Unmanned aerial vehicles have undergone rapid development since 1990s. Originally developed for military applications, they have been extremely successful for missions where direct human participation is deemed dull or dangerous. Thanks to their remarkable success, they began to enter the civil airspace for border patrol and aerial surveillance in late 1990s. During the last few years, the field of UAVs are seeing yet another revolution: the advent of small multirotor drones. This new type of drone is affordable yet very capable for aerial photography and even package deliveries. These two types of drones are entering the airspace in two distinct ways: the larger ones are now considered for full integration for civil airspace by International Civil Aviation Organization. The accommodation of small drones are trickier: they need a whole new way of integration, which is now being tackled by a number of researchers. The success of drone is in debt to the advances in computers, sensors, communications and algorithms: they can perform various missions with great flexibility and reliability. It is very impressive to see how a new robotic device is conceived, developed, and evolved into a mature system so that a whole new industry is formed and rules and regulations are created for them. The speaker intends to share his views obtained from his research since 1991 and also his activities as a member of ICAO RPAS Panel. .


Prof. Hyunchul Shim received the B.S. and M.S. degrees in mechanical design and production engineering from Seoul National University, Seoul, Korea, in 1991 and 1993, respectively, and the Ph.D. degree in mechanical engineering from the University of California Berkeley, Berkeley, USA in 2000. From 1993 to 1994, he was with Hyundai Motor Company, Korea. From 2001 to 2005, he was with Maxtor Corporation, Milpitas, CA, USA as Staff Engineer. From 2005 to 2007, he was with the University of California Berkeley as Principal Engineer, in charge of Berkeley Aerobot Team. In 2007, he joined the Department of Aerospace Engineering, Korea Advanced Institute of Science and Technology (KAIST), Daejon, Korea, as an Assistant Professor. He is now a tenured Associate Professor. He began his research on autonomous vehicles in 1991 and has been recognized as one of the pioneers in unmanned aerial vehicles for his work on flight control system design, collision avoidance, vision-based aerial mapping and navigation, and anti-drone technologies.

He received a number of awards including the 2nd prize in Global Student Design Competition by National Instrument, USA(2014), Outstanding award by the Minister of Science, ICT and Future Planning of Korean Government, Best paper awards by Qualcomm Inc, Korean Aerospace Industry('09,'14). He is serving as the Director of Intelligent UAV Research Laboratory and also Director of Korean Civil RPAS Research Center. He is serving as advisor for RPAS Panel in ICAO and a member of Global Future Council of World Economic Forum.
13:30-13:40: Welcome and Forum Introduction
----------- Hyunchul Shim
13:40-14:10: Decision Making and Manifold Representations in Intelligent Autonomous Systems
----------- Daniel Lee
14:10-14:40: Autonomy and Polar Robots
----------- Jianda Han
14:40-15:10: From Theory to Policy: the case of Unmanned Aerial Vehicles
----------- Hyunchul Shim
15:10-15:50: Panel Discussion  

Thursday, October 13, 13:30-15:50, Grand Ballroom

Organizer: Byung-Ju Yi, Professor, Hanyang University
Co-organizers: Makoto Hashizume, Professor and MD, Kyushu University Hospital

Efforts to develop and commercialize medical robots (specifically, surgical robots) has been made intensively all over the world. However, there are only small number of outputs that made success in commercialization. This forum is organized to discuss several aspects that must be taken into account to come up with successful medical devices. Specifically, education issue will be mainly discussed. There is a big difference between the industrial devices and medical devices. More attention has to be paid to "safety" rather than "benefit" of the patients. Although we have known this issue, relatively little effort has been made to educate engineers and medical doctors for this issue. The engineers must have many opportunities to observe the real clinical situation, understand how medical devices are being handled in OR, and listen to the needs from the clinical fields. Medical doctors should also understand the current state of art of robotic technology available through discussion with the engineers in various fields. Therefore, this forum will have three experts from engineering side, clinical side and company side, and ask them to present their experience on education or training of graduate students, medical doctors, and engineers for effective collaboration and successful commercialization of medical robots or devices. After three key-note speeches, a panel discussion will follow..


  Byung-Ju Yi, Hanyang University

Efforts to develop and commercialize medical robots (specifically, surgical robots) has been made intensively all over the world. However, there are only a few number of output that make success in commercialization. In light of this fact, this talk would like to discuss about several aspects that enable us to produce fruitful research output and allow translation to industry. Specifically, this talk would like to discuss about how society can contribute to raise engineer in medical robotics research. Education for engineers in medical robotics involves general education program in engineering and clinic. However, there exists another important issue that has been ignored or has not been paid much attention in medical robotics area. That is the education that considers the certification process in medical robotics. Many countries make an effort to get certification when they develop medical equipment (ME). Certification is necessary not only for commercialization but also for user safety. The certification authorities of the United States, Europe, and Korea are FDA, CE, and KFDA, respectively. Typically ME standards are defined by the Quality Assurance System which vouches the manufactured medical equipment being safe and effective. The Korean standard of medical equipment is followed by the International Standard IEC 60601 which is the same as Europe, Japan, and U.S.A. This talk briefly goes over medical equipment certification standards. Prior to development of ME, the grade of the device should be determined first. Usually, ME is classified by four or three grades according to the purpose of use and potential risks. For instance, the VI master-slave robot that inserts a catheter into the patient's vessel is classified as the 2nd grade. This is because the risk to life is low, even though there is some risk to human health in case of breakdown or malfunction. Initially, process of identifying four grades is also explained. Several successful and failed examples for each grade will be addressed. For certification, developers have to adjust to several standards. These standards are for biological safety, electrical safety, and mechanical et al. This talk will take otologic surgical device and vascular intervention robotic system as examples to explain the standards for safety and performance of medical devices. Specially, safety of electric and mechanical aspects will be mainly introduced through modified design examples of medical robots.


1984: Bachelor, Hanyang University, Mechanical Engineering, Korea
1986: Master degree from The University of Texas at Austin, Mechanical Engineering
1991: Ph.D. from The University of Texas at Austin, Mechanical Engineering, USA
1992-1995: Assistant Professor, Department of Mechanical Engineering, Korea University of Education and Technology, Korea
1995 ~: Professor, School of Electrical Engineering, Hanyang University, Korea 2004-2005: Visiting professor, Department of Mechanical Engineering, Johns Hopkin University, USA 2012: Visiting professor, Kyushu University Hospital, Japan.
2014 ~: Director, Center for ICT-based Medical robotics, Hanyang University, Korea

His current research interest is general robotic mechanism theory and its application to medical robotic system. Special interest is in the area of ENT, neurosurgical, and vascular intervention areas. He is currently the president of Korea Society of Medical Robotics and vice president of Korean Robotics Society.
  Makoto Hashizume, MD, PhD, FACS, Distinguished Professor, Kyushu University

Minimally invasive endoscopic surgery is now recognized as a standard operation in almost all of the surgical fields. Thanks to the development of information and robotic technology, the technical difficulties have been overcome in part, especially in the movement of the instruments and in the operative fields by introducing super-hand with 7 degrees of freedom and 3D endoscope, respectively. However, the surgical application of the robotic surgery is still limited to the pelvic cavity. There are many problems for further development to be solved in clinical situation. Among them education is the most important issue for both engineers and medical doctors to understand the whole process from creating an idea to producing commercial products. The final purpose is to contribute to human healthcare with medical devices which are useful to care the patients. However, the big difference between the industrial devices and medical devices is the contact to human. More attention has to be paid to “safety” rather than “benefit” of the patients. If “cost performance is poorer compared to that of conventional one, hospital director would not allow the each doctor to use or purchase it. When the medical devices are for the operative use, sterilization is often no marked by engineers. They must have an experience to observe the real clinical situation and to understand how the devices are used in OR and what is the needs in clinical situation. According to the current change of real condition, our interests are now being moved on personalized precision medicine based on multidisciplinary computational anatomy as well as on less invasiveness from the view point of quality of life. Integration of medicine and engineering is one of the solutions for the current problems. The ideal circumstance might be where multidisciplinary personnel with the same purpose is working together in the same room of the same hospital at the same time.


1979: Graduate from Kyushu University, School of Medicine (MD)
1984: Graduate from Kyushu University, Graduate School of Medical Sciences (PhD)
1998-1999: Associate Professor, Department of Surgery II, Kyushu University Graduate School of Medical Sciences
1999-: Professor, Division of Disaster and Emergency Medicine, Department of Advanced Medical Initiatives, Kyushu University Graduate School of Medical Science
2003-: Director, Department of Advanced Medicine and Innovative Technology, Kyushu University Hospital
2006-2012: Director, Emergency and Critical Care Center, Kyushu University Hospital
2010-: Director, Center for Advanced Medical Innovation, Kyushu University

2016-: Director, International Research Center for Multidisciplinary Computational Anatomy His current research interest is in development of minimally invasive surgical robotic system and multidisciplinary computational anatomy. He received an official commendation for innovative technology from the Minister of Education, Culture, Sports, Science and Technology in 2006.
  Salvatore J. Brogna, Executive Vice President, Product Operations, Intuitive Surgical Co.

Robotic assisted surgery has given surgeons open surgery-like capabilities in minimally invasive format. To advance the capabilities of the next generation of robotic assisted surgery, we should challenge ourselves to increase usability and convenience for the operating room staff while improving the information available to surgeons helping them to make faster decisions and achieve better patient outcomes. The next generation of engineers developing robotic assisted surgical systems must be able to apply the latest advancements in technologies (haptics, optics, controls, and mechatronics) with improved application of human factors and usability design. To foster cross-discipline learning for engineers, training pathways designed to promote advanced learning through deep understanding of engineering sciences and experimental methods is essential, so they can best meet the needs of the surgical community. Mr. Brogna will share insights and real company experiences in bridging medical needs with engineering technology for the next generation of surgery.


Salvatore "Sal" J. Brogna joined Intuitive Surgical in 1999 as part of the product design team. He has held the titles of Vice President, Engineering, Senior Vice President, Product Development as in now the Executive Vice President, Product Operations. Prior to joining Intuitive, Mr. Brogna led design and development of complex robotic systems for 15+ years at Unimation, Genesis Automation and Adept Technology. Mr. Brogna holds a B.S. and a M.S. in Mechanical Engineering from Clarkson University in New York.

Panel lists
Prof. Guang-Zhong Yang (Director, The Hamlyn Centre for Robotic Surgery, Imperial College London, UK. Plenary speaker of IROS2016)
Prof. Pierre Jannin (INSERM Research Director, Medical School of the University of Rennes, France. President of ISCAS)
Prof. Danail Stoyanov (University College London, UK, Board member of ISCAS)  
13:30-14:10: How to Educate Engineers in Medical Robotics?
----------- Byung-Ju Yi
14:10-14:40: Education in Medical Robotics for Successful Translation to Industry
----------- Makoto Hashizume
14:40-15:10: Educating the Next Generation of Robotic Assisted Engineers
----------- Salvatore J. Brogna
15:10-15:40: Presentation of Panel Lists
15:40-16:00: General discussion