Wearable devices have served in various application scenarios such as interaction, healthcare, and robot learning. Due to the advantages on capturing the information of the humans' anipulation and providing high-quality demonstrations for robotic imitation, wearable devices are applied on robots to acquire manipulation skills. However, the robotic manipulation learning using wearable device faces great challenges. Wearable devices often need multiple sensors to capture human operation information, which brings in many challenges such as wearable device design, sensor calibration, wear calibration, information fusion of different sensors, and so on. In addition to that, none of the existing work exploits the complete motion including that of not only hands but also arms. Meanwhile, the demonstration dataset is difficult to be reused due to the different settings (such as in sensing and mechanics) of the teachers and the learners. Furthermore, minimal parameter tuning and learning times requiring few training examples are desirable in learning strategies. To address the above-mentioned challenges, we developed a novel wearable device that captured more information of gestures, built the manipulation demonstration and the robot learning technology was introduced to improve the manipulation performance.
This book is divided into three parts: The first part presents the research background and motivation and introduces the development of wearable technologies and applications of wearable devices. The second part focuses on wearable technologies. In Chap. 2, wearable sensors including inertial sensors and tactile sensors are presented. In Chap. 3, multisensor fusion methods are developed to tackle the accurate motion capture by wearable devices. Chapter 4 demonstrates their applications including gesture recognition, tactile interaction, and tactile perception. The third part presents the methods of robotic manipulation learning. Chapter 5 tackles the problem of manipulation learning from the teleoperation demonstration of a wearable device using dynamical movement primitive. Chapters 6 addresses the problem of manipulation learning from visual-based teleoperation demonstration by developing deep neural networks methodology. Chapter 7 focuses on learning from a wearable-based indirect demonstration. The fourth part contains Chap. 8, which summarizes this book and presents some prospects. For a clear illustration, an outline of the logical dependency among chapters is demonstrated in Fig. 1. Please note that we try our best to make each chapter self-contained.
This book reviews the current study status and application of wearable devices to summarize the design methods currently used, analyze their strengths and weak points, and list the future research trend.
The remainder of the book is organized as follows: Part II describes the wearable devices. Part III reviews robotic manipulation and learning. Conclusion with future research trends is outlined in Part IV.
This book is suitable as a reference book for graduate students with a basic knowledge of machine learning as well as professional researchers interested in robotic tactile perception and understanding and machine learning.

Part I Background

1 Introduction 3

      1.1 The Overview of Wearable Devices 3

      1.1.1 Wrist-Worn 4

      1.1.2 Head Mounted 6

      1.1.3 Body Equipped 7

      1.1.4 Smart Garment 9

      1.1.5 Smart Shoes 9

      1.1.6 Data Gloves 10

      1.2 Sensors of Wearable Device 11

      1.2.1 Motion Capture Sensor 11

      1.2.2 Tactile Sensors 13

      1.2.3 Physiological Parameter Measurement Sensors 14

      1.3 Wearable Computing Algorithms 15

      1.3.1 Motion Capture Related Algorithms 15

      1.3.2 Motion Recognition Related Algorithms 16

      1.3.3 Comparison of Different Wearable Computing Algorithms 17

      1.4 Applications 18

      1.4.1 Interaction 19

      1.4.2 Healthcare 20

      1.4.3 Manipulation Learning 21

      1.5 Summary 22

      References 22

Part II Wearable Technology

2 Wearable Sensors 33

      2.1 Inertial Sensor 33

      2.1.1 Analysis of Measurement Noise 34

      2.1.2 Calibration Method 37

      2.1.3 Experimental Results 47

      2.2 Tactile Sensor 50

      2.2.1 Piezo-Resistive Tactile Sensor Array 53

      2.2.2 Capacitive Sensor Array 56

      2.2.3 Calibration and Results 58

      2.3 Summary 61

      References 61

3 Wearable Design and Computing 65

      3.1 Introduction 65

      3.2 Design 66

      3.2.1 Inertial and Magnetic Measurement Unit Design 67

      3.2.2 Wearable Design 68

      3.3 Motion Capture Algorithm 70

      3.3.1 Models of Inertial and Magnetic Sensors 70

      3.3.2 QEKF Algorithm 72

      3.3.3 Two-Step Optimal Filter 75

      3.4 Experimental Results 82

      3.4.1 Orientations Assessment 82

      3.4.2 Motion Capture Experiments 83

      3.5 Summary 86

      References 87

4 Applications of Developed Wearable Devices 89

      4.1 Gesture Recognition 89

      4.1.1 ELM-Based Gestures Recognition 90

      4.1.2 CNN-Based Sign Language Recognition 93

      4.1.3 Experimental Results 98

      4.2 Tactile Interaction 108

      4.3 Tactile Perception 110

      4.3.1 Tactile Glove Description 111

      4.3.2 Visual Modality Representation 112

      4.3.3 Tactile Modality Representation 114

      4.3.4 Visual-Tactile Fusion Classification 116

      4.3.5 Experimental Results 117

      4.4 Summary 121

      References 122

Part III Manipulation Learning from Demonstration

5 Learning from Wearable-Based Teleoperation Demonstration 127

      5.1 Introduction 127

      5.2 Teleoperation Demonstration 129

      5.2.1 Teleoperation Algorithm 129

      5.2.2 Demonstration 131

      5.3 Imitation Learning 132

      5.3.1 Dynamic Movement Primitives 132

      5.3.2 Imitation Learning Algorithm 134

      5.4 Experimental Results 136

      5.4.1 Robotic Teleoperation Demonstration 136

      5.4.2 Imitation Learning Experiments 136

      5.4.3 Skill-Primitive Library 139

      5.5 Summary 141

      References 142

6 Learning from Visual-Based Teleoperation Demonstration 145

      6.1 Introduction 145

      6.2 Manipulation Learning of Robotic Hand 147

      6.3 Manipulation Learning of Robotic Arm 152

      6.4 Experimental Results 159

      6.4.1 Experimental Results of Robotic Hand 159

      6.4.2 Experimental Results of Robotic Arm 163

      6.5 Summary 169

      References 170

7 Learning from Wearable-Based Indirect Demonstration 173

      7.1 Introduction 173

      7.2 Indirect Wearable Demonstration 177

      7.3 Learning Algorithm 182

      7.3.1 Grasp Point Generalization on Incomplete Point Cloud 182

      7.3.2 Grasp Model Built by "Thumb" Finger 185

      7.3.3 Wrist Constraints Estimation 187

      7.4 Experimental Results 191

      7.4.1 Object Classification Based on Shape Descriptors 191

      7.4.2 Comparisons of Shape Descriptors for Grasp Region Detection 192

      7.4.3 Grasp Planning 196

      7.5 Summary 201

      References 202

Part IV Conclusions

8 Conclusions 207

Bin Fang is an Assistant Researcher at the Department of Computer Science and Technology, Tsinghua University. His main research interests include wearable devices and human-robot interaction. He is a leader guest editor for a number of journals, including Frontiers in Neurorobotics, and Frontiers in Robotics and AI, and has served as an associate editor for various journals and conferences, e.g. the International Journal of Advanced Robotic Systems, and the IEEE International Conference on Advanced Robotics and Mechatronics. PA\Fuchun Sun is a Full Professor at the Department of Computer Science and Technology, Tsinghua University. A recipient of the National Science Fund for Distinguished Young Scholars, his main research interests include intelligent control and robotics. He serves as an associate editor for a number of international journals, including IEEE Transactions on Systems, Man and Cybernetics: Systems, IEEE Transactions on Fuzzy Systems, and Mechatronics, Robotics and Autonomous Systems. PA\Huaping Liu is an Associate Professor at the Department of Computer Science and Technology, Tsinghua University. His main research interests include robotic perception and learning. He serves as an associate editor for various journals, including IEEE Transactions on Automation Science and Engineering, IEEE Transactions on Industrial Informatics, IEEE Robotics & Automation Letters, Neurocomputing, and Cognitive Computation. PA\​Chunfang Liu is an Assistant Professor at the Department of Artificial Intelligence and Automation, Beijing University of Technology. Her research interests include intelligent robotics and vision. PA\Di Guo received her Ph.D. degree from the Department of Computer Science and Technology, Tsinghua University, Beijing, in 2017. Her research interests include robotic manipulation and sensor fusion.

中科院文献情报中心