The idea of interfacing minds with machines has long captured the human imagination. Recent advances in neuroscience and engineering are making this a reality, opening the door to restoration and augmentation of human physical and mental capabilities. Medical applications such as cochlear implants for the deaf and neurally controlled prosthetic limbs for the paralyzed are becoming almost commonplace. Brain-computer interfaces (BCIs) are also increasingly being used in security, lie detection, alertness monitoring, telepresence, gaming, education, art, and human augmentation. This introduction to the field is designed as a textbook for upper-level undergraduate and first-year graduate courses in neural engineering or brain-computer interfacing for students from a wide range of disciplines. It can also be used for self-study and as a reference by neuroscientists, computer scientists, engineers, and medical practitioners. Key features include questions and exercises in each chapter and a supporting website.

Preface xiii

1. Introduction 1

PartI Background 7

2. Basic Neuroscience 7

2.1 Neurons 7

2.2 Action Potentials or Spikes 8

2.3 Dendrites and Axons 9

2.4 Synapses 9

2.5 Spike Generation 10

2.6 Adapting the Connections:Synaptic Plasticity 11

      2.6.1 LTP 11

      2.6.2 LTD 11

      2.6.3 STDP 11

      2.6.4 Short-Term Facilitation and Depression 13

2.7 Brain Organization, A natomy, and Function 13

2.8 Summary 16

2.9 Questions and Exercises 17

3. Recordi ng and Stimulating the Brain 18

3.1 Recording Signals from the Brain 18

      3.1.1 Invasive Techniques 18

      3.1.2 Noninvasive Techniques 26

3.2 Stimulating the Brain 32

      3.2.1 Invasive Techniques 32 32

      3.2.2 Noninvasive Techniques 33

3.3 Simultaneous Recording and Stimulation 34

      3.3.1 Multielectrode Arrays 35

      3.3.2 Neurochip 35

3.4 Summary 36

3.5 Questions and Exercises 37

4. Signal Processing 39

4.1 Spike Sorting 39

4.2 Frequency Domain Analysis 40

      4.2.1 Fourier Analysis 40

      4.2.2 Discrete Fourier Transform (DFT) 43

      4.2.3 Fast Fourier Transform (FFT) 45

      4.2.4 Spectral Features 45

4.3 Wavelet Analysis 45

4.4 Time Domain Analysis 46

      4.4.1 Hjorth Parameters 46

      4.4.2 Fractal Dimension 48

      4.4.3 Autoregressive (AR) Modeling 49

      4.4.4 Bayesian Filtering 49

      4.4.5 Kalman Filtering 52

      4.4.6 Particle Filtering 54

4.5 Spatial Filtering 54

      4.5.1 Bipolar, Laplacian, and Common Average Referencing 55

      4.5.2 Principal Component Analysis (PCA) 56

      4.5.3 Independent Component Analysis (ICA) 60

      4.5.4 Common Spatial Patterns (CSP) 61

4.6 Artifact Reduction Techniques 63

      4.6.1 Thresholding 64

      4.6.2 Band-Stop and Notch Filtering 65

      4.6.3 Linear Modeling 65

      4.6.4 Principal Component Analysis (PCA) 66

      4.6.5 Independent Component Analysis (ICA) 66

4.7 Summary 68

4.8 Questions and Exercises 68

5. Machine Learning 71

5.1 Classification Techniques 72

      5.1.1 Binary Classification 72

      5.1.2 Ensemble Classification Techniques 78

      5.1.3 Multi-Class Classification 80

      5.1.4 Evaluation of Classification Performance 84

5.2 Regression 87

      5.2.1 Linear Regression 88

      5.2.2 Neural Networks and Backpropagation 89

      5.2.3 Radial Basis Function (RBF) Networks 92

      5.2.4 Gaussian Processes 93

5.3 Summary 96

5.4 Questions and Exercises 96

PartII Putting It All Together

6. Building a BCI 101

6.1 Major Types of BCis 101

6.2 Brain Responses Useful for Building BCis 101

      6.2.1 Conditioned Responses 101

      6.2.2 Population Activity 102

      6.2.3 Imagined Motor and Cognitive Activity 103

      6.2.4 Stimulus-Evoked Activity 103

6.3 Summary 104

6.4 Questions and Exercises 105

PartIII Major Types of BCis

7. Invasive BCis 109

7.1 Two Major Paradigms in Invasive Brain-Computer Interfacing 109

      7.1.1 BCis Based on Operant Conditioning 109

      7.1.2 BCIs Based on Population Decoding 111

7.2 Invasive BCis in Animals 113

      7.2.1 BCis for Prosthetic Arm and Hand Control 113

      7.2.2 BCis for Lower-Limb Control 126

      7.2.3 BCis for Cursor Control 129

      7.2.4 Cognitive BCis 132

7.3 Invasive BCis in Humans 137

      7.3.1 Cursor and Robotic Control Using a Multielectrode Array Implant 138

      7.3.2 Cognitive BCis in Humans 143

7.4 Long-Term Use of Invasive BCis 143

      7.4.1 Long-Term BCI Use and Formation of a Stable Cortical Representation 144

      7.4.2 Long-Term Use of a Human BCI Implant 144

7.5 Summary 146

7.6 Questions and Exercises 147

8. Semi-Invasive BCis 149

8.1 Electrocorticographic (ECoG) BCis 149

      8.1.1 ECoG BCis in Animals 150

      8.1.2 ECoG BCis in Humans 151

8.2 BCis Based on Peripheral Nerve Signals 169

      8.2.1 Nerve-Based BCis 170

      8.2.2 Targeted Muscle Reinnervation (TMR) 173

8.3 Summary 174

8.4 Questions and Exercises 175

9. Noninvasive BCis 177

9.1 Electroencephalographic (EEG) BCis 177

      9.1.1 Oscillatory Potentials and ERD 178

      9.1.2 Slow Cortical Potentials 187

      9.1.3 Movement-Related Potentials 189

      9.1.4 Stimulus-Evoked Potentials 193

      9.1.5 BCis Based on Cognitive Tasks 199

      9.1.6 Error Potentials in BCis 200

      9.1.7 Coadaptive BCis 201

      9.1.8 Hierarchical BCis 203

9.2 Other Noninvasive BCis：fMRI, MEG, and fNIR 203

      9.2.1 Functional Magnetic Resonance Imaging-Based BCis 204

      9.2.2 Magnetoencephalography-Based BCis 205

      9.2.3 Functional Near Infrared and Optical BCis 206

9.3 Summary 206

9.4 Questions and Exercises 207

10. BCis that Stimulate 210

10.1 Sensory Restoration 210

      10.1.1 Restoring Hearing: Cochlear bηplants 210

      10.1.2 Restoring Sight: Cortical and Retinal Implants 213

10.2 Motor Restoration 216

      10.2.1 Deep Brain Stimulation (DBS) 216

10.3 Sensory Augmentation 217

10.4 Summary 219

10.5 Questions and Exercises 219

11. Bidirectional and Recurrent BCis 221

11.1 Cursor Control with Direct Cortical Instruction via Stimulation 221

11.2 Active Tactile Exploration Using a BCI and Somatosensory Stimulation 224

11.3 Bidirectional BCI Control of a Mini-Robot 226

11.4 Cortical Control of Muscles via Functional Electrical Stimulation 229

11.5 Establishing New Connections between Brain Regions 230

11.6 Summary 234

11.7 Questions and Exercises 234

PartIV Applications and Ethics

12. Applications of BCis 239

12.1 Medical Applications 239

      12.1.1 Sensory Restoration 239

      12.1.2 ν1otor Restoration 240

      12.1.3 Cognitive Restoration 240

      12.1.4 Rehabilitation 240

      12.1.5 Restoring Communication with Menus, Cursors, and Spellers 241

      12.1.6 Brain-Controlled Wheelchairs 241

12.2 Nonmedical Applications 242

      12.2.1 Web Browsi ng and Navigating Virtual Worlds 243

      12.2.2 Robotic Avatars 245

      12.2.3 High Throughput Image Search 248

      12.2.4 Lie Detection and Applications in Law 248

      12.2.5 Monitoring Alertness 253

      12.2.6 Estimating Cognitive Load 256

      12.2.7 Education and Learning 258

      12.2.8 Security, Identification , and Authentication 260

      12.2.9 Physical Amplification with Exoskeletons 261

      12.2.10 Mnemonic and Cognitive Amplification 262

      12.2.11 Applications in Space 263

      12.2.12 Gaming and Entertainm ent 265

      12.2.13 Brain -Controlled Art 267

12.3 Summary 269

12.4 Questions and Exercises 269

13. Ethics of Brain -Computer Interfacing 272

13.1 Medical, Health, and Safety Issues 272

      13.1.1 Balancing Risks versus Benefits 272

      13.1.2 Informed Consent 273

13.2 Abuse of BCI Technology 273

13.3 BCI Security and Privacy 274

13.4 Legal Issues 275

13.5 Moral and Social Justice Issues 276

13.6 Summary 277

13.7 Questions and Exercises 277

14. Conclusion 279

Appendix: Mathematical Background 281

A.1 Basic Mathematical Notation and Units of M easurement 281

A.2 Vectors, Matrices, and Linear Algebra 282

      A.2.1 Vectors 282

      A.2.2 Matrices 284

      A.2.3 Eigenvectors and Eigenva lues 287

      A.2.4 Lines, Planes, and Hyperp lanes 288

A.3 Probability T11eory 288

      A.3.1 Random Variables and Axioms of Probability 288

      A.3.2 Joint and Conditional Probability 289

      A.3.3 Mean, Variance, and Covariance 290

      A.3.4 Probability Density Function 291

      A.3.5 Uniform Distribution 291

      A.3.6 Bernoulli Distribution 291

      A.3.7 Binomial Distribution 292

      A.3.8 Poisson Distribution 292

      A.3.9 Gaussian Distribution 293

      A.3.10 Multivariate Gaussian Distribution 293

References 295

Index 307

Color plates follow page 176

中科院文献情报中心