Brain—Machine Interface: Insights on Sensorimotor Stability from the First Bilateral Microelectrode Array Implants in a Human

Nickl, RW et al. Characteristics and stability of sensorimotor activity driven by isolated-muscle group activation in a human with tetraplegia. Sci Rep 12, 10353 (2022). https://doi.org/10.1038/s41598-022-13436-2
Like

One September morning, a man sitting at breakfast with a knife and fork sliced off a piece of cake, lifted it to his mouth, and made history.  What made this event truly notable was that he did not feed himself physically.  Instead, he guided two robotic arms to do this for him using a brain-machine interface (BMI) [2].

Figure 1: Self-feeding using a brain-machine interface (BMI) controlled by bilaterally-implanted microelectrode arrays.  Adapted from Supplementary Material from Handelman et al. [2].  Click on image to view full video (at https://www.frontiersin.org/articles/10.3389/fnbot.2022.918001/full#supplementary-material).

This self-feeding demonstration capped a joint research study between the Johns Hopkins School of Medicine and the Johns Hopkins Applied Physics Lab (APL), where, for the first time, a human with tetraplegia used a movement-assistive BMI to simultaneously read from both sides of his brain via implanted microelectrode arrays (MEAs)[1-5].  Our collaboration exemplifies a recent trend of expanding MEA coverage in the brain beyond the dominant primary motor cortex [6-7], into additional areas such as sensory cortex[8], and now both brain hemispheres.

A fundamental component of BMIs is the neural decoder, which translates brain signals into directives for controlling a robot or other device.  If a BMI user is like the composer of a symphony, then the decoder is like the conductor, blending the user’s brain signals, interpreting his or her intentions.  Brain signals can embed rich information, but can also be highly unstable---often to the point that the decoder must be recalibrated multiple times during a single BMI session [8-10].  Such need to frequently retune the orchestra hinders BMIs’ readiness for upcoming tasks, and is a barrier to their full potential as a rehabilitation technology.  Studying movement representation stability across brain regions may shed light on how to design more robust decoders to overcome this barrier.

Testing for neural stability in tetraplegia

In our recent article in Scientific Reports, we investigate stability across brain areas and hemispheres in our participant with tetraplegia, slightly over 1 year after he was implanted with MEAs in brain areas associated with wrist/hand control and sensation (6 total; Fig. 2A below).  Altogether, we measured primary motor and sensory cortices in the dominant (left) and nondominant (right) sides of the brain.

Figure 2:  Measurement sites and experimental paradigm for human BMI participant with quadriplegia, for left wrist extensions.  (A)  Measurements derived from 6 MEA brain implants (2 each in the primary motor and primary sensory cortices of the left hemisphere; 1 each in the primary motor and sensory cortices of the right), and from EMGs placed on target (here, the left wrist extensor) and surrounding muscles to verify isolation.  (B)  The paradigm entailed contracting the target muscle in isolation to a metronome (period: 4 s).  We monitored EMGs (red traces) to confirm that surrounding muscles were relaxed during metronome ticks (light gray regions).  Adapted from Fig. 1 of Nickl et al. [1].

We designed a metronome-paced experiment to repeatedly measure neural activity induced by a simple task (Fig. 2B). Our criteria for the task---besides that it be represented within the MEAs’ coverage---were that the participant be able to attempt it without fatiguing, and that it be isolatable to [ideally] one muscle, to restrict how it could be accomplished.  Consulting with physical therapists and other clinicians, we chose wrist extensions.  Not only did they satisfy our criteria, but nerve connectivity to both wrist extensor muscle groups (extensor carpi radialii: ECR) was spared for our participant.  This added two advantages: the abilities (1) to do quality control by directly reading muscle activity with electromyography (EMG; see Fig. 2B curves in red), and (2) to study matching movements on both sides of the body.

Below are our key findings from 11 sessions over approximately 200 days.  Although we focus on the left wrist (see main article), results were similar for the right (see supplemental).

Sensorimotor Activity Localized in Hotspots

Each session began with a “roll call” of MEA channels where wrist extensions registered significant activity.  We consolidated our results across all sessions into heat maps (Fig. 3), coloring each channel according to how frequently it was active (percentage of sessions).  Adjacent channels on each array correspond to adjacent brain volumes, so that these maps can be interpreted geographically.

Figure 3:  MEA channel activity footprint for left wrist extensor contractions for arrays implanted in ipsilateral (left column) and contralateral (right column) hemispheres.  The top row shows motor (M) arrays, and the bottom row sensory (S).  Arrays are organized into channels (grid boxes), and each channel is color coded by the percentage of sessions it was active.  Adapted from Figs 1 and 3 of Nickl et al.[1].  Corresponding results from right wrist extensions are in Fig. S3 of the paper supplemental.

The most frequently active channels localized in hotspots, more focal in motor arrays, and more dispersed in sensory.  We considered channels active for at least 1 day (colored cells) to be in the footprint of ECR representation.

Defining Stability

Revisiting our orchestral analogy, if each array grid were an instrumental section, and each footprint a seating chart, how would we evaluate musicians’ consistency?  We considered this across brain areas and hemispheres from several angles (Fig. 4).

Figure 4:  Comparative stabilities in the longitudinal sense (A), within-channel signal attributes (B), and ensemble (C).  The top row presents hemispheric comparisons, and the bottom area comparisons.  Results are shown for the left wrist; i.e. the contralateral and ipsilateral hemispheres refer to the right and left, respectively.  Adapted from Figs. 4 and 5 of Nickl et al. [1]

Longitudinal stability: “Attendance”

We defined channel “lifetimes” as the number of sessions they were active: their “attendance record”.  Channels in contralateral hemisphere to the muscle contraction (the right side of the brain for left wrist extensions) had longer “lifetimes” (higher stability) than the ipsilateral, and channels in sensory areas had longer “lifetimes” than in motor (Fig. 4A).

Within-channel stability: “Musicianship”

MEA channel signals occur as series of electrical “spikes” between neurons, which can be described in terms of their frequency and temporal patterning. 

High firing strength stability denoted that an activated channel fired at similarly rapid or slow rates between measurements (Fig. 4B, left column).  Such channels are like musicians that play consistently in key, at high or low pitches.  High dynamic stability indicated that the time profile of an activated channel’s response was preserved between measurements (Fig. 4B, right column), similar to a musician with a faithful sense of rhythm.  Both stability measures ranged from 0 to 1, higher values indicating more stability.

Consistent with longitudinal stability, within-channel stability in the contralateral hemisphere was greater than in the ipsilateral, and greater in sensory than motor areas.  In addition, strength stability exhibited time effects, carrying higher values when it was assessed within session (“hours”) versus across sessions (“days”).

Ensemble-level stability: “Harmonization”

Finally, we assessed ensemble- (population-) wide stability of regions in terms of how consistently their constituent channels mutually interacted during contractions (their so-called “latent-variable” structure).  In essence, how well does each section of musicians play together as a group despite individual differences?  Extracting latent structure with principal-component analysis (PCA), we found no significant differences between hemispheres or areas (Fig. 4C).

Stability balances out at the population level

In terms of stability, channels in the contralateral side of the brain and in the sensory cortex were the best individual performers.  This makes sense in light of what we understand about these brain areas during arm movement.  Contralateral stability should be higher because sensorimotor representation lies primarily in the contralateral brain.  Higher stability in sensory areas coincides with the importance of reliable sensory coding, but “more flexible” motor coding [11, 12].  However, zooming out to the ensemble scale, we no longer observed sectional differences that were present between individual channels.  Therefore, as BMIs expand to more areas and hemispheres, a more robust approach may be to decode behavior at the level of brain subpopulations during BMI performance [13]: to perform its repertoire not by highlighting individual players, but by weaving and balancing sections into an expressive whole.

References

[1]  Nickl RW, Anaya MA, Thomas TM, Fifer MS, Candrea DN, McMullen DP, Thompson MC, Osborn LE, Anderson WS, Wester BA, Tenore FV, Crone NE, Cantarero GL, Celnik PA.  Characteristics and stability of sensorimotor activity driven by isolated-muscle group activation in a human with tetraplegia.  Sci Rep 12, 10353  (2022).  doi: 10.1038/s41598-022-13436-2

[2]  Handelman DA, Osborn LE, Thomas TM, Badger AR, Thompson M, Nickl RW, Anaya MA, Wormley JM, Cantarero GL, McMullen D, Crone NE, Wester B, Celnik PA, Fifer MS, Tenore, FV.  Shared control of bimanual robotic limbs with a brain-machine interface for self-feeding. Front. Neurorobot.  16:918001140 (2022).  doi: 10.3389/fnbot.2022.918001

[3]  McMullen D, Thomas TM, Fifer MS, Candrea DN, Tenore FV, Nickl RW, Pohlmeyer EA, Coogan C, Osborn L, Schiavi A, Wojtasiewicz T, Gordon C, Cohen AB, Ramsey NF, Schellekens W, Bensmaia S, Cantarero GL, Celnik PA, Wester BA, Crone NE. Novel intraoperative online functional mapping of somatosensory finger representations for targeted stimulating electrode placement: Technical note.  Neurosurg 1 (aop), 1–8 (2021).  doi: 10.3171/2020.9.JNS202675

[4]  Fifer MS, McMullen DP, Osborn LE, Thomas TM, Christie BP, Nickl RW, Candrea DNPohlmeyer EAThompson MCAnaya MASchellekens W, Ramsey NFBensmaia SJAnderson WSWester BACrone NECelnik PACantarero GLTenore FV.  Intracortical Somatosensory Stimulation to Elicit Fingertip Sensations in an Individual With Spinal Cord Injury.  Neurology 98(7), e679-687 (2022).  doi: 10.1212/WNL.0000000000013173 

[5]  Thomas TM, Nickl RW, Thompson MC, Candrea DN, Fifer MS, McMullen DP, Osborn LE, Pohlmeyer EA, Anaya M, Anderson WS, Wester BA, Tenore FV, Cantarero GL, Celnik PA, Crone NE. Simultaneous classification of bilateral hand gestures using bilateral microelectrode recordings in a tetraplegic patient.  medRxiv (2020).  doi: 1101/2020.06.02.20116913

[6]  Hochberg, LR, Serruya MD, Friehs GM, Mukand JA, Saleh M, Caplan AH, Branner A, Chen D, Penn RD, and Donoghue JP. Neuronal Ensemble Control of Prosthetic Devices by a Human with Tetraplegia. Nature 442 (7099): 164–71  (2006).  doi: 1038/nature04970

[7]  Collinger JL , Wodlinger B, Downey JE, Wang W, Tyler-Kabara EC, Weber DJ, McMorland AJC, Velliste M, Boninger ML, Schwartz AB. High-performance neuroprosthetic control by an individual with tetraplegia.  The Lancet  381 (9866): 557–64  (2013).  doi: 10.1016/S0140-6736(12)61816-9

[8]  Klaes C, Shi Y, Kellis S, Minxha J, Revechkis B, Andersen RA.  A Cognitive Neuroprosthetic That Uses Cortical Stimulation for Somatosensory Feedback.   Journal of Neural Engineering 11 (5): 056024  (2014).  doi: 10.1088/1741-2560/11/5/056024.

[9]  Downey JE, Schwed N, Chase SM, Schwartz AB, Collinger JL.  Intracortical recording stability in human brain—computer interface users.  Journal of neural engineering 5(4) 46016 (2018).  doi: 10.1088/1741-2552/aab7a0

 [10]  Ajiboye AB, Willett FR, Young DR, Memberg WD, Murphy BA, Miller JP, Walter BL, Sweet JA, Hoyen HA, Keith MV, Peckham PH, Simeral JD, Donoghue JP, Hochberg LR, Kirsch RF.  “Restoration of Reaching and Grasping Movements through Brain-Controlled Muscle Stimulation in a Person with Tetraplegia: A Proof-of-Concept Demonstration.” The Lancet 389 (10081): 1821–30  (2017).  doi: 10.1016/S0140-6736(17)30601-3

[11]  Makin TR, Bensmaia SJ.  Stability of sensory topographies in adult cortex. Trends Cogn. Sci.21(3), pp.195-204 (2017).  doi: 10.1016/j.tics.2017.01.002

[12]  Todorov E. “Optimality principles in sensorimotor control.” Nat Neurosci7(9), pp.907-915 (2004).  doi:  10.1038/nn1309

[13]  Degenhart AD, Bishop WE, Oby,ER, Tyler-Kabara EC, Chase SM, Batista AP, Yu BM. Stabilization of a brain–computer interface via the alignment of low-dimensional spaces of neural activity. Nat Biomed Eng, 1-14 (2020). doi: 10.1038/s41551-020-0542-9