Neural Sub-Processes of Speech
This is the largest thing I’ve built, and the source of most of my PyTorch and signal-processing experience: the analysis behind a first-author paper on how the brain turns a heard word into a spoken one.
The question
Repeating a word you just heard is a basic bridge between perception and production, but the textbook picture is a single serial pathway: sound in one region, relayed to another for articulation. Behavioral dissociations in aphasia and the spatially scattered, mixed responses seen in intracranial recordings suggest something more parallel. The paper tests whether speech repetition is instead supported by multiple parallel neural sub-processes that run at once — and, because nobody knows in advance which electrodes belong to which sub-process, it needs a method that finds that structure from the data rather than assuming it.
The task and the data
31 patients undergoing intracranial monitoring performed a delayed repetition task with three conditions — listen-and-speak, listen-and-mime, and just-listen — designed to pull perception, a memory delay, and production apart in time. From roughly 1,600 electrodes, sites were first grouped into Auditory, Production, and Sensory-Motor classes by their high-gamma (50–300 Hz) response profiles. The Sensory-Motor group was the messy one: anatomically spread out and spectrotemporally heterogeneous — exactly the signature you’d expect if several distinct processes were overlapping inside it.
The method: tensor decomposition, not clustering
To pull those overlapping processes apart without assuming which channels group together, I used sliceTCA (slice tensor component analysis) — an unsupervised decomposition that extends non-negative matrix factorization into the tensor domain. Where a matrix method flattens the data and can only capture one kind of covariance, sliceTCA keeps the frequency × time × electrode tensor intact and fits a small sum of rank-one “slice” components — each a set of electrode weights times a shared time-frequency pattern — by gradient descent, minimizing reconstruction error. Cross-validation on held-out data picked four components.
Those four components resolved four latent sub-networks inside Sensory-Motor cortex — a Visual network locked to the cue, a Feedback network over auditory cortex, a Motor-transformation network over premotor/parietal cortex, and a Working-Memory network with sustained, strongly left-lateralized activity through the delay. Time-resolved decoders (PCA-LDA) then read word identity and task condition out of each network over the trial. The headline result: only the Working-Memory sub-network held onto which word through the delay and predicted the patient’s reaction time — marking it as the most plausible substrate for holding a word “in mind” before speaking it. The upshot is a reframing of the speech sensory-motor interface as parallel, specialized sub-processes rather than one serial relay.
The engineering
The science rests on three codebases I wrote or extended:
- slicetca — my fork of Arthur Pellegrino’s PyTorch/Lightning sliceTCA library, and where most of my PyTorch work lives. To make it fit iEEG-scale data and converge on these tensors I added L2 and orthogonality regularization, learning-rate decay, and a dynamic-time-warping reconstruction loss, plus a round of memory-footprint and autograd fixes (a transpose that was silently killing gradients, among others). It runs on GPU via CUDA.
- IEEG_Pipelines — my
open-source
ieegpackage (pip install ieeg), which did all the upstream signal processing: multitaper line-noise removal and high-gamma extraction (31 log-spaced bands, Hilbert envelope, baseline z-scoring). It’s cited as the pipeline in the paper’s methods. There’s a fuller writeup here. - SentenceRep_analysis — the analysis itself: functional grouping, the sliceTCA decomposition, the decoders, and the permutation statistics that tie it together.
The scale is what makes it real engineering rather than a notebook: dense recordings across 31 patients, decompositions fit by gradient descent on GPU, and enough cross-validation, permutation testing, and random-seed replication to know the four components are signal and not an artifact of one lucky fit.
Where my PyTorch experience comes from
When my other work here mentions PyTorch, GPU-accelerated tensor methods, or gradient-based optimization, this is the project underneath it — a real model, fit to real data, at a scale that forced the memory and convergence work rather than a tutorial.