# Liquid State XR

The application runs on the viewing device. GitHub Pages or another static host serves the files; no microphone samples or audio files are uploaded. GPU rendering is always active. Optional WebGPU simulation must pass both a numerical parity check and a live timing check; slower devices retain the sparse CPU solver. It also falls back to CPU if GPU processing later exceeds the audio budget.

## Recovered research

Model: `Replicate/res/spokenDigitsTI/runTI46_3/runTI46_test23.mat`, fixed fold 1, saved epoch 200. The inference export contains PARAM1 timing/ear settings, PARAM2.G, PARAM2.Gin, PARAM2.E, and RESULT(1).W(:,:,200). It contains no dataset or speaker records.

- 77 cochlear channels at 12 kHz; 1 kHz encoder output.
- 125 leaky integrate-and-fire neurons; 1,150 directed recurrent synapses and 308 input synapses.
- 10 spiking readout neurons. Counts are output spikes, not softmax probabilities.
- Double exponential synapses: excitatory 8/4 ms; inhibitory 4/2 ms. Membrane time constant 32 ms, threshold 20, refractory period 2 ms, fixed Euler step 1 ms. Recurrent propagation uses the preceding step's synaptic current.

Research: [Predicting Performance using Approximate State Space Model for Liquid State Machines — Gorad, Saraswat, Ganguly](https://arxiv.org/abs/1901.06240); [Hardware-Friendly Synaptic Orders and Timescales in Liquid State Machines for Speech Classification](https://arxiv.org/abs/2104.14264).

## Audio encoding

Streaming formulation of `DesignLyonFilters.m`, `LyonPassiveEar.m`, `soscascade.c`, `agc.c`, `sosfilters.c`, `BSA.m` and `BSA_filter.m` from the supplied research directories. The Lyon equations follow Malcolm Slaney's [Auditory Toolbox, Version 2, Interval Research TR 1998-010](https://engineering.purdue.edu/~malcolm/interval/1998-010/). Original Auditory Toolbox copyright: (c) 1998 Interval Research Corporation. BSA implementation reference: Ajinkya Gorad's supplied MATLAB research code.

The pipeline is anti-aliased resampling → 77-channel cascade → half-wave rectification → four spatial AGC stages → channel difference → two-pole decimation → clipping at 4e-4 → BSA matching pursuit. The trained ear uses Q=8, step=.25, decimation=12, and tauf=32. The 33-tap BSA kernel is normalized, with tau=4 ms, tau2=1 ms and threshold=.955. BSA requires 32 ms lookahead. The ear additionally smooths the signal with its trained lowpass; capture and GPU readback add device-dependent latency.

The compiled C/WebAssembly implementation is compared with the JS reference, and the JS ear is compared with an independent SciPy implementation. Both preemphasis channels are consistently zeroed at every sample, correcting the scalar-indexing artifact in the supplied MATLAB wrapper. This change and live speech segmentation prevent claiming exact end-to-end equivalence with historic TI-46 evaluation scores.

## Organic structure

Visual positions are relaxed with graph springs, repulsion and collision avoidance. The model's saved grid positions, IDs, matrices and weights remain untouched. Each source gets one shared proximal axon directed toward the mean of its targets. Curved branches preserve every individual directed connection and converge on short dendritic ends. This is a schematic graph layout, not measured biological morphology.

Synaptic pulses use actual source spike timestamps and travel for 140 ms for visibility. That visible travel time is not the neural solver's 1 ms propagation delay. Repeated rapid spikes can overlap visually; the raster retains individual events. The membrane trace samples the simulation at display-report rate, not every 1 ms step. Membrane color maps rest to blue and threshold to amber.

## Live classifier limits

The trained weights are recovered, but microphone-domain digit recognition is experimental. The live pipeline segments sound at an amplitude gate followed by 500 ms silence (2.5 s maximum utterance) and resets the readout at onset. The original training setting RESET_VAR_PER_SAMPLE was 0. Continuous reservoir state and room acoustics also differ from isolated training samples. No historical test score is presented as browser accuracy. Silence and tied/all-zero readout counts produce no candidate. Synthetic tones never produce a claimed recognized digit. Changing recurrent strength disables the trained-model interpretation.

## Rendering and device support

[Three.js WebXRManager](https://threejs.org/docs/pages/WebXRManager.html), vendored under the Three.js MIT license. WebGL2 instancing, shared geometry, one neuron-state texture and GPU pulse shaders. Quest requests [immersive-ar passthrough with a transparent background](https://developers.meta.com/horizon/documentation/web/webxr-mixed-reality/). No camera pixels are read. Optional WebGPU computes the reservoir in a worker and must pass a 128-step CPU comparison with zero spike differences and maximum voltage error below .01 before use. WebGPU renderer/XR binding is not required.

Physical Quest 3 comfort, microphone permission behavior, hand/controller tracking and sustained thermal performance require an on-device test. Desktop capability checks and numerical tests do not establish a headset frame-rate guarantee.

## Cell histories and personal learning

Every neuron and input channel has a GPU billboard raster; the ten output neurons have wider, taller strips. These show 256 bins of 8 ms (2.048 seconds), populated from actual spike timestamps. Binning affects display only. The rounded, bent graph layout, variable soma shapes, thick shared axon trunks and curved branches change drawing geometry only. No grid or bounding cage is rendered.

Personal training uses recursive ridge regression with regularization 0.5, a 0.15 bias feature and L2-normalized square roots of 125 utterance spike counts. Each labelled microphone sample updates only this separate 10-class linear readout. At least two labels and sufficient neural activity are required for a guess. Guesses only select taught labels; scores are not calibrated probabilities. Training restores the CPU solver, gain 1 and recurrent strength 1. Personal predictions are disabled under different settings. Samples shorter than 120 ms of non-trailing-silence duration are rejected. Segmentation uses the existing 500 ms silence detector and 2.5-second cap.

Only reservoir feature vectors and labels are persisted, never raw recordings. Profiles carry a SHA-256 fingerprint of the full exported model plus encoder and feature-version identifiers. A mismatching profile is rejected. The original TI-46 spiking readout remains separate and unchanged. Few-shot accuracy on Quest microphones has not been measured.