IS-01 · real EEG · completed pilot
Experiments and results
A record of what we have run, what the data supports and which questions remain open.
| Analysis | Balanced accuracy | Macro-F1 |
|---|---|---|
| EEG · inner speech | 26.5% | 0.2637 |
| Majority baseline | 25.0% | 0.1000 |
| EOG only | 28.5% | 0.2801 |
| EMG only | 27.5% | 0.2624 |
| EEG during visual cue | 21.0% | 0.2097 |
What we evaluated
Four imagined Spanish words: arriba, abajo, derecha and izquierda. One Nieto2022 participant, three sessions on the same day and 200 balanced trials. Each session was predicted with a model fitted on the other two. This analyzes public recordings; it is not a newly conducted human study.
What we found
Pooled balanced accuracy was 26.5%, against 25% expected by chance. Macro-F1: 0.2637. The permutation test returned p = 0.46: no evidence of above-chance performance. That p-value tests the unweighted session mean (25.4167%), not the pooled 26.5% score.
What the controls mean
EOG mainly records ocular activity; EMG, muscular activity. Their results are descriptive: differences from EEG were not significance-tested. Slightly higher scores do not establish that artifacts caused the result. Nor does the visual-cue control rule out later attention or visual-memory effects.
Limits that matter
One participant and one day cannot establish performance in new users or days. Three sessions cannot provide a reliable population interval either. Probability calibration was worse than uniform predictions: model confidence should not gate communication. This pilot does not recognize intent or validate a mental password.
Next experiment: across participants
Not yet executed. Extend dataset version/hash locking; verify events and documented corrections for participant 3; hold out all records from one person in each leave-one-subject-out fold. Select models and fit transforms using training participants only, retain EEG/EOG/EMG controls and publish per-subject results. Final sample size and criteria must be fixed before this evaluation is run.
How to reproduce it
The linked implementation fixes dependencies, dataset version, file sizes and hashes, windows, seeds and software tests. The report preserves aggregate metrics and provenance of the executed source. Synthetic tests verify software; they do not add evidence of neural decoding.


