01Why this matters
If samples from the same person land in both training and test sets, a model can learn to recognise individuals rather than the disease, and the reported accuracy goes up for the wrong reason.
This project evaluates the same problem with subject-wise splits, so every participant appears in exactly one fold, and measures how much of the earlier signal was leakage.
02Data & features
Keystroke logs and metadata are parsed and cleaned, physiologically implausible timings are filtered out, and only users with at least 2,000 keystrokes are kept. Each remaining user gets 51 features across hold time, latency and flight time:
- Central tendency, plus variability: std, skew, kurtosis, IQR, CV, MAD and burstiness
- Hand asymmetry
- Directional statistics for left/right transitions (LL, LR, RL, RR)
03Try it yourself
Type anything below. It calculates the same kind of timing features the models used, live, from your own typing.
Phone keyboards don't report key timings reliably, so this works best with a physical keyboard.
- Keystrokes
- 0
- Mean hold
- – ms
- Hold variability (SD)
- – ms
- Mean flight
- – ms
- Flight variability (SD)
- – ms
- Mean latency
- – ms
- Hand asymmetry (L − R hold)
- – ms
| Transition | LL | LR | RL | RR |
|---|---|---|---|---|
| Mean latency (ms) | – | – | – | – |
| Count | 0 | 0 | 0 | 0 |
Runs entirely in your browser. Nothing you type is stored or sent anywhere. This shows the features, it is not a diagnostic tool.
Hold time is how long a key stays down. Flight time is the gap between releasing one key and pressing the next. Latency is press to press. Pauses over 2 seconds are ignored, much like the cleaning step filtered out implausible timings.
04Evaluation design
- Nested GroupKFold: 5 outer folds and 3 inner folds, grouped by
user_id, so tuning and testing never share a participant. - Random Forest and XGBoost, each tuned with
GridSearchCVon the inner folds, optimising F1. - A majority-class baseline as the honest reference point.
05Robustness checks
To make sure the negative result wasn't an artefact of one choice, I added three further experiments:
- Feature-group ablation: central-tendency vs variability vs combined feature sets.
- Feature stability: which features are repeatedly selected across folds, then retraining on those subsets.
- SMOTE: oversampling the minority class, to test whether class imbalance explains the result.
06Result
Under leakage-resistant evaluation, neither Random Forest nor XGBoost reliably separates Parkinson's from control above a majority-class baseline.
That's the finding I set out to test. It's a reproducible benchmark showing that earlier results on this dataset don't hold once the same participant is kept out of both sides of the split.
07Reproducibility
The pipeline is seven scripts, each reading the outputs of the one before: