research.md

Research-Driven Biometrics Accuracy Enhancement Plan

Executive Summary

This plan transforms the biometrics system from basic vital sign monitoring to a clinically-relevant, research-backed sleep analysis platform. Key innovations: (1) Hybrid posture-adaptive cardiac detection using wavelets and template matching, (2) Clinical actigraphy-based movement taxonomy, (3) Multi-signal apnea/hypopnea screening, (4) Gradient Boosting sleep stage classifier with multi-scale features, (5) Rigorous validation against PSG gold standard using Cohen's Kappa and Bland-Altman analysis.

---

Phase A: Foundational Signal Integrity

A.1 Hybrid Cardiac Peak Detection (biometrics/heart/peakdetection.py)

Replace current fixed-threshold approach with research-validated three-phase algorithm:

Phase 1 - Posture-Informed Template Generation:

• Integrate motion sensor data to determine stable sleep position (supine/prone/left/right lateral)
• Implement unsupervised clustering algorithm to autonomously learn user- and posture-specific IJK complex waveforms
• Training phase: Analyze 30-second stable-posture segments to extract principal BCG morphology
• Store templates: user_supine_template.dat, user_left_lateral_template.dat, etc.
• Dynamically load appropriate template when posture changes detected
• Rationale: BCG morphology varies significantly with posture; adaptive templates eliminate primary source of false positives/negatives

Phase 2 - CWT Candidate Localization:

• Apply Continuous Wavelet Transform using Gaussian derivative ('Gaus2') or B-spline mother wavelets
• Configure with adaptive thresholds based on initial learning phase (per Alvarado-Serrano methodology)
• Output: List of high-probability J-peak candidates with temporal precision
• Rationale: CWT excels at localizing transient high-energy cardiac events even in noise, superior to simple threshold crossing

Phase 3 - Template Matching Validation:

• For each CWT candidate, extract windowed BCG segment centered on peak location
• Compute normalized cross-correlation with active posture-specific template
• Accept candidate only if correlation >0.85 threshold
• Reject morphologically dissimilar artifacts (movement, coughs, etc.)
• Rationale: Morphological validation provides final layer of specificity

Dynamic BPM Ranges: Adjust acceptable ranges by detected sleep stage: Wake (50-100), Light (45-75), Deep (40-60), REM (55-90) bpm

A.2 Multi-Stage Filtration Architecture (biometrics/heart/filtering.py)

Implement research-validated sequential pipeline (DO NOT change order):

Stage 1 - DC Component Removal:

• Filter: 2nd-order Butterworth high-pass, cutoff 0.5 Hz
• Purpose: Remove baseline wander from respiration and slow movements
• Critical: Must precede bandpass to prevent filter instability

Stage 2 - Cardiac Band Isolation:

• Filter: 4th-order Butterworth bandpass, 0.7-15 Hz (refined from plan's 0.5-20 Hz)
• Lower cutoff (0.7 Hz): Separates cardiac from respiratory band (0.1-0.5 Hz)
• Upper cutoff (15 Hz): Retains 99%+ of cardiac energy while rejecting high-frequency noise
• Evidence-based: Literature consensus on optimal BCG passband

Stage 3 - Powerline Interference Removal:

• Filter: Adaptive notch targeting 50/60 Hz with frequency tracking
• Purpose: Eliminate AC electrical noise without static notch limitations

Stage 4 - Wavelet Denoising:

• Transform: Discrete Wavelet Transform (DWT) with Daubechies 'db4' or Symlet 'sym7'
• Method: Soft thresholding on detail coefficients
• Purpose: Suppress non-stationary noise/artifacts while preserving sharp IJK complex morphology
• Critical: Apply to waveform BEFORE peak detection, not after

Stage 5 - Quality-Aware Kalman Filter:

• IMPORTANT: Apply to JJ-interval time series OUTPUT from peak detection, NOT to raw waveform
• Purpose: Smooth beat-to-beat heart rate, reject outliers from missed/false peaks
• Innovation: Dynamic measurement noise covariance $R_k = \alpha / (SQI_k + \epsilon)$
  • High signal quality → small $R_k$ → trust new measurement
  • Low signal quality (movement) → large $R_k$ → rely on state prediction
• Result: Automatic artifact rejection without manual thresholding

A.3 Signal Quality Index (biometrics/heart/quality_metrics.py - NEW)

Implement reference-free SQI for BCG without requiring ECG:

Metrics to compute per 30-second epoch:

• Cross-correlation between consecutive detected heartbeat waveforms (morphology consistency)
• Periodicity score from autocorrelation of peak time series
• Amplitude consistency (coefficient of variation of peak heights)
• Composite SQI: Weighted combination, normalized to 0.0-1.0

Outputs:

• cardiac_sqi: Quality score for heart rate/HRV measurements
• respiratory_sqi: Quality score for breathing measurements (based on signal variance and regularity)
• Store per-epoch in database for quality-aware analytics

Usage:

• Gate quality-aware Kalman filter ($R_k$ calculation)
• Flag low-quality epochs in database (e.g., SQI <0.6)
• Enable downstream analysis to filter/weight data by quality

A.4 Direct Respiratory Analysis (biometrics/vitals/respiratory_analysis.py - NEW)

Extract high-fidelity breathing signal from piezo sensors:

Signal Extraction:

• Bandpass filter: 0.1-0.5 Hz on raw piezo signal (captures 6-30 breaths/min physiological range)
• Peak/trough detection: Identify respiratory cycles, calculate breath-to-breath intervals
• Rate calculation: Instantaneous breathing rate (breaths/min)

Robustness via Cross-Validation:

• Also compute respiratory rate from HRV (Respiratory Sinus Arrhythmia via FFT/Welch)
• Primary: Direct piezo-derived rate (more accurate)
• Fallback: HRV-derived rate when piezo signal quality degrades
• Confidence scoring: High when both methods agree (<10% difference)

Advanced Metrics:

• Respiratory rate variability: Standard deviation of breath-to-breath intervals
• Breathing regularity: Coefficient of variation
• Critical for: REM detection (irregular breathing) and apnea screening (cessation detection)

---

Phase B: Clinical-Grade Kinematics

B.1 Robust Sleep Position Classification (biometrics/motion/position_tracker.py - NEW)

Two-tier implementation for accuracy and granularity:

Tier 1 - Threshold-Based Classifier (Initial Implementation):

• Low-pass filter accelerometer signals (1st-order Butterworth, 0.5 Hz cutoff) to isolate gravitational component
• Identify stable posture windows: std of filtered accel < threshold for 5-10 seconds
• Thresholding rules on mean x,y,z values:
  • Supine (back): mean(z) < -0.9g
  • Prone (front): mean(z) > 0.9g
  • Right lateral: mean(x) > 0.9g
  • Left lateral: mean(x) < -0.9g
• Performance: F1-score ~0.90 for 4-class problem (research-validated)

Tier 2 - ML Classifier (Advanced Enhancement):

• Features: Mean, variance, median of x,y,z axes + calculated pitch/roll/yaw from accel+gyro
• Model: Random Forest or Extra-Trees (95%+ accuracy in literature)
• Training: Requires labeled calibration dataset (user performs known postures)
• Benefit: Can distinguish finer postures (e.g., log vs. fetal lateral positions)

Integration:

• Position fed to cardiac peak detector for template selection (A.1)
• Position logged with snoring/apnea events for correlational analysis
• Position changes counted as sleep fragmentation metric

B.2 Clinical Movement Taxonomy (biometrics/motion/movement_detector.py - NEW)

Implement 3-level hierarchical classification based on clinical actigraphy standards:

Level 1: Micro-Movements (Activity Counts)

• Definition: Minor transient limb/body movements without posture change (analogous to actigraphy counts)
• Detection:
  • Calculate accelerometer vector magnitude: $a_{mag} = \sqrt{a_x^2 + a_y^2 + a_z^2}$
  • High-pass filter to remove gravity, isolate dynamic acceleration
  • Flag events where filtered magnitude >0.01g for 0.5-5 seconds
• Metric: Count per epoch (30s) = restlessness score
• Clinical significance: Associated with brief arousals, periodic limb movements, REM twitches

Level 2: Postural Shifts

• Definition: Major body movement resulting in sleep position change (e.g., supine → left lateral)
• Detection:
  • Identify high-intensity movement period: sustained high $a_{mag}$ and gyroscope angular velocity
  • Verify stable posture before and after movement
  • Confirm angular change ≥30° between pre/post postures
• Metrics: Log transition type (SUPINE→LEFT_LATERAL), duration, intensity, frequency per hour
• Clinical significance: Demarcates stage transitions, indicates sleep fragmentation

Level 3: Bed Exits

• Definition: User physically leaves bed
• Detection (multi-sensor fusion):
  • Prolonged (>30s) chaotic high-intensity signals from accel+gyro (standing/walking pattern)
  • Concurrent complete loss of BCG cardiac signal from piezo sensors
  • Concurrent loss of respiratory signal from piezo
• Metrics: Exit duration, frequency, circadian timing
• Clinical significance: Nocturia, REM behavior disorder screening

Database Storage:

• movement_events table: timestamp, type (micro/shift/exit), intensity, duration, from_position, to_position

B.3 Frank Module UART Interface (biometrics/motion/frank_reader.py - NEW)

Implement hardware interface for motion sensors:

UART Configuration:

• Port: /dev/ttyS1 (per SENSORS.md)
• Baud rate: 38400
• Protocol: LSP (Link Serial Protocol) packet parsing
• Note: Port already open by frankenfirmware process (PID 1201); may require concurrent access strategy or modification to firmware process

Data Streams:

• pillow-accel: Pillow-area accelerometer (3-axis)
• ttc_accel: Touch-to-control accelerometer (3-axis)
• gyro: Gyroscope (3-axis angular velocity)
• Sampling rate: Query from LSP packets (likely 50-100 Hz based on typical IMU rates)

Processing:

• Parse LSP packet structure (refer to decompiled C# code in Eight.Cavia for protocol spec)
• Extract timestamp and synchronize with piezo data timestamps
• Buffer into 30-second epochs aligned with sleep staging windows

Error Handling:

• Monitor sensor heartbeat file: /var/volatile/tmp/sensor.heartbeat
• Detect sensor disconnection if heartbeat stale (>30s unchanged)
• Log sensor availability status

---

Phase C: Expanded Diagnostic Capabilities

C.1 Vibro-Acoustic Snoring Detection (biometrics/snoring/snore_detector.py - NEW)

Leverage piezo sensors for mechanical vibration snoring detection (superior to microphone due to ambient noise immunity):

Signal Processing:

• Bandpass filter: 20-300 Hz on piezo signal (captures primary snoring vibration energy)
• Rationale: Snoring fundamental frequency typically <500 Hz, often 10-100 Hz, with harmonics

Detection Algorithm (STFT-Based):

1. Respiratory Synchronization: Analyze only during expiratory phase (snoring is expiratory phenomenon)

   • Use respiratory signal from respiratory_analysis.py to gate analysis windows

2. Spectrogram Generation: Apply Short-Time Fourier Transform (STFT) to create time-frequency representation

3. Event Criteria:

   • Transient power increase in 20-300 Hz band (>2x baseline)
   • Harmonic structure present (fundamental frequency + integer multiples)
   • Duration: 0.5-5 seconds (typical snore event length)

4. Validation: Cross-reference with current sleep position and absence of other movements (to distinguish from coughs, position changes)

Classification:

• Intensity: Mild (2-5x baseline), Moderate (5-10x), Severe (>10x) based on peak spectral power
• Duration: Total snoring time per hour
• Frequency: Events per hour

Database Storage:

• snoring_events table: timestamp, duration_sec, intensity (mild/moderate/severe), peak_frequency_hz, sleep_position (FK to position_tracker)

C.2 Apnea/Hypopnea Screening (biometrics/snoring/apnea_detector.py - NEW)

NOVEL CAPABILITY: Preliminary screening for Obstructive Sleep Apnea (OSA) using multi-signal fusion

Clinical Definition (AASM):

• Apnea: Complete cessation of breathing ≥10 seconds
• Hypopnea: ≥30% reduction in airflow ≥10 seconds

Three-Step Detection Algorithm:

Step 1: Respiratory Event Detection

• Monitor envelope (amplitude) of filtered respiratory signal (0.1-0.5 Hz) continuously
• Calculate rolling 2-minute baseline amplitude
• Flag respiratory event if:
  • Apnea: Amplitude drops to <10% of baseline for ≥10 seconds
  • Hypopnea: Amplitude drops to <25-50% of baseline for ≥10 seconds

Step 2: Arousal Pattern Validation (30s window post-event)

• Search for corroborating signatures of arousal:
  • Movement Arousal: Micro-movement or postural shift from movement_detector.py
  • Cardiac Arousal: Characteristic cyclical heart rate pattern:
    • During apnea: Heart rate slows (bradycardia)
    • Post-apnea: Sudden acceleration (tachycardia, often +10-20 bpm spike)
    • This cyclical variation is OSA hallmark

Step 3: Event Classification

• High-Confidence Event: Respiratory cessation + movement arousal + cardiac arousal
• Low-Confidence Event: Respiratory cessation alone (could be central apnea, signal dropout, or artifact)
• Calculate Estimated AHI (Apnea-Hypopnea Index): High-confidence events per hour of sleep
  • Mild OSA: 5-15 events/hour
  • Moderate: 15-30 events/hour
  • Severe: >30 events/hour

Database Storage:

• apnea_events table: timestamp, type (apnea/hypopnea), duration_sec, confidence (high/low), sleep_position, arousal_type, hr_change_bpm
• sleep_summary table: Add estimated_ahi column

Clinical Value:

• Not diagnostic but powerful screening tool to identify at-risk individuals
• Correlate with position: "Your AHI is 3x higher in supine vs. lateral positions"
• Measure intervention effectiveness: Did bed angle adjustment terminate apnea event?

C.3 Snore Mitigation Integration (biometrics/snoring/snore_mitigation.py)

Real-time intervention system:

Detection-to-Action Pipeline:

1. snore_detector.py identifies severe snoring event

2. position_tracker.py confirms user in supine position (highest snore/apnea risk)

3. Trigger API call to bed controller: POST /bedframe/setangle

4. Request body (per snoring-research.md):

   {
     "FeedRate": 50,
     "TorsoAngle": 10,  // Gentle 10° incline
     "SnoreMitigation": true,
     "IgnoreMovementSensor": false,
     "Side": "right"  // Or "left" based on which side user on
   }

5. Monitor post-adjustment: Did snoring cease? Did apnea event terminate?

6. Log effectiveness in database

Effectiveness Tracking:

• Before adjustment: Snoring intensity, duration, AHI
• After adjustment: Resolution time, snoring recurrence
• Build personalized intervention profile: "7° incline optimal for this user"

---

Phase D: ML-Based Sleep Stage Classification

D.1 Comprehensive Feature Engineering (biometrics/sleep_stages/feature_extraction.py - NEW)

Multi-scale feature architecture for 30-second epoch classification (AASM standard):

Critical Insight from Literature: Standard 30s epoch too short to capture slow autonomic dynamics. Solution: Multi-scale features combining instantaneous state + contextual autonomic state + temporal transitions.

Feature Set Specification:

Instantaneous Features (30s epoch):

• micro_movement_count: Count of Level-1 movements (strongest wake indicator)
• max_movement_intensity: Peak accelerometer magnitude
• movement_duration_sec: Total time in motion
• is_postural_shift: Binary flag for Level-2 movement
• is_bed_exit: Binary flag for Level-3 movement
• current_sleep_position: Categorical (supine/prone/left/right)

Contextual Features (5-minute window centered on epoch):

• Heart Rate/HRV (Time Domain):
  • hr_mean_5min, hr_std_5min: Mean and variability (Deep: low & stable; REM: high & variable)
  • sdnn_5min: Standard deviation of NN intervals (overall HRV)
  • rmssd_5min: Root mean square of successive differences (parasympathetic tone)
  • pnn50_5min: % of intervals differing >50ms
• Heart Rate/HRV (Frequency Domain):
  • lf_power_5min: Low-frequency power (0.04-0.15 Hz, sympathetic)
  • hf_power_5min: High-frequency power (0.15-0.4 Hz, parasympathetic)
  • lf_hf_ratio_5min: Sympathovagal balance
• Respiration:
  • resp_rate_mean_5min: Average breathing rate (Deep: slow & regular)
  • resp_rate_std_5min: Breathing variability (REM: highly irregular)
  • resp_regularity_5min: Coefficient of variation (low = regular, high = irregular)

Temporal/Lag Features (sequence context):

• lag_hr_mean: Heart rate from previous epoch (t-1)
• lag_movement_count: Movement from previous epoch
• lead_hr_mean: Heart rate from next epoch (t+1)
• lead_movement_count: Movement from next epoch
• Rationale: Provides non-recurrent models with transition information (rare to go Wake→Deep directly)

Signal Quality Metadata:

• cardiac_sqi: From quality_metrics.py
• respiratory_sqi: From quality_metrics.py
• Used to weight or filter epochs in training/inference

Feature Normalization:

• Z-score normalization per user (personalized baselines)
• Store normalization parameters (mean/std) per user in database

Output:

• DataFrame with ~35 features per epoch
• Save as .parquet for efficient storage/loading

D.2 Rule-Based Classifier (biometrics/sleep_stages/rule_based_classifier.py - NEW)

Expert heuristic system for unambiguous epochs:

Purpose: First-pass high-confidence filter; catches clear-cut cases, defers ambiguous cases to ML model.

Rule Set:

Definite Wake:

if (micro_movement_count > 10 per minute) OR is_bed_exit OR (is_postural_shift AND hr_mean > 65):
    return (stage="WAKE", confidence=0.95)

Definite Deep Sleep (N3):

if (hr_mean < 55 AND hr_std < 3 AND micro_movement_count < 0.5 per minute 
    AND resp_regularity < 0.15 AND duration_in_state > 5 minutes):
    return (stage="DEEP", confidence=0.95)

Definite REM:

if (hr_std > 8 AND resp_rate_std > 3 AND micro_movement_count < 1 per minute 
    AND NOT is_postural_shift):
    return (stage="REM", confidence=0.85)

Ambiguous:

else:
    return (stage=None, confidence=0.0)  # Defer to ML model

Output:

• For ~20-30% of epochs: High-confidence prediction
• For ~70-80% of epochs: Defer to ML model

D.3 Gradient Boosting Classifier (biometrics/sleep_stages/ml_classifier.py - NEW)

Primary ML architecture (state-of-the-art for tabular heterogeneous data):

Model Choice: LightGBM (Gradient Boosted Decision Trees)

• Rationale: Consistently achieves 75-85% accuracy on cardiorespiratory+movement sleep staging (literature consensus)
• Advantages: Handles heterogeneous features (continuous + categorical), robust to scaling, built-in feature importance, fast training
• Alternative: XGBoost (similar performance)

Training Protocol:

1. Dataset Requirements:

   • Minimum 10-20 nights of labeled sleep data per user for personalized model
   • OR 100+ nights from diverse users for generalized model
   • Ground truth: PSG or research-grade actigraphy labels

2. Class Balancing: Address class imbalance (N2 typically 50% of sleep, N3 only 15%)

   • Use weighted loss function OR SMOTE oversampling of minority classes

3. Hyperparameter Tuning:

   • Learning rate: 0.01-0.1
   • Num trees: 100-500
   • Max depth: 5-10
   • Min samples per leaf: 50-100 (prevent overfitting on noisy epochs)

4. Cross-Validation: 5-fold time-series cross-validation (respect temporal ordering)

5. Feature Importance: Analyze SHAP values to identify most discriminative features

Model Outputs:

• Stage prediction: WAKE, LIGHT (N1/N2), DEEP (N3), REM
• Confidence scores: Probability distribution over 4 classes

Future Enhancement: LSTM Classifier (Phase 2)

• Architecture: Bidirectional LSTM with 2-3 layers, 64-128 hidden units
• Input: Sequence of 15-30 epochs (~7.5-15 minutes) of feature vectors
• Advantage: Learns temporal dependencies automatically (no need for manual lag features)
• Drawback: Requires much larger training dataset (500+ nights), longer training time, risk of overfitting
• Recommendation: Implement after collecting large validated dataset via initial Gradient Boosting system

D.4 Stage Detection Pipeline (biometrics/sleep_stages/stage_detector.py - NEW)

Integration and post-processing for physiologically plausible hypnogram:

Execution Flow:

1. Feature Extraction: Run feature_extraction.py on all epochs of sleep session

2. Rule-Based Pass: Apply rule_based_classifier.py

   • If confidence >0.90: Accept prediction
   • Else: Pass to ML model

3. ML Prediction: Run ml_classifier.py on deferred epochs

4. Post-Processing: Apply temporal smoothing to remove noise

Temporal Smoothing Methods:

Method 1: Moving Median Filter (Simple)

• Apply 3-5 epoch median filter to prediction sequence
• Effect: Eliminates isolated misclassifications (e.g., single REM epoch surrounded by N2)

Method 2: Hidden Markov Model (Advanced)

• Train HMM with physiological transition probabilities:
  • High probability: N2→N3, N3→N2, N2→REM, REM→N2
  • Low probability: WAKE→N3 (impossible transition), N3→REM (rare)
• HMM finds most likely state sequence given ML model's observations
• Advantage: Enforces biological constraints, produces much cleaner hypnogram

Sleep Statistics Calculation:

• Total Sleep Time (TST): Sum of all non-WAKE epochs
• Time in each stage: WAKE, LIGHT, DEEP, REM minutes and %
• Sleep Efficiency: (TST / Time in Bed) × 100%
• Sleep Latency: Time from lights-out to first 10 minutes of sleep
• REM Latency: Time from sleep onset to first REM
• Number of Awakenings: Count of WAKE episodes
• WASO (Wake After Sleep Onset): Total wake time after first sleep

Database Storage:

• sleep_stages table: timestamp, epoch_index, stage (WAKE/LIGHT/DEEP/REM), confidence, method (rule/ml), sleep_session_id
• sleep_summary table: session_id, date, tst_min, wake_min, light_min, deep_min, rem_min, sleep_efficiency, awakenings, waso_min

---

Phase E: Data Architecture & Validation

E.1 Enhanced Database Schema (biometrics/db.py, server/prisma/schema.prisma)

Quality-Aware Architecture:

Modified Tables:

model Vitals {
  timestamp         Int
  side              String
  heart_rate        Int
  hrv               Int
  breathing_rate    Int
  cardiac_sqi       Float  // NEW: Signal quality 0.0-1.0
  respiratory_sqi   Float  // NEW
}

model SleepStages {
  id                String @id
  sleep_session_id  String
  epoch_index       Int    // 0-based index within session
  timestamp         Int
  stage             String // WAKE/LIGHT/DEEP/REM
  confidence        Float  // NEW: 0.0-1.0
  method            String // NEW: "rule" or "ml"
  cardiac_sqi       Float  // NEW: Quality at time of classification
}

model MovementEvents {
  id                String @id
  timestamp         Int
  type              String // "micro" / "postural_shift" / "bed_exit"
  intensity         Float  // Accelerometer magnitude
  duration_sec      Float
  from_position     String? // For postural shifts
  to_position       String?
  sleep_session_id  String
}

model SnoringEvents {
  id                String @id
  timestamp         Int
  duration_sec      Float
  intensity         String  // "mild" / "moderate" / "severe"
  peak_frequency_hz Float
  sleep_position    String  // FK link to position at event time
  sleep_session_id  String
}

model ApneaEvents { // NEW TABLE
  id                String @id
  timestamp         Int
  type              String  // "apnea" / "hypopnea"
  duration_sec      Float
  confidence        String  // "high" / "low"
  sleep_position    String
  arousal_type      String  // "movement" / "cardiac" / "both"
  hr_change_bpm     Int     // Tachycardia magnitude
  sleep_session_id  String
}

model SleepSummary {
  id                String @id
  date              DateTime
  side              String
  entered_bed_at    DateTime
  left_bed_at       DateTime
  tst_minutes       Int   // Total Sleep Time
  wake_minutes      Int
  light_minutes     Int
  deep_minutes      Int
  rem_minutes       Int
  sleep_efficiency  Float // (TST/TIB) * 100
  sleep_latency_min Int   // Time to first sleep
  rem_latency_min   Int
  awakenings        Int   // Count of wake episodes
  waso_minutes      Int   // Wake After Sleep Onset
  estimated_ahi     Float // NEW: Apnea-Hypopnea Index
  snoring_events    Int   // NEW: Total snore event count
}

Relational Integrity:

• All events link to sleep_session_id for temporal correlation queries
• Enable powerful analytics: "Show me heart rate during REM vs. Deep sleep when SQI >0.8"

E.2 Rigorous Validation Framework

Gold Standard Protocol for Clinical Credibility:

Ground Truth Instrumentation:

Option A - Gold Standard (For Publication/Regulatory):

• Laboratory-grade Polysomnography (PSG) scored by certified technician
• Simultaneous monitoring: PSG + bed system on 20-50 subjects
• Multiple nights per subject (night-to-night variability)
• Expensive but provides indisputable ground truth

Option B - Scalable Development (For Iteration):

• Research-grade actigraphy OR well-validated consumer wearable
• Critical: Must use device with published PSG validation studies
• Warning: Most consumer trackers have poor PSG agreement (especially for staging); unsuitable as primary truth
• Acceptable for heart rate/breathing rate validation

Standard Performance Metrics (Epoch-by-Epoch Comparison):

1. Confusion Matrix:

• Rows: True stage from PSG
• Columns: Predicted stage from bed system
• Reveals specific error patterns (e.g., N2/REM confusion)

2. Per-Class Metrics:

• Precision: Of epochs predicted as stage X, how many were truly X?
• Recall: Of all true stage X epochs, how many were detected?
• F1-Score: Harmonic mean of precision and recall
• Compute for each stage: WAKE, LIGHT, DEEP, REM

3. Cohen's Kappa ($\kappa$) - PRIMARY METRIC:

• Measures agreement corrected for chance
• Interpretation:
  • 0.61-0.80: Substantial Agreement
  • 0.81-1.00: Almost Perfect Agreement
• Target: κ >0.65 for 4-stage classification (research-backed benchmark)

4. Overall Accuracy:

• Total % of correctly classified epochs
• Target: >80% (state-of-the-art for non-EEG cardiorespiratory staging)

Continuous Variables (Bland-Altman Analysis):

For heart rate, HRV, breathing rate, Total Sleep Time, etc.:

• Mean Difference (Bias): Systematic over/under-estimation
• 95% Limits of Agreement: Range containing 95% of differences
• Bland-Altman Plot: X-axis = mean of two methods, Y-axis = difference
• Target Limits of Agreement:
  • Heart Rate: ±5 bpm
  • HRV (SDNN): ±15 ms
  • Breathing Rate: ±2 breaths/min

Validation Milestones:

Milestone 1 (Phase A Complete):

• Validate cardiac and respiratory signals against reference
• Target: HR <5 bpm MAE, RR <2 breaths/min MAE
• Method: Wearable with ECG + respiratory band

Milestone 2 (Phase B Complete):

• Validate movement detection and position classification
• Target: Position F1-score >0.90
• Method: Video recording + manual annotation

Milestone 3 (Phase C Complete):

• Validate snoring detection against audio recording
• Validate apnea events against known OSA subjects' PSG
• Target: Apnea detection sensitivity >80%, specificity >85%

Milestone 4 (Phase D Complete):

• Validate sleep stage classification against PSG
• Target: κ >0.65, Accuracy >80%
• Method: Full PSG validation study (20-50 nights)

Milestone 5 (Continuous):

• Establish normative database from large user population
• Track longitudinal performance metrics
• Iterative model retraining with expanded dataset

---

Implementation Roadmap & Dependencies

Phase A (Weeks 1-4): Foundational Signal Processing

• Priority 1: Multi-stage filtering + SQI (A.2, A.3)
• Priority 2: Hybrid peak detection (A.1)
• Priority 3: Direct respiratory analysis (A.4)
• Validation: Milestone 1 (HR/RR accuracy)

Phase B (Weeks 5-8): Motion Sensor Integration

• Priority 1: Frank UART interface (B.3) - hardware dependency
• Priority 2: Position classifier (B.1)
• Priority 3: Movement taxonomy (B.2)
• Validation: Milestone 2 (position/movement accuracy)
• Critical Dependency: Phase A complete (posture feeds into cardiac detection)

Phase C (Weeks 9-12): Diagnostic Capabilities

• Priority 1: Snoring detection (C.1)
• Priority 2: Apnea screening (C.2) - requires Phase A+B complete
• Priority 3: Mitigation integration (C.3)
• Validation: Milestone 3 (snoring/apnea accuracy)

Phase D (Weeks 13-20): Sleep Stage Classification

• Priority 1: Feature engineering (D.1) - requires Phase A+B+C outputs
• Priority 2: Rule-based classifier (D.2)
• Priority 3: Gradient Boosting classifier (D.3) - requires labeled training data
• Priority 4: Detection pipeline (D.4)
• Validation: Milestone 4 (PSG study)
• Critical Path: Collecting labeled training data is bottleneck

Phase E (Ongoing): Infrastructure

• Database schema (E.1) - implement incrementally with each phase
• Validation framework (E.2) - continuous throughout

---

Expected Performance Targets (Research-Benchmarked)

| Metric | Current | Target | Rationale |

|--------|---------|--------|-----------|

| Heart Rate MAE vs. ECG | ~8-10 bpm | <5 bpm | Achievable with modern BCG in low-motion |

| HRV (SDNN) MAE vs. ECG | ~20 ms | <15 ms | Good agreement for non-contact HRV |

| Breathing Rate MAE | ~3-4 bpm | <2 bpm | Standard target for respiratory sensors |

| Wake vs. Sleep Accuracy | N/A | >90% | State-of-the-art 2-stage classification |

| Wake vs. Sleep Kappa | N/A | >0.70 | Substantial agreement benchmark |

| 4-Stage Accuracy | N/A | >80% | High-end non-EEG performance |

| 4-Stage Kappa | N/A | >0.65 | Substantial agreement for 4-class problem |

| Position Classification F1 | N/A | >0.90 | Research-validated for accel-based |

| Snoring Detection F1 | N/A | >0.85 | Vibro-acoustic has high specificity |

| Apnea Detection Sensitivity | N/A | >80% | Screening tool benchmark (not diagnostic) |

---

Key Technical Parameters Summary

Cardiac Signal:

• Bandpass: 0.7-15 Hz (refined)
• Peak detection: Hybrid CWT + template matching
• Kalman filter: Applied to JJ-interval series with dynamic $R_k$

Respiratory Signal:

• Bandpass: 0.1-0.5 Hz (6-30 breaths/min)
• Method: Direct piezo + HRV cross-validation

Motion Sensors:

• UART: /dev/ttyS1, 38400 baud, LSP protocol
• Sampling: 50-100 Hz (typical IMU)
• Position update: Every 5-10 seconds during stability

Sleep Staging:

• Epoch length: 30 seconds (AASM standard)
• Feature window: 5 minutes for contextual features
• Model: LightGBM with ~35 features
• Post-processing: HMM temporal smoothing

Apnea Screening:

• Detection threshold: ≥10 seconds respiratory reduction
• Validation: 30-second post-event arousal window
• Classification: High-confidence requires 3-signal fusion

---

This research-driven plan transforms the system from basic monitoring to a clinically-relevant platform capable of providing scientifically-valid, actionable sleep health insights. Every algorithm is grounded in peer-reviewed literature, and every performance target is benchmarked against established clinical standards.