SOURCE DOCUMENT (МедЛаб Диагностика, 5 дат):
┌──────────────────┬────────┬─────────┬──────────┬──────────┬──────────┬──────────┬──────────┐
│ Показатель │ Ед. │ Норма │ 15.01.24 │ 28.02.24 │ 10.04.24 │ 22.06.24 │ 05.09.24 │
├──────────────────┼────────┼─────────┼──────────┼──────────┼──────────┼──────────┼──────────┤
│ Гемоглобин (HGB) │ г/л │ 130–160 │ 118 ↓ │ 122 ↓ │ 135 │ 142 │ 138 │
│ Холестерин общий │ ммоль/л│ <5.2 │ 6.8 ↑ │ 6.2 ↑ │ 5.5 ↑ │ 4.9 │ 4.7 │
│ Креатинин │ мкмоль │ 62–115 │ 128 ↑ │ 119 ↑ │ 105 │ 98 │ 92 │
└──────────────────┴────────┴─────────┴──────────┴──────────┴──────────┴──────────┴──────────┘
═══════════════════════════════════════════════════════════════════════════════════════════════
APPROACH │ EXAMPLE OUTPUT │ RESULT
════════════════════════╪═════════════════════════════════════════╪══════════════════════════
│ │
Step 0: Original idea │ Save pages to SQL, numbered by page. │ Starting point.
(Vasya, daily 25.03) │ Agent "spiders" come back to fix │ How to preserve data
Save pages + agents │ dates, ranges, consistency. │ so pipeline can
│ Pole A: workflow | Pole B: agent │ come back to it?
────────────────────────┼─────────────────────────────────────────┼──────────────────────────
│ │
Step 1: Raw text │ Гемоглобин (HGB) г/л 130–160 118 │ ❌ Loses everything
PDF → OCR → text │ 122 135 142 138 Холестерин общий │ Which 118 is which date?
│ ммоль/л <5.2 6.8 6.2 5.5 4.9 4.7 │ No structure at all.
────────────────────────┼─────────────────────────────────────────┼──────────────────────────
│ │
Step 2a: Markdown │ | Показатель | 15.01 | 28.02 | 10.04 | │ ⚠️ Partial
VLM → markdown │ |------------|-------|-------|-------| │ Columns keep dates ✓
│ | HGB | 118 | 122 | 135 | │ But cards/mixed break
│ | Холестерин | 6.8 | 6.2 | 5.5 | │ Notes detached
────────────────────────┼─────────────────────────────────────────┼──────────────────────────
│ │
Step 2b: HTML │ <table><tr><td>HGB</td> │ ⚠️ Very verbose
VLM → HTML │ <td>118</td><td>122</td>... │ 3 params = 500+ chars
│ </tr></table> │ Huge token cost
────────────────────────┼─────────────────────────────────────────┼──────────────────────────
│ │
Step 3: Table objects │ {columns: ["Показатель","15.01", │ ❌ Not universal
Extract tables │ "28.02","10.04"], │ Cards, narratives,
│ rows: [["HGB","118","122","135"]]} │ mixed layouts lost
────────────────────────┼─────────────────────────────────────────┼──────────────────────────
│ │
Step 4a: obs+metadata │ {type:"lab_result", name:"HGB", │ ⚠️ 33% recall!
Two types only │ value:"118", unit:"г/л"} │ Only latest date!
│ {type:"metadata", │ 122,135,142 — lost.
│ key:"patient", val:"Петров"} │ Metadata = catch-all
────────────────────────┼─────────────────────────────────────────┼──────────────────────────
💡 FHIR already formalized what exists in medical documents! fhir-services has 7 types.
────────────────────────┼─────────────────────────────────────────┼──────────────────────────
│ │
✅ Fact-based │ {type: "observation", │ ✅ 92%+ recall
8 FHIR types │ name: "Гемоглобин (HGB)", │ ✅ 98% value accuracy
anyOf strict schema │ value: "118", │
+ date per fact │ unit: "г/л", │ ✓ All 5 dates
+ typed notes │ range: "130–160", │ ✓ 100 observations
+ alt_name │ date: "2024-01-15", │ ✓ Notes attached
Single-pass GPT-5.2 │ category: "laboratory", │ ✓ 6+ languages
│ page: 1} │ ✓ 11 dates/18 pages
│ │ ✓ 308 facts/27 pages
│ {type: "observation", │ ✓ Page-break notes ✓
│ name: "Гемоглобин (HGB)", │
│ value: "122", │
│ date: "2024-02-28", ...} │
│ │
│ {type: "observation", │
│ name: "Холестерин общий", │
│ value: "6.8", │
│ date: "2024-01-15", │
│ notes: [{type:"risk", │
│ text:"<5.2 норма"}]} │
════════════════════════╧═════════════════════════════════════════╧══════════════════════════
🔑 Problem was in PROMPT + SCHEMA, not in PIPELINE.
Old: "extract parameters" → 33% New: "extract facts" + anyOf → 100%
Same model. Single pass. No SQL. No agents.
Last active
March 27, 2026 09:05
-
-
Save theotheo/c4bc74160c15aa31f50012e8a0e0587d to your computer and use it in GitHub Desktop.
BloodGPT: Extraction evolution diagram (draw.io XML)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| <mxGraphModel dx="1400" dy="900" grid="1" gridSize="10" guides="1" tooltips="1" connect="1" arrows="1" fold="1" page="1" pageScale="1" pageWidth="1400" pageHeight="1800"> | |
| <root> | |
| <mxCell id="0"/> | |
| <mxCell id="1" parent="0"/> | |
| <mxCell id="h1" value="<b>Approach</b>" style="text;html=1;align=center;verticalAlign=middle;fillColor=#37474F;strokeColor=none;fontSize=13;fontColor=#FFFFFF;rounded=1;shadow=1;" vertex="1" parent="1"><mxGeometry x="30" y="20" width="220" height="35" as="geometry"/></mxCell> | |
| <mxCell id="h2" value="<b>Example Output</b>" style="text;html=1;align=center;verticalAlign=middle;fillColor=#37474F;strokeColor=none;fontSize=13;fontColor=#FFFFFF;rounded=1;shadow=1;" vertex="1" parent="1"><mxGeometry x="270" y="20" width="380" height="35" as="geometry"/></mxCell> | |
| <mxCell id="h3" value="<b>Result</b>" style="text;html=1;align=center;verticalAlign=middle;fillColor=#37474F;strokeColor=none;fontSize=13;fontColor=#FFFFFF;rounded=1;shadow=1;" vertex="1" parent="1"><mxGeometry x="670" y="20" width="220" height="35" as="geometry"/></mxCell> | |
| <mxCell id="r0a" value="<b>Step 0: Original idea</b><br><font style="font-size:9px">Save pages to SQL (text + images)<br>Agent spiders come back to fix<br>Two poles: workflow vs agent</font>" style="rounded=1;whiteSpace=wrap;html=1;shadow=1;arcSize=15;fillColor=#FFE0B2;strokeColor=#E65100;strokeWidth=1.5;fontSize=11;fontColor=#BF360C;" vertex="1" parent="1"><mxGeometry x="30" y="80" width="220" height="100" as="geometry"/></mxCell> | |
| <mxCell id="r0b" value="<font style="font-size:9px">Two poles discussed:<br><b>A:</b> Strict workflow (steps, cheap)<br><b>B:</b> Agent + tools (flexible, $$$)<br>Need combination of both</font>" style="rounded=1;whiteSpace=wrap;html=1;shadow=0;fillColor=#FFF3E0;strokeColor=#E65100;strokeWidth=1;fontSize=9;fontColor=#555;" vertex="1" parent="1"><mxGeometry x="270" y="80" width="380" height="100" as="geometry"/></mxCell> | |
| <mxCell id="r0c" value="<font style="font-size:9px">Starting point.<br>How to preserve data<br>so pipeline can come back?</font>" style="rounded=1;whiteSpace=wrap;html=1;shadow=0;fillColor=#FFF3E0;strokeColor=#E65100;strokeWidth=1;fontSize=9;fontColor=#BF360C;" vertex="1" parent="1"><mxGeometry x="670" y="80" width="220" height="100" as="geometry"/></mxCell> | |
| <mxCell id="r1a" value="<b>Step 1: Raw text</b><br><font style="font-size:9px">PDF → OCR → plain text</font>" style="rounded=1;whiteSpace=wrap;html=1;shadow=1;arcSize=15;fillColor=#FFCDD2;strokeColor=#C62828;strokeWidth=1.5;fontSize=11;fontColor=#B71C1C;" vertex="1" parent="1"><mxGeometry x="30" y="210" width="220" height="60" as="geometry"/></mxCell> | |
| <mxCell id="r1b" value="<font style="font-size:9px;font-family:Consolas,monospace">Hemoglobin 145 138 g/L 130-160<br>ESR 12 18 mm/hr 2-15<br>Cholesterol 4.9 6.2 mmol/L &lt;5.2<br>Target (ESC): Very high &lt;4.0</font>" style="rounded=1;whiteSpace=wrap;html=1;shadow=0;fillColor=#FAFAFA;strokeColor=#BDBDBD;strokeWidth=1;fontSize=9;fontColor=#333;" vertex="1" parent="1"><mxGeometry x="270" y="200" width="380" height="80" as="geometry"/></mxCell> | |
| <mxCell id="r1c" value="<font color="#C62828">❌ Loses everything</font><br><font style="font-size:9px">Which 145 is which date?<br>No structure, no columns<br>Notes detached from params</font>" style="rounded=1;whiteSpace=wrap;html=1;shadow=0;fillColor=#FFEBEE;strokeColor=#C62828;strokeWidth=1;fontSize=9;fontColor=#555;" vertex="1" parent="1"><mxGeometry x="670" y="200" width="220" height="80" as="geometry"/></mxCell> | |
| <mxCell id="r2a" value="<b>Step 2a: Markdown</b><br><font style="font-size:9px">VLM → markdown text<br>Tables preserved</font>" style="rounded=1;whiteSpace=wrap;html=1;shadow=1;arcSize=15;fillColor=#FFF9C4;strokeColor=#F9A825;strokeWidth=1.5;fontSize=11;fontColor=#E65100;" vertex="1" parent="1"><mxGeometry x="30" y="310" width="220" height="70" as="geometry"/></mxCell> | |
| <mxCell id="r2b" value="<font style="font-size:9px;font-family:Consolas,monospace">| Param | 15.03 | 10.01 | Unit | Range |<br>|------------|-------|-------|--------|---------|<br>| Hemoglobin | 145 | 138 | g/L | 130-160 |<br>| Cholesterol| 4.9 | 6.2 | mmol/L | &lt;5.2 |<br><br>Target (ESC): Very high &lt;4.0, High &lt;5.0</font>" style="rounded=1;whiteSpace=wrap;html=1;shadow=0;fillColor=#FAFAFA;strokeColor=#BDBDBD;strokeWidth=1;fontSize=9;fontColor=#333;" vertex="1" parent="1"><mxGeometry x="270" y="300" width="380" height="105" as="geometry"/></mxCell> | |
| <mxCell id="r2c" value="<font color="#F9A825">⚠️ Partial</font><br><font style="font-size:9px">Columns keep dates ✓<br>Visual position lost<br>Card layouts break<br>Risk tiers detached from Chol</font>" style="rounded=1;whiteSpace=wrap;html=1;shadow=0;fillColor=#FFFDE7;strokeColor=#F9A825;strokeWidth=1;fontSize=9;fontColor=#555;" vertex="1" parent="1"><mxGeometry x="670" y="300" width="220" height="100" as="geometry"/></mxCell> | |
| <mxCell id="r3a" value="<b>Step 2b: HTML</b><br><font style="font-size:9px">VLM → HTML markup</font>" style="rounded=1;whiteSpace=wrap;html=1;shadow=1;arcSize=15;fillColor=#FFF9C4;strokeColor=#F9A825;strokeWidth=1.5;fontSize=11;fontColor=#E65100;" vertex="1" parent="1"><mxGeometry x="30" y="430" width="220" height="55" as="geometry"/></mxCell> | |
| <mxCell id="r3b" value="<font style="font-size:9px;font-family:Consolas,monospace">&lt;table&gt;&lt;tr&gt;&lt;td&gt;HGB&lt;/td&gt;&lt;td&gt;145&lt;/td&gt;...<br>&lt;/tr&gt;&lt;/table&gt;</font>" style="rounded=1;whiteSpace=wrap;html=1;shadow=0;fillColor=#FAFAFA;strokeColor=#BDBDBD;strokeWidth=1;fontSize=9;fontColor=#333;" vertex="1" parent="1"><mxGeometry x="270" y="425" width="380" height="55" as="geometry"/></mxCell> | |
| <mxCell id="r3c" value="<font color="#F9A825">⚠️ Verbose</font><br><font style="font-size:9px">3 params = 500+ chars of tags<br>Huge token cost</font>" style="rounded=1;whiteSpace=wrap;html=1;shadow=0;fillColor=#FFFDE7;strokeColor=#F9A825;strokeWidth=1;fontSize=9;fontColor=#555;" vertex="1" parent="1"><mxGeometry x="670" y="425" width="220" height="65" as="geometry"/></mxCell> | |
| <mxCell id="r4a" value="<b>Step 3: Table objects</b><br><font style="font-size:9px">Extract structured tables</font>" style="rounded=1;whiteSpace=wrap;html=1;shadow=1;arcSize=15;fillColor=#FFCDD2;strokeColor=#C62828;strokeWidth=1.5;fontSize=11;fontColor=#B71C1C;" vertex="1" parent="1"><mxGeometry x="30" y="520" width="220" height="55" as="geometry"/></mxCell> | |
| <mxCell id="r4b" value="<font style="font-size:9px;font-family:Consolas,monospace">{columns:["Param","15.03","10.01"],<br> rows:[["HGB","145","138"], ...]}</font>" style="rounded=1;whiteSpace=wrap;html=1;shadow=0;fillColor=#FAFAFA;strokeColor=#BDBDBD;strokeWidth=1;fontSize=9;fontColor=#333;" vertex="1" parent="1"><mxGeometry x="270" y="515" width="380" height="55" as="geometry"/></mxCell> | |
| <mxCell id="r4c" value="<font color="#C62828">❌ Not universal</font><br><font style="font-size:9px">Cards, narratives, mixed layouts<br>Risk tiers lost (not a row)</font>" style="rounded=1;whiteSpace=wrap;html=1;shadow=0;fillColor=#FFEBEE;strokeColor=#C62828;strokeWidth=1;fontSize=9;fontColor=#555;" vertex="1" parent="1"><mxGeometry x="670" y="515" width="220" height="65" as="geometry"/></mxCell> | |
| <mxCell id="r5a" value="<b>Step 4a: obs + metadata</b><br><font style="font-size:9px">Two fact types only<br>lab_result + metadata</font>" style="rounded=1;whiteSpace=wrap;html=1;shadow=1;arcSize=15;fillColor=#FFF9C4;strokeColor=#F9A825;strokeWidth=1.5;fontSize=11;fontColor=#E65100;" vertex="1" parent="1"><mxGeometry x="30" y="610" width="220" height="70" as="geometry"/></mxCell> | |
| <mxCell id="r5b" value="<font style="font-size:9px;font-family:Consolas,monospace">{type:"lab_result", name:"HGB", value:"145"}<br>{type:"metadata", key:"esc_risk", val:"&lt;4.0"}<br>{type:"metadata", key:"esc_high", val:"&lt;5.0"}</font>" style="rounded=1;whiteSpace=wrap;html=1;shadow=0;fillColor=#FAFAFA;strokeColor=#BDBDBD;strokeWidth=1;fontSize=9;fontColor=#333;" vertex="1" parent="1"><mxGeometry x="270" y="605" width="380" height="70" as="geometry"/></mxCell> | |
| <mxCell id="r5c" value="<font color="#E65100">⚠️ 33% recall</font><br><font style="font-size:9px">Only latest date! 138 lost!<br>Risk tiers = random metadata keys<br>Metadata = catch-all dump</font>" style="rounded=1;whiteSpace=wrap;html=1;shadow=0;fillColor=#FFFDE7;strokeColor=#F9A825;strokeWidth=1;fontSize=9;fontColor=#555;" vertex="1" parent="1"><mxGeometry x="670" y="605" width="220" height="80" as="geometry"/></mxCell> | |
| <mxCell id="r6" value="<font style="font-size:13px">💡 <b>Insight:</b> FHIR already formalized what exists in medical documents! fhir-services has 7 types. No need to invent taxonomy.</font>" style="rounded=1;whiteSpace=wrap;html=1;shadow=1;arcSize=15;fillColor=#BBDEFB;strokeColor=#1565C0;strokeWidth=2;fontSize=11;fontColor=#0D47A1;" vertex="1" parent="1"><mxGeometry x="30" y="720" width="860" height="45" as="geometry"/></mxCell> | |
| <mxCell id="r7a" value="<b>✅ Fact-based extraction</b><br><font style="font-size:9px">8 FHIR types, anyOf strict schema<br>+ date per fact<br>+ typed notes [{type, text}]<br>+ alt_name (bilingual)<br>Single-pass GPT-5.2</font>" style="rounded=1;whiteSpace=wrap;html=1;shadow=1;arcSize=15;fillColor=#C8E6C9;strokeColor=#2E7D32;strokeWidth=2;fontSize=11;fontColor=#1B5E20;" vertex="1" parent="1"><mxGeometry x="30" y="795" width="220" height="130" as="geometry"/></mxCell> | |
| <mxCell id="r7b" value="<font style="font-size:9px;font-family:Consolas,monospace">{type:"observation", name:"HGB",<br> value:"145", <b>date:"2026-03-15"</b>,<br> category:"laboratory", notes:[]}<br><br>{type:"observation", name:"HGB",<br> value:"<b>138</b>", <b>date:"2026-01-10"</b>}<br><br>{type:"observation", name:"Chol",<br> value:"6.2", date:"2026-01-10",<br> <b>notes</b>:[{type:"risk_classification",<br> text:"ESC: VHigh &lt;4.0, High &lt;5.0"}]}</font>" style="rounded=1;whiteSpace=wrap;html=1;shadow=0;fillColor=#F1F8E9;strokeColor=#2E7D32;strokeWidth=1;fontSize=9;fontColor=#333;" vertex="1" parent="1"><mxGeometry x="270" y="795" width="380" height="200" as="geometry"/></mxCell> | |
| <mxCell id="r7c" value="<font color="#2E7D32"><b>✅ 92%+ recall<br>✅ 98% value accuracy</b></font><br><br><font style="font-size:9px">✓ Both dates extracted<br>✓ Risk tiers in notes<br>✓ Atomic facts → FHIR<br>✓ 6+ languages<br>✓ 11 dates / 18 pages<br>✓ 308 facts / 27 pages<br>✓ Page-break notes ✓</font>" style="rounded=1;whiteSpace=wrap;html=1;shadow=0;fillColor=#E8F5E9;strokeColor=#2E7D32;strokeWidth=1.5;fontSize=10;fontColor=#1B5E20;" vertex="1" parent="1"><mxGeometry x="670" y="795" width="220" height="185" as="geometry"/></mxCell> | |
| <mxCell id="r8" value="<font style="font-size:13px">🔑 <b>Problem was in PROMPT + SCHEMA, not in PIPELINE.</b> Old: extract parameters → 33%. New: extract facts + anyOf → 100%. Same model. Single pass.</font>" style="rounded=1;whiteSpace=wrap;html=1;shadow=1;arcSize=15;fillColor=#E1BEE7;strokeColor=#7B1FA2;strokeWidth=2;fontSize=11;fontColor=#4A148C;" vertex="1" parent="1"><mxGeometry x="30" y="1030" width="860" height="45" as="geometry"/></mxCell> | |
| </root> | |
| </mxGraphModel> |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment