ChatGPT SearchSep 26, 2026
A healthcare back office can treat inbound faxes as an AI-powered intake and referral pipeline rather than as documents that staff manually open, read, and re-key.
A practical architecture looks like this:
Fax → document ingestion → OCR/layout AI → classification → patient/referral extraction → validation → routing → EHR/work queue → human review → audit trail
1. Automatically capture every inbound fax
Connect the existing fax number or e-fax system to an intake service.
For each fax, capture:
- Sender/fax number
- Date/time
- Number of pages
- Attachments/images
- Cover sheet
- Document ID
- Source location
The important design principle is don't have AI make decisions directly from a raw fax image. First turn the fax into a structured, traceable document.
2. Use AI to understand the document
OCR alone isn't enough because healthcare faxes are messy: handwritten notes, checkboxes, stamps, multi-page referrals, insurance cards, clinical notes, etc.
Use a combination of:
- OCR
- Document layout analysis
- Document classification
- LLM/clinical NLP extraction
The classifier might identify:
- New patient referral
- Existing-patient referral
- Prior authorization
- Clinical records
- Lab results
- Imaging report
- Insurance information
- Prescription/refill request
- Scheduling request
- Administrative correspondence
- Duplicate/unrelated fax
Then extraction can produce something like:
{
"document_type": "specialty_referral",
"patient": {
"name": "Jane Doe",
"dob": "01/14/1982",
"mrn": null
},
"referring_provider": {
"name": "Dr. Smith",
"npi": "...",
"organization": "ABC Medical"
},
"reason_for_referral": "Chronic knee pain",
"requested_specialty": "Orthopedics",
"insurance": {
"payer": "Example Health",
"member_id": "..."
},
"urgency": "routine",
"missing_information": [
"insurance card"
]
}The key is that every extracted field should retain a link back to the page/region of the original fax so a staff member can verify it.
3. Match the referral to the right patient
This is one of the highest-value AI tasks.
The system can compare extracted demographics against the EHR:
Fax
Jane Marie Doe DOB 01/14/1982 Phone ending 4821 EHR candidates
| Candidate | DOB | Name | Phone | Confidence |
|---|---|---|---|---|
| Jane M. Doe | Match | Strong | Match | 99.2% |
| Jane Doe | Match | Moderate | Different | 82% |
Then establish rules such as:
- ≥99% → automatically associate
- 90–99% → staff verification
- <90% → manual matching
Don't let an LLM independently decide patient identity. Make it a probabilistic matching + deterministic rules + human escalation process.
4. Determine what needs to happen next
This is where an AI workflow becomes more valuable than simple OCR.
For example:
New orthopedic referral
→ identify patient → determine specialty → check required referral fields → check whether records are attached → check insurance information → determine whether authorization may be required → create referral work item → route to orthopedic referral queue
Or:
Referral missing insurance card
→ create "Missing Information" task → generate staff-facing checklist → optionally draft fax/request back to referring office
Or:
Urgent clinical information
→ route immediately to designated clinical queue → don't allow the AI to independently diagnose or triage the patient.
5. Create an AI-generated referral summary
Instead of making staff read 15 pages, give them a concise, source-grounded summary:
Referral: Orthopedics
Patient: Jane Doe, DOB 01/14/1982 Referring provider: ABC Medical / Dr. Smith Reason: Chronic right knee pain Relevant records: X-ray report + office note Insurance: Example Health Missing: Insurance card Suggested next workflow: Verify insurance and referral requirements.
Source: Fax pages 1–8 AI confidence: 96% Crucially, the summary should not invent clinical facts. Every important statement should be traceable to the source document.
6. Put humans at the right points
The goal shouldn't be:
"AI processes everything." It should be:
AI processes everything that is predictable; humans handle ambiguity and consequential decisions. For example:
Straight-through
Fax → classify → extract → patient match → validate → EHR/work queue
Human review
Fax → uncertain patient match → staff verifies → continue
Clinical escalation
Fax contains potentially urgent clinical information → designated clinical workflow
Low-confidence extraction
AI can't determine insurance/member ID → highlight the exact page and field → staff enters it.
This can dramatically reduce the amount of reading and typing without removing accountability.
7. Integrate with the EHR
Ideally, the AI system shouldn't become another standalone inbox.
It should write structured information into the organization's existing workflow using available interfaces such as:
- FHIR APIs
- HL7 interfaces
- EHR vendor APIs
- Referral-management interfaces
- Document-management systems
For example:
Inbound Fax
↓
AI Intake
↓
Patient Matching
↓
Referral Object
↓
EHR
┌───────────────┐
│ Referral │
│ Patient │
│ Specialty │
│ Referring MD │
│ Documents │
│ Missing items │
└───────────────┘
↓
Referral Work Queue8. Add a rules engine around the AI
This is important.
Don't put all the business logic inside a prompt.
Use AI for things that require interpretation:
"What kind of document is this?" "What is the referring provider?" "What information appears to be missing?" Use deterministic software for things that should be predictable:
If specialty = cardiology → cardiology queue. If patient match confidence < threshold → human review. If required field missing → incomplete referral. If duplicate document ID → don't create another referral. That combination is substantially safer and easier to audit.
9. Build an audit trail
For healthcare, every automated action should be explainable.
Store:
- Original fax
- OCR output
- Extracted fields
- AI model/version
- Confidence scores
- Rules that fired
- Human corrections
- Final disposition
- Timestamp
- User/system responsible for the action
That lets you answer:
"Why did this referral get routed to this queue?" rather than simply:
"The AI did it."
10. HIPAA/security needs to be designed in
Because these workflows handle PHI, the AI/cloud architecture needs appropriate HIPAA safeguards. HHS says a cloud provider that creates, receives, maintains, or transmits ePHI on behalf of a covered entity is generally a business associate, and a HIPAA-compliant BAA is required when applicable. HHS also emphasizes risk analysis and appropriate safeguards.
That means you should specifically evaluate:
- BAA availability
- Encryption in transit/at rest
- Access controls/RBAC
- Audit logging
- Data retention/deletion
- Model-training/data-use policies
- PHI isolation
- Vendor/subprocessor controls
- Disaster recovery
- Human access to documents
- Prompt/input/output logging
HHS specifically notes that AI services involving PHI can fall within the business-associate framework depending on the service arrangement.
A strong MVP
If I were designing the first version, I wouldn't start with an autonomous referral agent.
I'd build:
Phase 1 — Intake automation
- Receive fax
- OCR
- Classify
- Extract patient/referral/provider/insurance fields
- Match patient
- Display extracted data beside original fax
- Human confirms
- Push to EHR/work queue
Phase 2 — Workflow automation
Add:
- Missing-document detection
- Duplicate detection
- Referral completeness checks
- Automatic routing
- Work queues
- Status tracking
- Staff notifications
Phase 3 — Intelligent operations
Then add:
- Automated requests for missing information
- Referral prioritization based on explicit organizational rules
- Automated status updates
- Referral aging monitoring
- Analytics
- Staff workload forecasting
The biggest opportunity is usually not the LLM itself. It's connecting document understanding + patient matching + business rules + EHR integration + human review into one workflow.
If you're thinking about this as a product/business, I can also map out a specific end-to-end AI referral architecture—including the components, APIs, database schema, EHR integration, human-review UI, and estimated per-fax cost.
4. Determine what needs to happen next
This is where an AI workflow becomes more valuable than simple OCR.
For example:
New orthopedic referral
→ identify patient → determine specialty → check required referral fields → check whether records are attached → check insurance information → determine whether authorization may be required → create referral work item → route to orthopedic referral queue
Or:
Referral missing insurance card
→ create "Missing Information" task → generate staff-facing checklist → optionally draft fax/request back to referring office
Or:
Urgent clinical information
→ route immediately to designated clinical queue → don't allow the AI to independently diagnose or triage the patient.
5. Create an AI-generated referral summary
Instead of making staff read 15 pages, give them a concise, source-grounded summary:
Referral: Orthopedics
Patient: Jane Doe, DOB 01/14/1982 Referring provider: ABC Medical / Dr. Smith Reason: Chronic right knee pain Relevant records: X-ray report + office note Insurance: Example Health Missing: Insurance card Suggested next workflow: Verify insurance and referral requirements.
Source: Fax pages 1–8 AI confidence: 96% Crucially, the summary should not invent clinical facts. Every important statement should be traceable to the source document.
6. Put humans at the right points
The goal shouldn't be:
"AI processes everything." It should be:
AI processes everything that is predictable; humans handle ambiguity and consequential decisions. For example:
Straight-through
Fax → classify → extract → patient match → validate → EHR/work queue
Human review
Fax → uncertain patient match → staff verifies → continue
Clinical escalation
Fax contains potentially urgent clinical information → designated clinical workflow
Low-confidence extraction
AI can't determine insurance/member ID → highlight the exact page and field → staff enters it.
This can dramatically reduce the amount of reading and typing without removing accountability.
7. Integrate with the EHR
Ideally, the AI system shouldn't become another standalone inbox.
It should write structured information into the organization's existing workflow using available interfaces such as:
- FHIR APIs
- HL7 interfaces
- EHR vendor APIs
- Referral-management interfaces
- Document-management systems
For example:
Inbound Fax
↓
AI Intake
↓
Patient Matching
↓
Referral Object
↓
EHR
┌───────────────┐
│ Referral │
│ Patient │
│ Specialty │
│ Referring MD │
│ Documents │
│ Missing items │
└───────────────┘
↓
Referral Work Queue8. Add a rules engine around the AI
This is important.
Don't put all the business logic inside a prompt.
Use AI for things that require interpretation:
"What kind of document is this?" "What is the referring provider?" "What information appears to be missing?" Use deterministic software for things that should be predictable:
If specialty = cardiology → cardiology queue. If patient match confidence < threshold → human review. If required field missing → incomplete referral. If duplicate document ID → don't create another referral. That combination is substantially safer and easier to audit.
9. Build an audit trail
For healthcare, every automated action should be explainable.
Store:
- Original fax
- OCR output
- Extracted fields
- AI model/version
- Confidence scores
- Rules that fired
- Human corrections
- Final disposition
- Timestamp
- User/system responsible for the action
That lets you answer:
"Why did this referral get routed to this queue?" rather than simply:
"The AI did it."
10. HIPAA/security needs to be designed in
Because these workflows handle PHI, the AI/cloud architecture needs appropriate HIPAA safeguards. HHS says a cloud provider that creates, receives, maintains, or transmits ePHI on behalf of a covered entity is generally a business associate, and a HIPAA-compliant BAA is required when applicable. HHS also emphasizes risk analysis and appropriate safeguards.
That means you should specifically evaluate:
