In the pharmaceutical industry, reviewing promotional materials for regulatory approval is a critical process. From advertisements to brochures, every claim made must be backed by scientific evidence and go through a structured review. One major challenge we faced—many of these materials come in the form of PDFs with embedded images, visuals, or non-selectable text. This makes automated claim detection a complex task.

Manual reviews—going page-by-page to find marketing or scientific claims—were slow, inconsistent, and hard to scale. To speed things up and improve accuracy, we explored how AI combined with OCR (Optical Character Recognition) could help us automate large parts of the process.

This wasn’t just about solving a technical problem—it helped us learn how deep tech can be applied to real-world, compliance-driven challenges.

---

### Problem We Faced

Pharma promotional files are highly unstructured and complex. Claims may be hidden in:
- Non-selectable text or scanned images
- Superscripts and footnotes
- Visual charts or image banners
- Surrounding text that holds context or citations

Manual review was time-consuming, inconsistent, and missed edge cases like inherited citations or implicit claims in visuals. We needed an automated, reliable way to extract both claims and their citations—accurately and at scale.

---

### Our Solution

We built an AI-powered pipeline that combined OCR, layout parsing, prompt engineering, and semantic search:
- Parsed PDFs using PyMuPDF and `pdf2image`
- Applied OCR (Tesseract + EasyOCR) on image-based pages
- Used preprocessing (OpenCV, PIL) to improve recognition
- Leveraged Claude Sonnet 3.7 to understand layout and extract claims
- Used OpenAI GPT-4 for intelligent citation mapping
- Embedded claims + context into a vector DB for semantic retrieval
- Exported structured outputs as JSON

This modular approach helped us manage complex layouts, varying formats, and noisy inputs with accuracy and scalability.

---

### Key Learnings from This Project

1. **PDFs in Pharma Are Extremely Complex**  
   Between mixed formatting, superscripts, visuals, and dense footnotes, parsing claims is not just a technical task—it’s a design challenge.

2. **Citation Context Matters**  
   A claim is only useful if it's clearly tied to a reference. Citations aren’t always directly attached—they often come from headers or nearby text. Mapping them required context-aware reasoning.

3. **OCR Alone Isn’t Enough—LLMs Complete the Picture**  
   OCR tools like Tesseract and EasyOCR extracted the text, but Claude Sonnet 3.7 helped us understand the structure and locate meaningful claims.

4. **OpenAI Was Reliable for Citation Mapping**  
   Associating claims with the correct sources required reasoning across paragraphs—something OpenAI models handled well.

5. **Charts and Images Include Critical Info**  
   Claims were often hidden in visuals. These required OCR plus extra logic or manual review.

6. **Vector DB Made Search Smarter**  
   Storing claims and their context in a vector database helped with semantic filtering, deduplication, and audit tracking.

7. **Prompt Engineering Was a Game Changer**  
   How we asked the AI mattered. Prompting with the right structure made results cleaner and more usable.
   
---

### Our Tech Stack

- **PDF parsing**: PyMuPDF for layout-aware text extraction, `pdf2image` for rendering visuals.
- **OCR**: Tesseract and EasyOCR for robust and multilingual OCR.
- **Preprocessing tools**: OpenCV and PIL for cleaning images.
- **LLMs**: Claude Sonnet 3.7 for layout and claim recognition; OpenAI GPT-4 for citation mapping.
- **Vector DB**: For storing embeddings of claims and nearby text for search.
- **Prompt Engineering**: To guide the AI in consistent output formatting.

---

### Generalized Workflow

1. Parse PDF using PyMuPDF.
2. Convert visual-heavy pages to images using pdf2image.
3. Apply OCR via Tesseract/EasyOCR.
4. Clean and deduplicate extracted text using preprocessing scripts.
5. Feed results into Claude to locate and describe claims.
6. Use OpenAI to reason through and attach proper citations.
7. Embed structured data into a vector DB for fast lookup.
8. Export output as structured JSON for downstream systems.

![ChatGPT Image Apr 18, 2025, 04_14_23 PM.png](https://blog.radicalloop.com/uploads/Chat_GPT_Image_Apr_18_2025_04_14_23_PM_288a0faae7.png)

---

### Looking Ahead

This experience opens doors to more applications:
- Cross-document claim clustering
- Training domain-specific claim extractors
- Enabling auditors with semantic dashboards
- Expanding to other regulated fields like finance or legal

---

### Final Thoughts

AI and OCR aren’t magic buttons—they're powerful tools when used right. Success came from combining the right models, a good understanding of the domain, and structured workflows. With this setup, even messy, scanned PDFs can be turned into usable, structured data.

If you're working with complex documents and want to speed up review without missing key info, this approach might be worth trying.




large_pharma-ai.jpg

medium_pharma-ai.jpg

small_pharma-ai.jpg

thumbnail_pharma-ai.jpg

pharma-ai.jpg

Learn how we used OCR, Claude, GPT-4, and vector DBs to automate claim and citation extraction from complex pharma PDFs. Insights, challenges, and results.

What We Learned: Automating Pharma PDF Compliance with OCR & AI

Jimit Shah