Case study · 2019 – 2023
Information Extraction from Receipts
CV + NLP pipeline that extracts and decodes purchased items from scanned receipts. Deployed in multiple countries; 4 patents and a KDD workshop paper.
Multi-country
Production deployment
4
Patents filed
1
Peer-reviewed paper (KDD workshop)
Context
Blackstraw.ai, working with a client's technical team, 2019 – 2023. I was lead developer on the receipt information-extraction system.
Problem
The client needed every purchased item - description, quantity and price - extracted from photos and scans of printed receipts. Real receipts arrive with heavy perspective distortion, creases and widely varied layouts, so text recognition alone doesn't produce structured, usable data.
Approach
The pipeline chains several computer-vision models (text detection, OCR, line detection) with NLP models that structure the result: text classification, named-entity recognition, coreference resolution, dependency parsing and knowledge-graph embeddings for product matching.
A key component is a lightweight Graph Attention Network that groups OCR detections into text lines, which became a published paper.
Models were served with TF Serving and TensorRT, with custom CUDA kernels via PyCUDA.
Results
- Deployed in production across multiple countries
- 4 patents filed and 1 research paper published
Stack
- Python
- C++
- PyCUDA
- PyTorch
- TensorFlow
- DGL
- TF Serving
- TensorRT
He collaborated and communicated well as part of a diverse, international and multidisciplinary team and he participated in key innovations also pushing the boundaries of the state of the art.
Javier Yebes · Director, AI Research Science, NielsenIQ
Have a similar problem?
Tell me what you're working on and I'll tell you honestly whether I can help.