Reads a photographed handwritten fit sheet for a footwear style into labelled fields, including which yes or no box was circled.
Document Understanding & OCR
Footwear retail
Pipeline
Productised
Local/private
2025
The problem
Buyers still fill in a paper fit sheet for each style. Retyping the date, materials, fastening, width and circled yes or no boxes holds up the product record. A misread tick is hard to spot afterwards.
What it does
Once an image is recognised as a document, a local Ollama vision model (LLaVA) transcribes it against the blank form. The instructions list the expected lines, so the answer follows the sheet. The text is kept with the original file location, so the product photos that follow can be filed against it.
Outcomes
Paper fit sheets become a field list without retyping.
Yes and no boxes are written out as words, so no tick is ambiguous.
The product photos that follow stay tied to the right sheet.
A failed read stops the batch, so no blank style is filed.
Simplified mockup
Interactive preview with sample data. Nothing is sent to a server.
Handwritten Specification Reader
Intake sheet
Two documents in today’s intake. The blank form is the shape of the answer.
Read
A local vision model follows the printed lines. A circle becomes the word yes or no. A smeared line stops the photographs that follow.
Two sheets are waiting
Sheets filed
0
Harrow Court
Sheets held
0
Width unreadable
Photos labelled
0
of 6 in the batch
HW-1842 Harrow Court filed
Style name
Harrow Court
Colour
Tan
Upper
Full grain leather
Sole
Leather commando
Width
G
Repairable
yes
Circled yes is written as a word. Three photographs are filed on HW-1842.
HW-1847 Mill Lane held
Style name
Mill Lane
Colour
Navy
Upper
Suede
Sole
Rubber cup
Width
unreadable
Repairable
no
Width line is smudged. The three photographs that follow stay unfiled.
Photographs that follow the sheet
HW-1842 Harrow Court 3 filed
Labelled from the clear read. Width G, repairable yes, tan full grain.
LateralSoleSwatch
HW-1847 Mill Lane held
Width line unreadable, so these frames stay in the hold folder and are not filed against a style.
Lateral · heldSole · heldSwatch · held
Read queue
Sheets read this week
Recent sheets
Sheet
Style
Width
Repairable
Photos
Status
HW-1842
Harrow Court
G
yes
3
filed
HW-1847
Mill Lane
unreadable
no
0
held
HW-1838
Priory Derby
F
yes
3
filed
HW-1831
Quay Boot
G
yes
5
filed
HW-1822
Chapel Loafer
E
yes
2
thin read
HW-1816
blank style
0
failed
How it works
Label each image as a document or a product photograph.
Send document images to the local vision model, with the blank form as the shape of the answer.
List the lines in the instructions, from date and name through materials, fastening, width and weight.
Write circled yes or no options out as words.
Keep the text with the file location and use it as the label for the photographs that follow.
Flag a weak or failed read and leave those photographs unfiled.
Pipeline
flowchart LR
A["Sheet photo"] --> B["Confirm document"]
B --> C["LLaVA on Ollama"]
C --> D["Field list"]
D --> E{"Readable?"}
E -->|Yes| F["Label following photos"]
E -->|No| G["Hold and warn"]
Output
Clear fit sheet for Harrow Court, with repairable circled yesFit sheet for Mill Lane, width line smudged out