About Services Tools Blog Contact us

Photo or Document Classifier

Looks at each image in a mixed import folder and answers with one word, document or product, so handwritten sheets and photographs can be grouped.

  • Visual Question Answering
  • Footwear retail
  • Pipeline
  • Productised
  • Local/private
  • 2025

The problem

A buying drop puts handwritten specification sheets and product photographs in one folder. Until each file is labelled, nobody can read the sheets, and the photographs have nothing to attach to.

What it does

Files are taken in name order, and each image is resized and shown to an image-reading model (LLaVA, run through Ollama) that is told to reply with one word. Documents go to a second model to be read. Product photos wait, and when a sheet has been read the waiting photos are filed with it. A set left incomplete at the end of the folder is reported and left unprocessed.

Outcomes

  • A mixed folder splits into sheets and photographs.
  • Each photograph is filed with the sheet that describes it.
  • An incomplete set is reported instead of being filed without its sheet.

Simplified mockup

Interactive preview with sample data. Nothing is sent to a server.

Import folder

Seven files, name order. Sheets and photographs are mixed. Each image gets one word.

Photograph of a handwritten buying sheet
001.jpgQueued
Side photograph of a chestnut ankle boot
002.jpgQueued
Three-quarter photograph of a chestnut ankle boot
003.jpgQueued
Photograph of a second handwritten buying sheet
004.jpgQueued
Side photograph of a black loafer
005.jpgQueued
Top photograph of a black loafer
006.jpgQueued
Side photograph of a navy lace-up with no sheet
007.jpgQueued
Waiting for classify. Nothing is sent out.

One-word answers

The model may add a stop or a capital. Only letters are kept.

FileRaw replyKeptNext
001.jpgdocument.documentHandwriting reader
002.jpgproductproductWait for the sheet
003.jpgProductproductWait for the sheet
004.jpgdocument!documentHandwriting reader
005.jpgproductproductWait for the sheet
006.jpgproduct.productWait for the sheet
007.jpgproductproductHeld

Incomplete set

007.jpg Not filed
Navy lace-up left in the import folder
Why it stopped
Labelproduct
SheetNone after Sable Row
ActionLeft unprocessed

Add a buying sheet in front of it and run the folder again, or move the file out. It is not attached to Sable Row.

How it works

  1. Take the files in name order and resize each image.
  2. Ask the image-reading model to reply with one word.
  3. Cut the answer down to the single word, document or product.
  4. Hand documents to a second model to be read.
  5. Hold product photos until the sheet they belong to has been read.
  6. Report a trailing photo with no sheet and leave it unprocessed.

Pipeline

flowchart LR
  A["Mixed folder"] --> B["LLaVA"]
  B --> C{"Document?"}
  C -->|Yes| D["Handwriting reader"]
  C -->|No| E["Wait as product"]
  D --> F["Group sheet and photos"]
  E --> F
  F --> G["Flag incomplete set"]