Under 1 GB CPU model vs. GPT-4o at tariff classification
A product title, a description, a photo, or any mix of them goes in. A 10-digit US Harmonized Tariff Schedule (HTS) code comes out, with a short customs description, a confidence, and the past customs rulings that back it up.
It runs on a CPU in under 1 GB of memory, with no GPU and no LLM call. A text query takes well under a tenth of a second. On the HSCodeComp benchmark it picks the exact 10-digit code for 34% of products. GPT-4o gets 18.5%.
The problem
Every product that crosses the US border needs a 10-digit HTS code, and the code sets the duty rate, so a wrong one costs money or draws penalties. The schedule is a legal tree with about 20,000 ten-digit codes at the bottom:
| digits | level | a men's cotton knit T-shirt |
|---|---|---|
| 2 | chapter | 61: knitted or crocheted apparel |
| 4 | heading | 6109: T-shirts, singlets, tank tops |
| 6 | subheading (worldwide) | 6109.10: of cotton |
| 8 and 10 | US duty rate and statistical breakdown | 6109.10.00.xx: by gender and garment type |
Three things make it hard:
- The text is legal, not descriptive. A code can read just "Other: Of man-made fibers". It only means something through the codes above it and what the codes beside it exclude.
- "Other" is defined by exclusion. A product lands there because it matches none of the named codes beside it.
- The deciding facts are often missing. "Cozy crew neck sweater" doesn't state the fiber, whether it's knit, or who it's for.
General-purpose LLMs know the first few digits well. The last four are US-specific bookkeeping that rarely shows up in web text, and that is where they fall apart.
Results against published benchmarks
Exact match on the top answer, in percent. A 10-digit answer counts only if all 10 digits are right.
HSCodeComp (632 real e-commerce products, paper, Table 1)
| system | runs on | 10-digit | 6-digit |
|---|---|---|---|
| This model, text | CPU, under 1 GB | 34.2 | 50.3 |
| GPT-5, text + image | API | 29.3 | 60.0 |
| Gemini 2.5 Pro, text + image | API | 24.2 | 59.0 |
| GPT-4o, text + image | API | 18.5 | 48.1 |
| GPT-4o, text | API | 18.4 | 45.7 |
| Claude Sonnet 4, text | API | 11.9 | 44.9 |
| Best agent: GPT-5 with web search | API + search | 46.8 | 62.4 |
ATLAS (200 questions written from US Customs (CBP) rulings, paper, Tables 2 and 3)
| system | runs on | 10-digit | 6-digit |
|---|---|---|---|
| This model, text | CPU, under 1 GB | 33.5 | 54.0 |
| GPT-5-Thinking | API | 25.0 | 55.5 |
| Gemini 2.5 Pro Thinking | API | 13.5 | 31.0 |
| DeepSeek-R1 | API | 2.5 | 26.5 |
| Atlas: LLaMA-3.3-70B fine-tuned on ATLAS | GPUs | 40.0 | 57.5 |
How to read it:
- At 10 digits it beats every general-purpose LLM in both papers, including GPT-5. It trails a 70B model fine-tuned on ATLAS's own training data and a GPT-5 agent that searches the web, both far larger and far costlier to run.
- The gap is in the last four digits. Those are US-only, and this model learned them from US customs rulings.
- Text only. The public HSCodeComp release has no images, so this model saw only the text while the "text + image" rows also saw the photo.
- No leakage. Anything in training that matches a benchmark item, exactly or as a near-duplicate with the same 6-digit code, was removed before training.
On test sets of my own, none of it used in training:
| set | input | first answer right |
|---|---|---|
| CBP rulings from 2024 onward (never seen in training) | full product description | 41.5% at 10 digits |
| 500 real product pages | title and page text | 78%, to as many digits as two independent labellers agreed on |
| Amazon Berkeley Objects, 725 product photos | photo + title | 89%, first 4 digits |
| 424 product pages with photos | photo + title | 79%, to the agreed digits |
How it works
Search, not a 20,000-way guess. The input is turned into a vector (a list of numbers that captures its meaning) and compared against two lists: every tariff code's own description, and about 200,000 past customs rulings. How close it is to each code's text and which codes the closest rulings used are combined into one score. Searching this way works for codes with few examples, shows its evidence for free, and new rulings can be added without retraining.
Training, in big steps:
- Collect labels customs already wrote. Published rulings from the US, EU, UK, Canada and Australia: an importer describes a product, customs answers with a code. US rulings are the only source of the last four digits; the others teach the first six, which are shared worldwide. No product was hand-labelled for this project.
- Clean and split by time. Duplicates are removed, rulings up to 2022 train the model, 2023 picks between options, and 2024 onward is the test. Anything overlapping a benchmark is dropped.
- Speak like a shopper. An open-source LLM rewrites ruling text into the way people title products online, so the model sees what users actually type.
- Fine-tune the text model. The text half of CLIP (an open image-and-text model) is trained to place each product next to its tariff code and away from look-alike codes, then trained again on the near misses it still gets wrong.
- Bring photos along. The image half of CLIP is left as is, and a small add-on model maps photos into the same space as the tuned text model.
- Re-rank. A small decision-tree model (LightGBM) re-orders the top 30 candidates using extra evidence: does the input name a material the code excludes, do the rulings and the code text agree?
- Shrink and ship. Both models and the search lists are stored as 8-bit numbers, about a quarter of the original size, and published as one bundle the site downloads at startup.
At serving time it also checks facts the text states plainly (men's, polyester, knit) against each code's place in the tree, and reports how many digits it is confident in. A vague title like "cozy crew neck sweater" gets the right heading with low confidence on the last digits, rather than a confident wrong 10-digit code.
Keeping the numbers honest
A small model beating GPT-4o is the kind of claim that usually turns out to be a leak: test answers that slipped into the training data. The guards:
- Split by time, not at random. Customs issues near-identical rulings for the same importer in the same year, so a random split would test on copies of training examples.
- Pick with one set, report on another. Every design choice was made on 2023 rulings and the product-page set. The benchmarks and the 2024+ rulings were only scored to report results, once per release, never to pick between options.
- Strict scoring. An answer counts only if it matches the correct code in full: the one customs assigned in the ruling, or the one the benchmark lists as the answer.
- Score what ships. The numbers come from the same 8-bit models the live site runs.
Use it from an agent
The model is also an MCP server (the standard way AI agents call outside tools), so an agent can
call it directly (classify_hts: text and/or an image URL in, ranked codes with evidence out):
claude mcp add --transport http hts https://www.calebdame.com/projects/hts/mcp
It is rate limited per client (20 calls a minute, 300 a day) to keep the shared CPU responsive.
What would be interesting to try with a larger budget
- A cross-encoder on a GPU. This is a model that reads the product and a candidate code together, so it picks up the small qualifiers ("of cotton", "for men") that separate neighboring codes. It needs one pass per candidate, which was too slow for this CPU, but a GPU could handle the top 30 easily.
- Pair this model with an LLM. Let this model cut 20,000 codes down to ten, with the rulings behind each, and let the LLM choose. The best published result is a GPT-5 agent with web search at 46.8%. I'd expect a short list backed by real rulings to be a cheaper and better starting point than a web search.
- Buy clean labelled data. Accuracy goes from 24% on codes with 1–5 training examples to 73% on codes with over 500, so the quickest gain is more examples of the rare codes, from brokers or licensed import records.
- Turn the legal notes into rules. Every section and chapter has notes on what it excludes. An LLM could pull those into a knowledge graph, and the model could drop candidates that break them.
- Read the text in the photo. Run an OCR step to pick up care labels, ingredient panels and spec plates, which hold the facts that set the last four digits. The training photos were mostly clean product shots with little text on them, so it was not useful here.
- Weigh mistakes by what they cost. A wrong code with the same duty rate costs the importer almost nothing; one with a different rate costs real money. With shipment values, the model could be trained to minimize duty error in dollars instead of just counting exact matches.
Not legal advice: classification depends on facts a photo or title may not show. Confirm with the HTS and a licensed customs broker.