How to identify a fragrance from a photo (and where the model gets it wrong)

So the whole pitch of Maison — the fragrance app that started life as Sillage — rests on one move: you point your camera at a bottle, and it tells you what it is. Claude's vision model does the identifying. It feels like magic the first few times, and then you hit a bottle it gets wrong, and you realize the interesting part of this feature isn't the magic. It's everything I had to build around the places it's confidently wrong.

Let me say the easy part first, because it's genuinely easy. For a well-known fragrance, photographed straight-on, in decent light, with the label readable — the model nails it. Aventus, Sauvage, Bleu de Chanel, the bottles that show up in a thousand photos online. The model has seen those shapes and labels so many times that there's basically no ambiguity. Those cases are solved. If every fragrance were a flagship bottle shot on a clean countertop, this post would end here.

They aren't, and here's where it gets honest.

Flankers are the killer. A "flanker" is a spin-off of a popular scent — same house, same base bottle, a different cap color or a one-word change on the label: Intense, Elixir, Parfum, Le Beau, Absolu. The bottles are designed to look nearly identical to the original, because the whole point commercially is to ride the parent's recognition. Which means the exact visual signal the model leans on — bottle shape, house, overall look — is the signal that can't tell a flanker apart from its parent. The thing that makes identification easy for the flagship is the thing that makes it fail for the flanker. The model will cheerfully return the parent scent with full confidence when you're holding the Intense. It's not being dumb; it's being fooled by a bottle that was built to fool humans too.

Angle and light do more damage than you'd think. The label is where the actual answer lives — the name, the concentration, the house. Catch a bottle at a three-quarter angle with a glare across the glass and the one piece of ground truth is gone, and now the model is identifying by silhouette, which for a plain rectangular bottle is a coin toss across a dozen brands. Dark bottles in dim rooms are the worst. A lot of "wrong" identifications aren't the model failing to know the fragrance — they're the model never getting a clean look at the part that would've told it.

And then there's the long tail. Niche houses, limited editions, discontinued bottles, regional releases — the stuff that fragrance people actually care about collecting, which is exactly the audience for an app like this. The model has seen few or no clean images of those. It won't say "I don't know." It'll pattern-match to the closest famous thing it does know and hand that to you in the same confident voice it uses when it's right. Which is the real problem, the same one I keep running into everywhere: a confident answer and a guess look identical. A vision model doesn't get less sure-sounding when it's reaching. It just reaches.

So the feature isn't "the model identifies the bottle." The feature is "the model proposes, and the design assumes it's wrong often enough to matter." Here's what that actually means in the app.

The identification is never the final word — it's a suggestion the user confirms. The scan returns its best guess, and the user taps to accept or correct it before anything lands in their collection. That one confirm step is doing almost all the real work. It turns a wrong guess from a silent corruption of your collection into a two-second fix, and it means the model only has to be in the right neighborhood, not perfect. Getting you to "Dior Sauvage, is this the EDT or the Elixir?" is a win even though it didn't fully nail it, because now you answer the one question it couldn't.

I'd rather the model narrow than guess. A confident single answer on an ambiguous bottle is worse than a short list — because the confident single answer invites you to tap accept without reading, and now your collection has the wrong concentration in it forever. Where the bottle is genuinely ambiguous (hello, flankers), the honest output is "here are the two or three this could be," and let the human — who is literally holding the bottle and can read the cap — break the tie. The person has information the camera didn't capture. Design for that instead of pretending the photo had everything.

And the collection, not the scan, is the source of truth. Once you've confirmed a bottle, that's a fact you entered, not a guess the model made. The scan is a fast way to enter a fragrance, not the thing your wardrobe is built on. That distinction matters because it means one bad identification doesn't quietly rot the data — you confirmed what's in there, so you can trust it later when you're standing in a store wondering if you already own something.

None of this is a knock on the vision model. It's shockingly good, and for the common case it's the entire feature. But "shockingly good for the common case" and "safe to trust blindly" are different claims, and the gap between them is the flanker you own, photographed at a bad angle, in a dark bathroom. The model will name it with total confidence. The job of the app is to make sure that confidence has to pass through your eyes before it becomes a fact.

That's the whole trick, honestly. The AI does the hard part — turning a photo into a plausible answer in a second. I just never let the plausible answer skip the one person who can actually check it.