Search that survives Hungarian

Hungarian is agglutinative. A single noun can carry case, number, and possession as suffixes, so the word an adopter types is frequently not the word in our database, and neither is a prefix of the other. Add nine accented vowels that people routinely omit on a phone keyboard, and naive `ILIKE` search fails constantly.

The four failures, in order of frequency

• Missing accents: "kutya nemet juhasz" should match "németjuhász".
• Suffixes: "kutyát", "kutyák", "kutyával" should all match "kutya".
• Compounds: "németjuhász" is one word, but people type "német juhász".
• Digraphs: "cs", "sz", "zs", "gy" are single letters and must not be split by a fuzzy matcher that thinks in ASCII characters.

What we run now

Postgres full-text search with the Hungarian configuration, plus `unaccent` in the dictionary chain so that folding happens before stemming rather than after. The order matters and it is easy to get backwards.

CREATE TEXT SEARCH CONFIGURATION hu_unaccent ( COPY = hungarian );

ALTER TEXT SEARCH CONFIGURATION hu_unaccent
  ALTER MAPPING FOR hword, hword_part, word
  WITH unaccent, hungarian_stem;

-- generated column, indexed with GIN
ALTER TABLE animals ADD COLUMN search tsvector
  GENERATED ALWAYS AS (
    setweight(to_tsvector('hu_unaccent', coalesce(name,'')),  'A') ||
    setweight(to_tsvector('hu_unaccent', coalesce(breed,'')), 'B') ||
    setweight(to_tsvector('hu_unaccent', coalesce(notes,'')), 'D')
  ) STORED;

The Hungarian Snowball stemmer handles the common suffix cases well — better than we expected — and the weighting keeps a breed match above a passing mention in a free-text note.

Compounds, and the idea we abandoned

We spent three weeks on a compound splitter: a dictionary-driven decomposition that would let "német juhász" find "németjuhász" and vice versa. It worked on breed names and produced nonsense on everything else, because Hungarian compounds are productive and our dictionary was not.

The splitter was right about 91% of the time. The 9% were funny in a demo and unacceptable in a search box a person uses once and judges forever.

We deleted it and added a trigram index instead, queried in parallel with the tsvector and merged by score. Trigrams do not understand morphology, but they do not need to: "nemetjuhasz" and "nemet juhasz" share almost all their trigrams. It is a dumber tool that fails in gentler ways.

CREATE INDEX animals_trgm
  ON animals USING gin (unaccent(breed || ' ' || name) gin_trgm_ops);

Digraphs and fuzzy matching

Levenshtein distance over UTF-8 code points treats "sz" as two edits away from "s", which makes "Szeged" and "Seged" look further apart than they are to a Hungarian reader, while "kosz" and "kos" look adjacent when they are unrelated words. We normalise digraphs to single private-use code points before computing distance, and map back afterwards. It is ugly and it fixed a whole class of bad suggestions.

Median search latency is 34ms across 41,000 historical listings. The part that took the longest was not performance; it was accepting that the linguistically correct approach lost to the crude one on real queries.