// AI Platform Drift
Cloudflare AI Search adds CJK keyword search via trigram tokenizer
Cloudflare developer changelog · published 2026-10-16 11:28 UTC
WHAT CHANGEDCloudflare AI Search now supports Japanese, Chinese, and Korean keyword queries when using the trigram tokenizer. Previously, queries without spaces in these languages were treated as single exact phrases and returned no results. The porter tokenizer does not support CJK languages. Changing the tokenizer setting triggers a full reindex.
WHY IT MATTERSIf you index CJK content with the default porter tokenizer, keyword search does not work for those languages. Switching to trigram fixes search but forces a complete reindex of your data, which affects availability and costs during the rebuild.
WHAT TO DOCheck if your AI Search indexes contain Japanese, Chinese, or Korean content. If so, set keyword_tokenizer to trigram in your index configuration and schedule the reindex during a maintenance window to avoid disrupting live search traffic.
Model-reported confidence 85% (not measured accuracy)
Primary source ↗
written by anthropic/claude-sonnet-4.5
Show the reasoning — raw diff, triage verdict, confidence
RAW DIFF
+## AI Search - AI Search keyword search supports Japanese, Chinese, and Korean queries +https://developers.cloudflare.com/changelog/post/2026-10-16-cjk-keyword-search/ +Fri, 16 Oct 2026 00:00:00 GMT + +AI Search keyword search with the trigram tokenizer now matches Japanese, Chinese, and Korean queries, including full natural-language questions such as 経費精算の締め日はいつですか?. Previously, a query without spaces was treated as a single exact phrase and usually returned no keyword results. +The porter tokenizer does not work well for these languages. If your content is in Japanese, Chinese, or Korean, set keyword_tokenizer to trigram. Changing the tokenizer triggers a full reindex. +Refer to Keyword search for tokenizer details.
HOW IT WAS JUDGED
Triage signals below are model-reported assertions, preserved verbatim; they are not independent arithmetic verification.
| triage verdict | MATERIAL at 0.92, needed 0.5 |
|---|---|
| triage reasoning | New language support and tokenizer configuration requirement affects query behavior and requires operational action. |
| signals | new feature capability added to AI Search · language support expansion (Japanese, Chinese, Korean) · breaking configuration change: tokenizer selection affects indexing · operational impact: 'full reindex' required on tokenizer change · future date: 16 Oct 2026 |
| suppression | — |
| triage model | anthropic/claude-haiku-4.5 ($0.00000) |
| writer model | anthropic/claude-sonnet-4.5 ($0.00000) |