早餐店點完餐，阿姨在小紙條上記了「少糖去冰」，結果把那張便條紙直接塞進袋子裡。打開袋子看到的是備註本身，飲料壓在下面。叫 LLM「先自我檢查再輸出 JSON」，很容易就變成這樣。先講結論：檢查步驟要嘛明講只能在思考區做，要嘛 parser 自己扛得住 JSON 前面多出來的字，最穩是兩層一起做。

症狀長這樣

常見寫法是 prompt 最後要求「Return ONLY valid JSON」，程式端拿到回應直接 JSON.parse(text)，中間沒有任何前處理。平常相安無事，直到 prompt 多加一條「輸出前先做 self-check」。然後 parse 開始一半一半地炸，錯誤訊息很直接：Unexpected token '#', "## Pre-Wri"...。把原始回應拉出來看，開頭是一段 markdown 的自我檢查筆記，後面才接你要的 JSON。

我碰到的那次，加這條指令之前失敗率接近零，加了之後跳到 50%。品管步驟本身變成了失敗來源。

為什麼會這樣

模型對「先做 X 再做 Y」的解讀不是固定的。有時候它把 X 放進 thinking，有時候直接寫進回應本文。開了 extended thinking 也不保證，模型只在自己判斷需要時才走那條路。Prompt 寫「先做 X」，它就可能把 X 當成輸出的一部分。

還有一個容易忽略的推手：prompt 本身用了一堆 markdown 標題。上下文一直在示範 markdown 格式，最後那句「ONLY valid JSON」就輸給了示範效果。

為什麼一開始看不出來

不是每次都失敗。成功的那一半讓問題看起來像偶發抽風，不像 prompt 設計的缺陷；加上 prompt 裡明明寫了只能回 JSON，直覺上會以為有寫就有保障。沒有。

怎麼確認

把失敗跟成功的原始回應並排比：失敗的開頭是 ## Pre-Write Self-Check，成功的開頭是 {。分佈有規律、不是隨機，就可以排除 API 抖動，問題在 prompt 沒有壓住輸出格式。

兩層防法

上游是 prompt。自我檢查那段的標題直接寫死「MANDATORY — perform this INSIDE your thinking/reasoning block only; NEVER include this section text in the JSON response body」，輸出格式那段再加一條 CRITICAL OUTPUT RULE，把後果講清楚：「if ANY markdown appears before the opening {, parsing fails」。

下游是 parser。JSON.parse() 之前先剝前綴：找第一個 { 到最後一個 }，開頭有沒有殘留文字都抓得到真正的 JSON。

// 去掉 JSON 前後的雜訊（markdown 筆記、```json 圍欄），只取第一個 { 到最後一個 }
function extractJson(text) {
  const start = text.indexOf('{');
  const end = text.lastIndexOf('}');
  if (start === -1 || end <= start) {
    throw new Error('model output has no JSON object');
  }
  return JSON.parse(text.slice(start, end + 1));
}

這招有個邊界：如果洩漏出來的筆記裡自己就有 {，切出來的範圍會錯，parse 照樣失敗。所以 parser 防呆只能兜底，不能取代 prompt 那一層；兜不住的時候就重試一次，別讓整批工作跟著掛。

Prompt 那層管掉大多數，parser 兜剩下的，只做一層都不夠。模型不是 deterministic 的，parser 得假設它偶爾不聽話。之後在 prompt 裡加任何「先做 X 再 Y」的指令，順手想一下 X 會不會跑進輸出；品管步驟本身，也可能是失敗的來源。

— 邱柏宇

---

The Self-Check Leaked Into the Answer

Think of ordering breakfast at a street-side shop in Taiwan. The auntie jots your special requests — less sugar, no ice — on a slip of paper, then stuffs that slip straight into your bag. You open the bag and find the note on top of your drink. Ask an LLM to “self-check before returning JSON” and you can get exactly that. Short version: either pin the check to the thinking block explicitly, or make the parser survive extra text before the JSON. The safe answer is both.

What It Looks Like

The usual setup: the prompt ends with “Return ONLY valid JSON”, and the code calls JSON.parse(text) on the raw response with no preprocessing. Fine for a while, until the prompt gains one more line: “perform a self-check before writing.” Then about half the calls blow up with Unexpected token '#', "## Pre-Wri".... Pull the raw response and the first thing in it is a markdown self-check note, with the expected JSON after it.

The time I hit this, failures went from near zero to 50% after that one instruction. The quality step had become the failure source.

Why It Happens

A model’s reading of “first do X, then Y” is not deterministic. Sometimes X lands in the thinking block, sometimes in the response body. Extended thinking is not a guarantee either; the model only routes through it when it decides to. Write “do X first” and X may come out as part of the answer.

There is a quieter push too: the prompt itself was full of markdown headers. The context kept demonstrating a markdown response style, and that outweighed the single “ONLY valid JSON” line.

Why It Wasn’t Obvious

It didn’t fail every time. The successful half made it look like random model variance rather than a prompt design flaw, and the prompt already said JSON only, so it felt covered. It wasn’t.

How to Confirm

Put failed and successful raw responses side by side: failures start with ## Pre-Write Self-Check, successes start with {. A consistent pattern rules out API flakiness and points at the prompt not holding the output format.

Two Layers of Defense

Upstream, the prompt. Rewrite the self-check heading to say it is “MANDATORY — perform this INSIDE your thinking/reasoning block only; NEVER include this section text in the JSON response body”, and add a CRITICAL OUTPUT RULE to the format section spelling out the consequence: “if ANY markdown appears before the opening {, parsing fails”.

Downstream, the parser. Before JSON.parse(), strip the prefix: take the first { through the last }, whatever comes before it.

// Strip noise around the JSON (markdown notes, ```json fences): keep first { to last }
function extractJson(text) {
  const start = text.indexOf('{');
  const end = text.lastIndexOf('}');
  if (start === -1 || end <= start) {
    throw new Error('model output has no JSON object');
  }
  return JSON.parse(text.slice(start, end + 1));
}

One edge: if the leaked note contains a { of its own, the slice is wrong and the parse still fails. So the parser guard is a backstop, not a replacement for the prompt layer; when it can’t recover, retry once rather than letting the whole batch die.

The prompt handles most cases and the parser catches the rest; either one alone is not enough. The model is not deterministic, so the parser has to assume occasional non-compliance. Any “do X then Y” instruction carries the same risk of X ending up in the output. Quality-control steps are not exempt from being failure sources.

— 邱柏宇

延伸閱讀

• 健身房鏡子教的事：台灣遊戲IP在東京也要練出線條
• Canva付出三分之一營收，買到一堂全棧自建課
• 巨型鐵象走過士林，台灣藝術場景已不在展廳裡
• 資金潮起，但不是每艘船都會漲
• 養的狗，正在吃掉蘭嶼最後的龜蛋