AIP-C01 practice question: Troubleshooting prompts and inconsistent responses
Testing, Validation, and Troubleshooting A free question from our original AIP-C01 practice exam. Try it, then open the answer to see why each option is right or wrong.
An HR analytics team asks a model to read job requisition text and return a JSON record for a hiring dashboard. The dashboard's loader expects exact key names such as title, department and openDate. In some replies the model adds a sentence of explanation before the JSON, renames a key, or leaves a trailing comma, and the loader then rejects the record.
The team needs parsing failures below half a percent, the same key names every time, and no changes to the loader. The responses should be as repeatable as possible from run to run.
Which change meets these requirements with the LEAST code change?
Choose one answer.
AA. State the exact keys and value types in the prompt, add two worked examples of requisition text with correct JSON output, and set a low temperature.
BB. Raise the temperature to 1.0 so that the model explores more varied answers, which helps it recover from requisitions with unusual wording.
CC. Keep the prompt as it is, and add a second model call that rewrites each reply into valid JSON, so that the loader always receives a clean record.
DD. Switch to a larger model in the same family, because larger models follow formatting instructions more closely and need no changes to the prompt.
Show the answer and explanation
Answer: A
A. Right: Explicit keys, types and examples steer the model to one format, and a low temperature makes replies more repeatable. The loader needs no change, and the prompt is the only file the team edits.
B. Wrong: A higher temperature makes replies less repeatable, which increases format drift. It works against the same-keys requirement.
C. Wrong: A rewrite call can fix formatting, but it doubles the model cost and latency for every record. The key names still depend on the model's first reply, so the design is not reliable.
D. Wrong: A larger model may follow instructions better, but the team gets no guarantee of exact keys and costs more per record. The prompt would still lack the explicit format the team needs.
Explanation
Format drift is reduced by stating the exact keys and types, showing worked examples, and lowering the temperature so replies repeat. This keeps the loader unchanged and the edit small. A higher temperature, a rewrite pass or a bigger model each add variation, cost or latency without fixing the format.
Want more on Troubleshooting prompts and inconsistent responses? The 5 full-length practice sets have 5 more Troubleshooting prompts and inconsistent responses questions, 375 in all, none of them in the free exam. One payment of $14.99, no subscription. See the practice sets →