FROM THE AI FRONTIER
Frontline news — November 20: check AI without hints
A recent Google DeepMind talk recording: why test answers should be kept separate. A practice example for an ordinary chat.

Check your assistant on separate examples whose answers you have not shown it in advance. Today I examine one recent AI Engineer talk: it offers an idea for checking without hints. My answer sheet can stay in the envelope for now.
Video: leave the answers with the reviewer Original: Post-Training Gemini 4 for Real Engineering Work — Melvin Johnson, Google DeepMind. - Channel: AI Engineer. Language: English. - Standalone recording published: November 19, 2026, at 01:30:09 Moscow time. YouTube gives November 18, 14:30:09, UTC−8. - Duration: 17:50 according to YouTube metadata.
This is a recording of a November 12 CODE Summit session, not a product launch today. In the official transcript, Johnson describes held-back test tasks revealed after a solution is submitted. He also warns about resubmitting a solution after seeing the test. This describes the team's approach to training models, rather than proving Gemini is error-free.
:::note How the talk was examined
The official transcript of the same session in the full livestream was used: 7:42:51–7:48:18. These are livestream timestamps, not times in the standalone video. Automatic captions may contain errors; the entire talk and its visuals were not checked.
:::
Try it with practice messages The following is an editorial exercise, not a test from the talk. You need an ordinary chat and fictional messages. You do not need to train a model or install software.
- Define a narrow task: extract the meeting date. If there is no exact date, return “not specified”.
- Explain the desired answer using development examples. Refine the instruction only on those examples.
- Keep different messages and their correct answers separately. After refining the instruction, give the assistant only the messages.
- Save its first answer, then compare it with the answer key yourself. Record each error. Do not count a retry after a hint as the original check's result.
Practice message: “The meeting is on November 25, 2026; time to be confirmed.” Expected date: “November 25, 2026”. Another case: “Let's meet next week.” Expected answer: “not specified”. These two visible examples explain the task; create different ones for your own check.
Where the conclusion stops Google's learning course recommends separating test data from training data and refreshing sets after repeated use. Anthropic recommends starting evaluation design with the question it should answer. These are recommendations from different organizations, not an independent test of the talk.
:::summary What to take away
Separate development examples from checking examples. Keep the answer key apart. A small practice exercise can reveal specific errors but does not measure a service's overall reliability. An ordinary chat does not reproduce Gemini training.
:::
Sources
- AI Engineer — Post-Training Gemini 4 for Real Engineering Work
- AI Engineer — CODE Summit Day 2 source livestream
- AI Engineer — official same-session timestamped transcript
- Google — Datasets: Dividing the original dataset
- Anthropic — Building strong safety evaluations