AI for Design Verification: A Practitioner Playbook

Milan Kubavat · May 14, 2026 · -- min read · Agentic Patterns, AI, AI Playbook, Context Engineering, Debug, Prompt Engineering, SystemVerilog, UVM, Verification,

This post is the foundational reference for the AI page. It is a research-backed playbook of patterns for using AI as a DV engineer today — prompt engineering, context engineering, pair programming, debugging, code generation, technical debt, agentic patterns, and the honest limits. Each subsequent card on the AI page deep-dives into one of these themes.

Two years ago the practitioner question was "how do I prompt the LLM?" Today the question is "how do I engineer the context, tools, and reasoning loop so the LLM behaves like a teammate?" This page is the playbook for that second question, written for DV engineers who want patterns they can use this week — not vendor pitches and not infrastructure projects.

Every section opens with a research callout citing the specific arxiv paper that informs it, lays out a small set of numbered patterns, and closes with the concrete DV/UVM/RTL application. We deliberately skip RAG-heavy setups (they require pipelines you do not yet have) and stay focused on what works inside a single conversation, a single agent, or a small orchestrated team of them. The references are listed in full at the end.

1. The Shift: Prompt → Context

After three years of "prompt engineering," the field reorganized in 2025 around a more useful frame. Prompt engineering optimizes one interaction. Context engineering decides what configuration of state, tools, and history most reliably produces the desired behavior over many turns.

Research Anthropic, Effective context engineering for AI agents (2025); arxiv 2509.21361 Context Is What You Need: The Maximum Effective Context Window. The 2026 paper shows that the effective context window is dramatically smaller than the maximum — some frontier models degrade severely past 1,000 tokens of operational context, regardless of advertised window size.
  1. Right Altitude. Direct, specific instructions at the level of the agent. Hardcoded brittle logic at one end and vague high-level guidance at the other are both failure modes; you want the Goldilocks middle.
  2. Token Efficiency. The smallest set of high-signal tokens that maximize the desired outcome. More context is not better — it dilutes attention and triggers the effective-window degradation above.
  3. Tool Design. Self-contained, error-robust, unambiguous. If a human engineer cannot decide which tool to call in a given situation, an agent cannot either — bloated tool sets are a leading failure mode.
  4. Sequence Thinking. What did previous turns establish? What tool outputs carry forward? What should still be present three steps from now? Single-turn prompting cannot answer these.
Apply to DV The default DV-engineer move — paste the entire failing regression log into the chat — is exactly the move the effective-window research argues against.
  • Send the last ~200 events from the structured log, not the full 80 MB.
  • Send the relevant RTL excerpt, not the whole module.
  • Send the failing checker output, not every checker that fired.
  • Keep one persistent "DV agent prompt" that survives turns; let the model rebuild context from compact tool outputs rather than re-pasting.

2. Prompt Engineering Patterns That Survived

The 2024-2025 surveys catalog dozens of techniques. Most are academic. A small set are battle-tested and apply cleanly to DV work.

Research arxiv 2406.06608 The Prompt Report — a PRISMA-grounded systematic survey of 58 LLM prompting techniques. arxiv 2402.07927 A Systematic Survey of Prompt Engineering. arxiv 2601.01954 Reporting LLM Prompting in Automated SE — 2026 guidelines distinguishing essential, desirable, and exceptional reporting elements.
  1. Specification-as-prompt. Paste the protocol section (or interface description) directly into the prompt. Stop paraphrasing — the spec is the highest-density signal you have.
  2. Few-shot from golden examples. Two or three working sequences from your codebase beat any generic example library. The model picks up your team's idioms, naming, and style.
  3. Chain-of-thought for timing or protocol reasoning. Prompt the model to walk through the timing diagram cycle by cycle, then produce code. Universal accuracy boost on multi-step problems.
  4. Self-critique loop. "Generate a UVM driver, then review it for race conditions and protocol violations." The model catches a sizeable fraction of its own mistakes when asked to.
  5. Decomposition. Break "verify this IP" into 6-8 explicit sub-tasks: agents, sequences, checkers, scoreboard, coverage, configurations, tests, regression. Generate per sub-task; never one-shot the whole TB.
  6. Role priming. "You are a senior DV engineer reviewing a 10-year-old testbench" routes the model into a more conservative, idiom-aware mode than the default helpful assistant.
Apply to DV
  • Patterns 1+2 when scaffolding a new UVM agent from a fresh protocol spec.
  • Pattern 3 when triaging a hard timing or arbitration bug.
  • Pattern 4 on every non-trivial code generation — cheap, catches own-goals.
  • Pattern 5 for any task that spans more than one file or component.
  • Pattern 6 when you need the model to flag issues it would otherwise be too "helpful" to call out.

3. Context Engineering for DV Workflows

Prompt engineering picks the words. Context engineering picks what is in the window at all. The latter is where the productivity gains in 2026 are coming from.

Research Anthropic, Effective context engineering (2025). arxiv 2603.04814 Beyond the Context Window: A Cost-Performance Analysis of Fact-Based Memory vs. Long-Context LLMs — for persistent agents, compact fact-based memory beats long-context inference on both accuracy and cumulative API cost. arxiv 2509.21361 reinforces the "less context, more signal" conclusion.
  1. Minimum viable context. Start with the smallest plausible context that could solve the task. Add only when the model demonstrably needs more — not preemptively.
  2. One tool per concept. Five focused tools beat one mega-tool. If you cannot describe a tool's purpose in one sentence, split it.
  3. Scratchpad / think tool. Give the agent a separate workspace to reason in without cluttering the main conversation. Anthropic's think tool is the canonical example; the pattern is reusable.
  4. Stale-context pruning. Re-summarize at checkpoints. A 30-turn conversation should re-emit a compact "state so far" every 8-10 turns so the model is not navigating its own debris.
  5. Output-grounding. Have the model cite the exact log line or RTL line it is reasoning from. Anchored outputs are auditable; floating outputs hallucinate.
Apply to DV
  • For triage, the "minimum viable context" is usually: failing checker output + ~200 surrounding events from the structured log + the bound checker or monitor code + the most recent RTL commit hash.
  • Build small tools the agent can call: get_rtl_excerpt(file, line, +/-N), query_log(filter), get_commit_diff(hash). Never one tool that does all three.
  • Re-summarize every time the agent calls a sim tool — sim output is dense and pollutes context fast.

4. AI as Pair Programmer

The productivity research is converging: AI assistants help most on scaffolding, refactoring, and explanation; least on novel architectural decisions and deep domain reasoning. Use them where they actually win.

Research arxiv 2507.03156 The Impact of LLM-Assistants on Software Developer Productivity: A Systematic Review. arxiv 2510.09244 Fundamentals of Building Autonomous LLM Agents. AI-pair-programming reliably accelerates code generation, completion, translation, debugging, documentation, and system-design tasks; gains are smaller on tasks requiring deep domain context.
  1. UVM scaffolding from interface descriptions. Driver, monitor, sequencer, agent skeleton. The model is good at boilerplate; you are good at the protocol corners. Generate, then fix the 20% that matters.
  2. Refactor proposals on inherited testbench code. Paste the legacy class, ask for a refactor with three options (minimal, moderate, ambitious). Pick the one that matches your risk tolerance.
  3. "Explain this commit." Drop an RTL diff into the chat; ask what it does behaviorally, not textually. Catches accidental functional changes hiding inside what looks like a rename.
  4. Rubber-duck conversations. Describe a hard bug or design choice out loud. Half the time the act of articulating produces the answer; the other half the model contributes something useful.
  5. Documentation pass. Generate component docs from class definitions + recent behavior. Best when the human edits afterward; weakest when shipped unreviewed.
Apply to DV
  • Pattern 1 for every new UVM agent — saves 60-90 minutes of skeleton work.
  • Pattern 2 when you inherit a 6-year-old testbench and need to understand it before touching it.
  • Pattern 3 on every PR that touches RTL you are responsible for verifying.
  • Pattern 4 instead of a long Slack thread — faster, and the conversation can be saved.
  • Pattern 5 to keep IP-level documentation alive without dedicated tech-writer time.

5. AI as Debugger (Pair Debug)

The strongest empirical results in the AI-for-code space are on debugging, not generation. The numbers are real and the patterns transfer to DV with minor adjustment.

Research arxiv 2603.19399 DePro — test-case-driven LLM debugging reduces debug attempts by up to 64% and saves 7.6 minutes per problem vs. zero-shot LLM and human baselines. arxiv 2511.08052 Dual-Process Scaffold Reasoning for code debug. arxiv 2304.05128 Teaching LLMs to Self-Debug. arxiv 2407.20898 ThinkRepair — few-shot chain-of-thought improves repair across model families. arxiv 2604.10508 shows iterative self-repair is universally effective across 7 models from 3 families.
  1. Hypothesis-rank. "Given this failing log slice, propose the three most likely root causes ranked by probability with one falsifying experiment each." You check the experiments; the model never decides.
  2. Chain-of-thought on the log slice. "Walk through this 50-event window. At each event, state what should happen and what did happen." The CoT structure is what produces the 64% attempt reduction in the DePro results.
  3. Self-debug iteration loop. Generate hypothesis → predict test outcome → run smoke → if wrong, refine. Three iterations beat one zero-shot answer almost universally.
  4. Invariant-violation check. "What invariant does this signal sequence violate?" Forces the model into a property-shaped frame — closer to how senior DV engineers actually reason.
  5. Two-track reasoning. Per the Dual-Process Scaffold paper: have one prompt analyze, a separate prompt scaffold a structural plan, then combine. Beats single-prompt chain-of-thought on hard cases.
Apply to DV
  • Pattern 1 is the highest-leverage entry point — spend zero infrastructure to start, just paste a log slice and ask.
  • Pattern 2 paired with structured logs (see the Debug page) is a force multiplier — the JSON events are cleaner CoT input than human prose.
  • Pattern 3 with a simulator-runnable smoke test closes the loop; this is where DV is structurally advantaged over generic SWE debug because re-running is fast.
  • Pattern 4 for protocol bugs in particular — framing it as "invariant violation" routes the model toward checker logic, not vibes.

6. Code Generation: What Works, What Does Not

This is the section where the research is most sobering. Hardware code generation is not a solved problem and the published numbers do not match the marketing.

Research arxiv 2506.14074 CVDP: Comprehensive Verilog Design Problems — 783 expert-authored problems across 13 hardware design and verification categories; SOTA LLMs achieve no more than 34% pass@1 on code generation. arxiv 2506.07945 ProtocolLLM — LLMs consistently struggle to generate even syntactically valid SystemVerilog testbench code for communication protocols. arxiv 2604.24621 Evaluation of LLM-Based SE Tools — ground truth, determinism, and objective correctness assumptions break under LLM outputs; multi-run reporting and calibrated hybrid judging are the recommended replacements. arxiv 2505.02133 shows multi-agent collaboration plus runtime debugging beats single-shot generation.
  1. Generate small, verify fast. One module, one sequence, one checker at a time. The CVDP numbers are pass@1 at problem scale; at sub-problem scale the success rate is much higher.
  2. Specification-first generation. The model writes from a structured spec, not from a vibe. Quality of output is bounded by quality of input.
  3. Multi-run rank stability. Generate the same module three times, keep the version that compiles and passes the smoke test. The variance is real; pretending it does not exist is the failure.
  4. Sandboxed execution gate. Never accept generated RTL or testbench code that has not run. The eval research (2604.24621) is explicit: executable checks plus small human audit sets are the only trustworthy combo right now.
  5. Multi-agent collaboration. Per 2505.02133: a generator + debugger pair outperforms a single-agent loop. The validator catches what the generator misses.
Apply to DV
  • Best targets: scoreboard skeletons, monitor templates, sequence libraries, register adapter glue, repetitive coverage points.
  • Bad targets right now: protocol-correct RTL for novel interfaces, complex multi-cycle datapaths, anything where the CVDP benchmark distribution applies.
  • Build a one-button smoke gate for any AI-generated file — compile + lint + 1 test. Treat anything that fails the gate as not done.
  • Track which categories of generated code succeed in your codebase. The hit rates differ by IP, protocol, and team style — learn yours, do not trust an averaged benchmark.

7. Technical Debt in AI-Assisted Code

AI-assisted code creates new categories of technical debt that did not exist before. The 2025-2026 research identifies them explicitly and gives detection and remediation strategies that are cheap to adopt.

Research arxiv 2601.06266 Self-Admitted Technical Debt in LLM Software — first empirical comparison across 477 repositories identifies three new debt types unique to LLM-era development: Model-Stack Workaround Debt, Model Dependency Debt, and Performance Optimization Debt. arxiv 2507.03536 ACE: Automated Technical Debt Remediation with Validated LLM Refactorings — combines LLM creativity with a validation layer to deliver refactorings at 98% precision. arxiv 2501.09888 on automated SATD repayment.
  1. Model-Stack Workaround Debt. Hacks added to compensate for a specific model's limitations (token limits, hallucinations, format quirks). They rot fast as models improve. Tag with a comment naming the model + date so future you knows when to retest.
  2. Model Dependency Debt. Logic that only works with a specific model or provider. The day you switch — or the day the provider deprecates a version — everything breaks silently. Abstract the call site behind a thin adapter.
  3. Performance Optimization Debt. Caching, batching, and prompt-trimming hacks that improve cost or latency at the expense of clarity. Document why the hack exists, not just what it does — otherwise future cleanup removes it and reintroduces the original problem.
  4. SATD comment convention. A standard inline tag (e.g., // AI-SATD(model=claude-3.5, date=2026-05): ...) so detection tools can grep for them and refactoring sprints can plan around them.
  5. Validation-layer refactoring. Per the ACE paper: never let AI-generated refactorings ship without a verifier (compile + lint + smoke test minimum). Precision drops without it.
Apply to DV
  • UVM testbenches accumulate all three debt types fast — testbenches change daily and model output gets committed quickly.
  • Audit AI-generated checker and scoreboard code quarterly; the silent failure mode is a checker that no longer detects what its author thought it detected.
  • Keep one canonical reference example per pattern in your codebase. When the model drifts (and it will), the reference is the truth.

8. Agentic Design Patterns × DV

Agentic AI is the most active research area in 2025-2026 and it has clean fusion points with UVM testbench construction. Three published systems (HAVEN, UVM², UVMarvel) demonstrate the patterns at real coverage numbers; the literature gives you the vocabulary to reuse them.

Research arxiv 2601.12560 Agentic AI: Architectures, Taxonomies, and Evaluation. arxiv 2510.09244 Fundamentals of Building Autonomous LLM Agents. arxiv 2604.00835 Agentic Tool Use in LLMs — the four-phase cycle: define tool schema, select & parameterize, invoke, integrate. arxiv 2510.25445 Agentic AI Comprehensive Survey. Applied to hardware: arxiv 2604.27643 HAVEN — 100% compile success and 90.6% code coverage on UVM testbench synthesis across 19 IPs. arxiv 2504.19959 UVM². arxiv 2605.04704 UVMarvel — 95.65% code coverage with 4.5 hours of automated execution at subsystem level.
  1. ReAct (Reason + Act). Interleave reasoning steps with tool calls. Apply to DV: an agent that reasons about a failing log, calls get_rtl_excerpt, reasons more, calls run_smoke_test, then proposes a fix.
  2. Reflexion / Self-Critique Loops. The agent reviews its own output and revises. UVM² uses exactly this loop to iteratively refine test stimuli based on coverage feedback — the loop is what drives the high coverage numbers.
  3. Plan-and-Execute. Generate the full plan first, then execute steps. HAVEN uses this explicitly: an architectural planning agent produces a structured TB plan, then a template engine generates the components. The split is what gets HAVEN to 100% compile success.
  4. Multi-agent specialization. Designer, generator, verifier, reviewer as separate agents with separate prompts and tools. UVMarvel orchestrates specialized agents per bus protocol; specialization is what makes subsystem-level generation tractable.
  5. Controllable orchestration with checkpoints. Explicit state transitions and human-approval gates. The 2026 research consensus is that graph-based orchestration with debuggability and checkpoints beats free-form agent loops in production — freedom degrades reliability.
Apply to DV
  • Start with Pattern 1 (ReAct) plus 3-4 carefully designed tools (log query, RTL excerpt, smoke run, commit diff). One agent, modest scope.
  • Add Pattern 2 (Reflexion) when scope grows — the agent improves measurably from one cycle of self-review.
  • Pattern 3 (Plan-and-Execute) for testbench scaffolding tasks — the planner produces the agent / sequencer / driver list; the executor generates each. This is the HAVEN recipe.
  • Patterns 4 and 5 are for multi-week or multi-team initiatives, not for next Monday. Adopt them only after Patterns 1-3 are working reliably.
  • Mirror the published research: every published agentic-DV system uses checkpoints and human approval. None of them are full autopilot. Plan accordingly.

9. The Honest Limits

What still fails. This section is what separates a useful playbook from vendor marketing.

Research arxiv 2411.09916 "Should I Give Up Now?" Investigating LLM Pitfalls in Software Engineering. CVDP (2506.14074) and ProtocolLLM (2506.07945) ceilings on hardware-specific tasks. Effective-context-window degradation (2509.21361). Together these establish where the limits genuinely are today.
  1. Multi-cycle RTL state reasoning. Models lose track of state across more than a handful of cycles. Anything that requires reasoning about deep pipelines, multi-clock interactions, or long-latency protocols is unsafe to delegate.
  2. Novel protocol corners. The model knows AXI, PCIe, and USB because the training data does. It does not know your proprietary protocol the way you do. Generated code for novel interfaces tends to look right and be wrong.
  3. Long-context degradation. Sending more than ~1,000 tokens of operational context degrades accuracy on many models even within the advertised window. The fix is context engineering, not bigger windows.
  4. Functional correctness gaps. The 34% pass@1 ceiling on CVDP is real. Generated RTL that compiles is not generated RTL that works.
  5. When to walk away. If three iterations of self-debug do not converge, the model is not going to get there on the fourth. Switch to a human or change the framing — do not just keep retrying.

Use AI Where the Research Says It Wins

  1. Debug: hypothesis ranking, CoT on log slices, self-debug loops with smoke tests. The strongest empirical results live here.
  2. Scaffolding: UVM agent skeletons, sequence libraries, repetitive coverage points.
  3. Explanation: "explain this commit," legacy-code walkthroughs, documentation.
  4. Refactoring: behind a validation gate. Never without one.

Reading List

Prompt & Context Engineering

  • Anthropic, Effective Context Engineering for AI Agents (2025) — the canonical industry essay on the shift.
  • arxiv 2406.06608 — The Prompt Report, 58 techniques, PRISMA-grounded.
  • arxiv 2402.07927 — Systematic Survey of Prompt Engineering.
  • arxiv 2407.12994 — Prompt Engineering Methods for NLP Tasks survey.
  • arxiv 2509.21361 — Maximum Effective Context Window.
  • arxiv 2603.04814 — Beyond the Context Window, fact-based memory vs. long-context.
  • arxiv 2601.01954 — Reporting LLM Prompting in Automated SE.

Pair Programming & Debugging

Code Generation

  • arxiv 2604.24621 — Evaluation of LLM-Based SE Tools.
  • arxiv 2503.01245 — LLMs for Code Generation Comprehensive Survey.
  • arxiv 2505.02133 — Multi-Agent Collaboration and Runtime Debugging.
  • arxiv 2506.14074 — CVDP, 783-problem hardware benchmark.
  • arxiv 2506.07945 — ProtocolLLM, SV testbench benchmark.

Technical Debt

  • arxiv 2601.06266 — SATD in LLM Software, three new debt types.
  • arxiv 2507.03536 — ACE: Validated LLM Refactorings.
  • arxiv 2501.09888 — Automated SATD Repayment.

Agentic Design Patterns

Limits & Pitfalls

  • arxiv 2411.09916 — "Should I Give Up Now?" LLM Pitfalls in SE.

For the debugging-specific deep-dive, see the Debug page. For UVM design patterns these techniques compose with, see UVM.

Author
Milan Kubavat
Sharing knowledge about silicon verification, hardware design, and engineering insights.

Comments (0)

Leave a Comment