output system prompt

--- name: output-eval-judge-prompt description: Design effective LLM judge .prompt files for evaluators. Use when creating judgeVerdict/judgeScore/judgeLabel prompts, or when existing judges produce unreliable results. allowed-tools: [Read, Write, Edit] --- # Designing LLM Judge Prompts ## Overview An LLM judge evaluates workflow output for a **single, specific failure mode** identified during error analysis. This skill covers how to design the `.prompt` file that powers `judgeVerdict()`, `judgeScore()`, or `judgeLabel()` calls. For the file format basics, see `output-dev-prompt-file`. For error analysis, see `output-eval-error-analysis`. ## Prerequisites Before writing a judge prompt: 1. **Error analysis is complete** — You have identified the specific failure mode this judge targets (from `output-eval-error-analysis`) 2. **20+ labeled examples** — At least 20 pass and 20 fail traces for this failure mode, with `ground_truth` labels in dataset YAML files 3. **Code-based check ruled out** — Confirmed that `Verdict.*` helpers (contains, matches, gte, etc.) cannot reliably detect this failure ## The Four Components Every effective judge prompt has exactly four components. ### 1. Task and Criterion State the single failure mode being evaluated. Be specific and observable. **Good criteria (specific, observable):** - "Does the blog post maintain a formal tone throughout, or does it slip into casual language?" - "Does the output contain any URLs that are fabricated rather than drawn from the input?" - "Does the summary faithfully represent the source material without adding claims not present in the original?" **Bad criteria (vague, holistic):** - "Is this output high quality?" - "Rate the overall effectiveness of this response" - "How good is this content?" ### 2. Pass/Fail Definitions Define exactly what constitutes pass and fail. **Always binary** — no Likert scales, no 1-5 ratings, no "partially meets criteria." ``` PASS: The blog post uses formal language throughout. Professional vocabulary, complete sentences, no slang, no contractions, no first-person casual asides. FAIL: The blog post contains one or more instances of casual language: slang, contractions ("don't", "can't"), informal asides ("pretty cool", "super important"), or conversational filler ("honestly", "basically"). ``` Why binary: Likert scales create ambiguous boundaries (what's the difference between a 3 and a 4?). Binary forces precise definitions that LLMs can apply consistently and that you can validate against human labels. ### 3. Few-Shot Examples Include at least three labeled examples: one clear pass, one clear fail, and one borderline case. **Borderline examples are the most valuable** — they teach the judge where the decision boundary lies. Draw examples from your **training split only** (see `output-eval-validate-judge`). Never use dev or test examples as few-shot — that's data leakage. Each example must include: - The relevant input/output excerpt - A detailed critique explaining the reasoning - The verdict (pass or fail) ### 4. Structured Output Request JSON output with **critique before verdict**. This forces the judge to reason before deciding, which improves accuracy. ```json { "critique": "Detailed analysis of the output against the criterion...", "verdict": "pass" } ``` Always put `critique` first in the schema. If `verdict` comes first, the judge commits to a decision before reasoning. ## Full `.prompt` File Example A judge for the "tone mismatch" failure mode: ``` # tests/evals/judge_tone@v1.prompt --- provider: anthropic # current as of 2026-05-04 — run output-dev-model-selection for the latest model: claude-haiku-4-5-20251001 temperature: 0 maxTokens: 1500 --- <system> You are an evaluation judge. Your task is to determine whether a blog post maintains the requested tone throughout. ## Criterion Assess whether the blog post consistently uses the requested tone. A single paragraph that breaks tone is a failure. ## Definitions PASS: The blog post maintains the requested tone in every paragraph. Word choice, sentence structure, and rhetorical style all align with the requested tone. FAIL: The blog post contains one or more paragraphs where the tone shifts away from what was requested. Common failures include: - Formal request but casual language appears ("pretty cool", "super important", contractions) - Professional request but opinionated editorializing appears - Technical request but oversimplified explanations appear ## Examples ### Example 1: PASS Requested tone: formal Blog excerpt: "The implications of quantum computing for cryptographic security are substantial. Current encryption standards rely on the computational infeasibility of factoring large prime numbers, a guarantee that quantum algorithms may undermine." Critique: The excerpt uses professional vocabulary ("implications", "computational infeasibility"), complete sentences, no contractions, and maintains an academic register. Consistent formal tone throughout. Verdict: pass ### Example 2: FAIL Requested tone: formal Blog excerpt: "Quantum computing is basically going to break all our encryption. It's pretty wild when you think about it — everything we thought was secure might not be." Critique: The excerpt contains multiple casual markers: "basically", "pretty wild", contractions ("It's", "might not be"), and conversational filler ("when you think about it"). This directly violates the formal tone request. Verdict: fail ### Example 3: BORDERLINE (fail) Requested tone: formal Blog excerpt: "Quantum computing represents a paradigm shift in computational capability. The technology is incredibly promising, though it's important to note the current limitations in qubit stability and error correction." Critique: Mostly formal, but contains "incredibly promising" (informal intensifier) and "it's" (contraction). While the overall register is professional, these lapses break the formal tone requirement. Even minor inconsistencies constitute a failure. Verdict: fail ## Output Format Return a JSON object with exactly two fields: - "critique": A detailed analysis (3-5 sentences) citing specific evidence from the blog post - "verdict": Either "pass" or "fail" </system> <user> Requested tone: {{ requested_tone }} Blog title: {{ blog_title }} Blog post: {{ blog_post }} Evaluate whether this blog post consistently maintains the requested tone. </user> ``` ## Wiring to `judgeVerdict()` After creating the `.prompt` file, wire it to an evaluator using `verify()` and `judgeVerdict()`: ```typescript // tests/evals/evaluators.ts import { verify, judgeVerdict } from '@outputai/evals'; import { z } from '@outputai/core'; import { blogInput, blogOutput } from './schemas.js'; export const checkTone = verify( { name: 'check_tone', input: blogInput, output: blogOutput }, async ({ input, output, context }) => judgeVerdict({ prompt: 'judge_tone@v1', variables: { requested_tone: String(context.ground_truth.expected_tone ?? input.tone ?? 'professional'), blog_title: output.title, blog_post: output.blog_post } }) ); ``` Then add it to the eval workflow: ```typescript // tests/evals/workflow.ts import { evalWorkflow } from '@outputai/evals'; import { checkTone } from './evaluators.js'; export default evalWorkflow({ name: 'blog_generator_eval', evals: [ { evaluator: checkTone, criticality: 'required', interpret: { type: 'verdict' } } ] }); ``` ## Choosing What Context to Pass Feed the judge only what it needs to evaluate the criterion. Extra context adds noise and cost. | Failure Mode | Required Variables | Not Needed | |-------------|-------------------|------------| | Tone mismatch | requested_tone, blog_post | topic, input constraints | | Off-topic drift | topic, blog_post | tone, length requirements | | Hallucinated claims | blog_post, source_material | topic, tone | | Faithfulness | summary, original_document | formatting requirements | | Missing requirements | requirements_list, blog_post | topic (unless relevant) | Use `context.ground_truth` for expected values that vary per dataset. Use `input.*` for values from the workflow input. Use `output.*` for the workflow output being evaluated. ## `judgeScore()` Variant Use `judgeScore()` when you need a numeric quality score rather than binary pass/fail. Apply the same four-component design. ### `.prompt` file for scoring ``` # tests/evals/judge_quality@v1.prompt --- provider: anthropic # current as of 2026-05-04 — run output-dev-model-selection for the latest model: claude-haiku-4-5-20251001 temperature: 0 maxTokens: 1500 --- <system> You are an evaluation judge. Score the overall writing quality of a blog post on a scale of 0.0 to 1.0. ## Scoring Criteria - 0.0-0.3: Major issues — incoherent, riddled with errors, or completely off-topic - 0.4-0.6: Mediocre — readable but has significant quality issues (poor structure, weak arguments, factual gaps) - 0.7-0.8: Good — well-structured, clear, minor issues only - 0.9-1.0: Excellent — polished, engaging, publication-ready ## Output Format Return a JSON object with: - "critique": Detailed analysis of quality strengths and weaknesses (3-5 sentences) - "score": A number between 0.0 and 1.0 </system> <user> Topic: {{ topic }} Blog title: {{ blog_title }} Blog post: {{ blog_post }} Score the writing quality of this blog post. </user> ``` ### Wiring to `judgeScore()` ```typescript export const checkQuality = verify( { name: 'check_quality', input: blogInput, output: blogOutput }, async ({ input, output }) => judgeScore({ prompt: 'judge_quality@v1', variables: { topic: input.topic, blog_title: output.title, blog_post: output.blog_post } }) ); ``` In the eval workflow, use `interpret: { type: 'number' }` with thresholds: ```typescript { evaluator: checkQuality, criticality: 'required', interpret: { type: 'number', pass: 0.7, partial: 0.4 } } ``` ## `judgeLabel()` Variant Use `judgeLabel()` when you need classification into named categories. ```typescript export const checkToneLabel = verify( { name: 'check_tone_label', input: blogInput, output: blogOutput }, async ({ output }) => judgeLabel({ prompt: 'judge_tone_label@v1', variables: { blog_title: output.title, blog_post: output.blog_post } }) ); ``` In the eval workflow, use `interpret: { type: 'string' }` with label lists: ```typescript { evaluator: checkToneLabel, criticality: 'informational', interpret: { type: 'string', pass: ['professional', 'formal'], partial: ['casual'] } } ``` ## Model Selection > Run [`output-dev-model-selection`](../output-dev-model-selection/SKILL.md) to resolve each tier below to a current model ID. | Tier | When to Use | Cost | |------|-------------|------| | Smallest in family (`speed`/`cost` priority) | Default for most judges. Fast, cheap, good at following structured instructions. | Low | | Mid-tier (`balance` priority) | Complex reasoning required (faithfulness checking, multi-step logical analysis). | Medium | | Top-tier (`reasoning` priority) | Only if mid-tier fails validation. Rarely needed. | High | Always set `temperature: 0` for judges. Reproducibility matters more than creativity. **Escalation strategy:** start with the smallest tier. If the judge fails validation (TPR/TNR below 80%), move up one tier before rewriting the prompt — the model upgrade alone often fixes it. ## Anti-Patterns - **Vague criteria** ("Is this good?") — Target one specific, observable failure mode - **Holistic judges** ("Rate overall quality on 5 dimensions") — One judge per failure mode - **No few-shot examples** — Always include pass, fail, and borderline examples - **Likert scales** (1-5 ratings) — Use binary pass/fail for verdict judges; use 0.0-1.0 with defined bands for score judges - **Verdict before critique** — Put critique first in the JSON schema to force reasoning - **Skipping validation** — Always validate judges against human labels (`output-eval-validate-judge`) - **Kitchen-sink context** — Pass only the variables the judge needs for its criterion - **Few-shot from dev/test set** — Only use training-split examples to avoid data leakage ## Related Skills - `output-eval-error-analysis` — Identify the failure mode this judge targets - `output-dev-eval-testing` — Implementation reference for `verify()`, `judgeVerdict()`, `evalWorkflow()` - `output-dev-prompt-file` — `.prompt` file format, Liquid.js templating, provider configuration - `output-eval-validate-judge` — Validate this judge against human labels after writing it - `output-eval-dataset-design` — Generate diverse datasets for judge validation

--- name: workflow-prompt-writer description: Use this agent when writing, reviewing, or debugging LLM prompt files (.prompt). Specializes in Liquid.js template syntax, YAML frontmatter configuration, and Output SDK prompt conventions. tools: Read, Write, Edit, Grep, Glob model: sonnet color: yellow --- # Output SDK Prompt Writer Agent ## Identity You are an Output SDK prompt engineering specialist who creates, reviews, and debugs LLM prompt files. You ensure prompts follow Output SDK conventions, use correct Liquid.js template syntax, and are optimized for their intended use case. ## Core Expertise - **Prompt File Format**: YAML frontmatter configuration and message structure - **Liquid.js Templates**: Variable interpolation, conditionals, loops, and filters - **Provider Configuration**: Anthropic, OpenAI, and Azure model settings - **Prompt Design**: System instructions, user prompts, and multi-turn conversations - **Output Optimization**: Structured output prompts for `generateText` with `Output.object()` - **Skills System**: Colocated skill files, frontmatter skill paths, inline `skill()` function - **Agent Class**: Prompts work with both `generateText` and `Agent` for multi-step tool loops ## Prompt File Format ### Basic Structure Prompt files (`.prompt`) consist of YAML frontmatter followed by message content: ```yaml --- provider: anthropic # current as of 2026-05-04 — run output-dev-model-selection for the latest model: claude-sonnet-4-6 temperature: 0.7 maxTokens: 2000 --- <system>You are a helpful assistant.</system> <user>{{ instructions }}</user> ``` ### YAML Frontmatter Options | Option | Type | Description | |--------|------|-------------| | `provider` | string | LLM provider: `anthropic`, `openai`, `vertex`, `azure` | | `model` | string | Model identifier (provider-specific) | | `temperature` | number | Creativity (0.0-1.0, lower = more deterministic) | | `maxTokens` | number | Maximum response length | ### Provider Consistency All prompt files in a workflow **must use the same provider** unless the user explicitly requests otherwise. Mixing providers requires API keys for every provider used, which causes runtime failures. When a workflow has no existing prompts, default to `anthropic`. Otherwise match what sibling prompts already use. ### Picking a model > See [`output-dev-model-selection`](../skills/output-dev-model-selection/SKILL.md) for the canonical decision tree (priority → provider → live AI Gateway lookup → ID translation). Walk through it any time you write or review the `model:` field on a `.prompt` file. ## Role-Based Message Organization Each message role serves a specific purpose. Understanding when to use each is critical for effective prompts. ### Message Tags | Tag | Purpose | Content Type | |-----|---------|--------------| | `<system>` | Define AI identity, rules, and methodology | Static instructions | | `<user>` | Provide data and specific requests | Dynamic content | | `<assistant>` | Show example responses for few-shot learning | Example outputs | ### When to Use Each Role **System Message**: Instructions that don't change between calls - Agent persona and expertise - Task methodology and approach - Output format requirements - Constraints and rules - Few-shot examples (input/output pairs) **User Message**: Dynamic content that changes each call - Input data wrapped in semantic tags - Specific request parameters - Context for this particular invocation **Assistant Message**: Only for few-shot examples - Demonstrate expected output format - Show reasoning patterns - Establish response style ## System Message Structure Structure system messages with clear markdown headers for readability and maintainability. This example is for a plain text output step (no `Output.object()`), so `## Output Format` is appropriate here. When using `Output.object()`, omit the Output Format section -- the schema handles structure. ```yaml <system> ## Role You are an expert competitive intelligence analyst with deep knowledge of market dynamics and business strategy. ## Expertise - Market positioning analysis - Competitive landscape assessment - Strategic recommendation development - Financial performance evaluation ## Task Analyze the provided company data and generate actionable competitive insights. ## Methodology 1. Assess the company's current market position 2. Identify direct and indirect competitors 3. Evaluate competitive advantages and weaknesses 4. Provide strategic recommendations ## Output Format Return a structured analysis with: - Executive summary (2-3 sentences) - Key findings (bullet points) - Strategic recommendations (prioritized list) ## Constraints - Base conclusions only on provided data - If information is insufficient, state what's missing - Maintain objective, professional tone </system> ``` ### Standard Section Headers | Header | Purpose | |--------|---------| | `## Role` | Define the AI's persona and expertise | | `## Expertise` | List specific knowledge areas | | `## Task` | Describe what the AI should accomplish | | `## Methodology` | Step-by-step approach to follow | | `## Output Format` | Specify expected response structure (**only when NOT using `Output.object()`** -- when using structured output, the schema handles format) | | `## Constraints` | Rules and limitations to follow | | `## Examples` | Few-shot examples (optional) | ## Semantic Content Tags Use XML-like tags within messages to clearly separate different types of content. This helps the model understand the structure and purpose of each section. ### Common Semantic Tags ```liquid <context> {{ backgroundInfo }} </context> <data> {{ inputData }} </data> <requirements> {{ taskRequirements }} </requirements> <constraints> {{ limitations }} </constraints> <examples> {{ referenceExamples }} </examples> ``` ### Domain-Specific Tags ```liquid <company-data> {{ companyInfo }} </company-data> <competitors> {{ competitorList }} </competitors> <financial-data> {{ financialMetrics }} </financial-data> <user-feedback> {{ feedbackData }} </user-feedback> <website-content> {{ scrapedContent }} </website-content> ``` ### Example: Well-Structured User Message ```yaml <user> Please analyze this company's competitive position: <company-data> {{ companyData }} </company-data> {% if competitors and competitors.size > 0 %} <competitors> {% for competitor in competitors %} - {{ competitor.name }}: {{ competitor.website }} {% endfor %} </competitors> {% endif %} {% if marketContext %} <market-context> {{ marketContext }} </market-context> {% endif %} <requirements> Focus areas: {{ focusAreas | default: "general competitive analysis" }} Analysis depth: {{ analysisDepth | default: "standard" }} </requirements> </user> ``` ## Liquid.js Template Syntax **CRITICAL**: Output SDK uses Liquid.js, NOT Handlebars. The syntax is different. ### Variables Variables use double curly braces with spaces: ```liquid {{ variable }} # Correct {{ companyName }} # Correct - descriptive camelCase {{variable}} # Wrong - missing spaces {{ x }} # Wrong - unclear name ``` ### CRITICAL: Variable Type Constraint The `variables` field in `generateText` and `Agent` only accepts **`string | number | boolean`** values. You cannot pass arrays or objects directly -- this causes TypeScript compilation errors. When a step has complex data (arrays, objects), it must pre-format them into strings before passing as variables. The prompt then uses the pre-formatted string directly instead of Liquid loops: ```typescript // In the step: pre-format before passing const itemsText = items.map( i => `- ${i.name}: ${i.value}` ).join( '\n' ); const tagsText = tags.join( ', ' ); const { result } = await generateText( { prompt: 'process@v1', variables: { items: itemsText, // string - OK tags: tagsText, // string - OK count: items.length // number - OK } } ); ``` ```yaml # In the prompt: use the pre-formatted string directly <user> Process these items: {{ items }} Tags: {{ tags }} Total: {{ count }} </user> ``` Do NOT use Liquid loops (`{% for %}`) or nested object access (`{{ item.name }}`) in prompts -- the data should already be formatted as a string by the step. ### Conditionals with Fallbacks Always provide fallbacks for optional variables: ```liquid {% if industry %} Industry: {{ industry }} {% else %} Industry: Not specified {% endif %} {% if analysisDepth == "comprehensive" %} Provide a comprehensive analysis including all aspects. {% elsif analysisDepth == "summary" %} Provide a brief summary of key points. {% else %} Provide a standard analysis. {% endif %} ``` ### Boolean Operators Combine conditions with `and`, `or`: ```liquid {% if includeFinancials or includeMetrics %} Include quantitative analysis. {% endif %} {% if status == "active" and priority == "high" %} Urgent: Requires immediate attention. {% endif %} ``` ### Filters Transform variables with filters: ```liquid {{ text | upcase }} # UPPERCASE {{ text | downcase }} # lowercase {{ text | capitalize }} # Capitalize {{ text | truncate: 100 }} # Truncate to 100 chars {{ text | strip }} # Trim whitespace {{ value | default: "fallback" }} # Default if nil/empty ``` ### Common Mistakes ```liquid # Wrong - Handlebars syntax {{#if condition}}...{{/if}} # Correct - Liquid.js syntax {% if condition %}...{% endif %} # Wrong - missing spaces in variables {{variable}} # Correct - spaces required {{ variable }} # Wrong - passing arrays/objects as variables (causes TS2322) variables: { items: itemArray, user: userObject } # Correct - pre-format complex data in the step variables: { items: itemsText, userName: user.name } ``` ## Prompt Engineering Techniques ### Chain-of-Thought Prompting Guide the model through explicit reasoning steps: ```yaml <system> ## Role You are a strategic business analyst. ## Methodology When analyzing competitive positioning, follow this reasoning process: ### Step 1: Market Assessment Analyze the overall market size, growth rate, and dynamics. ### Step 2: Competitive Landscape Identify direct competitors (same product/service) and indirect competitors (alternative solutions). ### Step 3: Differentiation Analysis Evaluate what makes this company unique compared to competitors. ### Step 4: SWOT Summary Summarize Strengths, Weaknesses, Opportunities, and Threats. ### Step 5: Strategic Recommendations Based on the above analysis, provide prioritized recommendations. ## Output Format Work through each step explicitly, showing your reasoning before providing final recommendations. </system> <user> Analyze this company step by step: <company-data> {{ companyData }} </company-data> Please work through each step of the methodology, showing your reasoning at each stage. </user> ``` ### Few-Shot Prompting Provide examples in the system message for consistent output. **Note**: This technique is for plain-text output steps (no `Output.object()`). When using structured output, the schema handles format automatically -- few-shot examples should focus on content quality and reasoning, not output structure. ```yaml <system> ## Role You extract key business metrics from company descriptions. ## Examples ### Example 1 **Input**: "Slack is a business communication platform founded in 2013 with over 18 million daily active users." **Output**: ```json { "name": "Slack", "founded": 2013, "category": "business communication", "keyMetric": "18 million daily active users" } ``` ### Example 2 **Input**: "Shopify provides e-commerce solutions and has powered over $200 billion in sales." **Output**: ```json { "name": "Shopify", "founded": null, "category": "e-commerce platform", "keyMetric": "$200 billion in sales" } ``` ## Task Extract metrics from the provided company description using the same format as the examples. </system> <user> Extract key metrics from this description: <company-description> {{ companyDescription }} </company-description> </user> ``` ### Structured Output Prompts (with Output.object()) When `generateText` is called with `Output.object()`, the Zod schema is sent to the LLM provider automatically as a tool definition. **Do not duplicate the schema in the prompt.** This is a best practice from both Anthropic and Google Vertex AI -- duplicating the schema reduces performance and creates maintenance risk when the schema changes. Instead, use `.describe()` on schema fields (in `types.ts`) for field-level guidance, and use the prompt for **task framing, methodology, and quality standards**: ```yaml <system> ## Role You are a content extractor that identifies the most important information. ## Methodology 1. Read the content carefully to identify the central argument or topic 2. Extract the main title -- prefer the author's own headline if present 3. Write a summary that captures the "why it matters", not just the topic 4. Select key points that are specific and actionable, not generic observations 5. Rate your confidence based on content quality -- lower if the text is ambiguous or incomplete ## Constraints - Base conclusions only on provided content, do not add outside knowledge - If the text is too short or unclear for a confident extraction, reflect that in your confidence score </system> <user> Extract structured data from this text: <content> {{ content }} </content> </user> ``` The corresponding schema in `types.ts` handles structure and field descriptions: ```typescript const ContentExtractionSchema = z.object( { title: z.string().describe( 'The main title or headline' ), summary: z.string().describe( 'A 1-2 sentence summary' ), keyPoints: z.array( z.string() ).describe( '3-5 key points' ), confidence: z.number().describe( 'Confidence score from 0.0 to 1.0' ) } ); ``` **When Output.object() is NOT used** (plain text output), including output format instructions in the prompt is appropriate. ## Skills System Prompts can use skills: lazy-loaded instruction packages that keep the initial context small. The LLM sees skill names/descriptions in the system message and calls `load_skill` to get full instructions on demand. ### Colocated Skills (Auto-Discovery) Place `.md` files in a `skills/` folder next to the prompt file. No configuration needed: ``` prompts/ ├── writing_assistant@v1.prompt └── skills/ ├── clarity_guidelines.md └── structure_guide.md ``` Each skill file has optional YAML frontmatter (`name`, `description`) and a markdown body with full instructions. Mention `load_skill` in the system message so the LLM knows to use it. ### Other Loading Methods - **Frontmatter paths**: Add `skills:` array to YAML frontmatter with file/directory paths - **Inline code**: Use `skill()` from `@outputai/llm` to create skills programmatically - **Disable**: Set `skills: []` in frontmatter to opt out of auto-discovery See `output-dev-skill-file` for the full skill creation guide. ## Using Prompts with Agent Prompts work with both `generateText` (single-shot) and the `Agent` class (multi-step tool loops). Agent extends AI SDK's `ToolLoopAgent` with Output prompt files and skills: ```typescript import { Agent, Output } from '@outputai/llm'; const agent = new Agent( { prompt: 'writing_assistant@v1', variables: { content_type: 'documentation', focus: 'clarity', content: input.content }, output: Output.object( { schema: reviewSchema } ), maxSteps: 5 } ); const { output } = await agent.generate(); ``` See `output-dev-agent-class` for the full Agent guide. ## Best Practices ### Variable Naming Use descriptive camelCase names: ```liquid {{ companyName }} # Good - clear and specific {{ analysisType }} # Good - describes the value {{ marketContext }} # Good - indicates purpose {{ x }} # Bad - unclear {{ data }} # Bad - too generic {{ temp }} # Bad - ambiguous ``` ### Separate Static from Dynamic Content Place dynamic content at the END of messages for better prompt caching: ```yaml <system> [Static instructions that rarely change - cached by provider] </system> <user> [Static context and framing] <dynamic-content> {{ variableContent }} </dynamic-content> </user> ``` ### Document Your Prompts Add comments at the top of complex prompts: ```yaml --- # Competitive Analysis Prompt v2.1 # # Variables: # - companyData (string, required): Company information to analyze # - competitors (string, optional): Comma-separated competitor names (pre-formatted in step) # - focusAreas (string, optional): Specific areas to focus on # - analysisDepth (string, optional): "summary" | "standard" | "comprehensive" # # Output: Structured competitive analysis with recommendations provider: anthropic # current as of 2026-05-04 — run output-dev-model-selection for the latest model: claude-sonnet-4-6 temperature: 0.7 maxTokens: 4000 --- ``` ## File Organization Prompts are stored in the workflow's `prompts/` directory with version suffixes: ``` src/workflows/{name}/ prompts/ analyze@v1.prompt # Initial version analyze@v2.prompt # Improved version extractData@v1.prompt classify@v1.prompt ``` Reference prompts by name and version in code: ```typescript // Pre-format array into string before passing as variable const competitorsText = competitors ? competitors.join( ', ' ) : ''; const { result } = await generateText( { prompt: 'analyze@v1', variables: { companyData, competitors: competitorsText } } ); ``` ## Common Pitfalls ### 1. Wrong Template Syntax Using Handlebars (`{{#if}}`) instead of Liquid.js (`{% if %}`). ### 2. Missing Variable Spaces Writing `{{variable}}` instead of `{{ variable }}`. ### 3. No Array Checks Looping without checking `{% if items and items.size > 0 %}`. ### 4. No Fallbacks Assuming optional variables exist without `{% if %}` or `| default:`. ### 5. Poor Variable Names Using generic names like `data`, `input`, `x` instead of descriptive names. ### 6. Unstructured System Messages Writing system prompts as walls of text instead of using `## Headers`. ### 7. Missing Semantic Tags Dumping data without wrapping in `<data>`, `<context>`, etc. ### 8. Duplicating Schema in Prompt Including `## Output Format` with JSON examples when the step uses `Output.object()`. The schema is sent to the provider automatically -- duplicating it in the prompt reduces performance and creates drift risk. Use `.describe()` on schema fields instead. ## Example Interactions **User**: "Create a prompt for summarizing articles" **Agent**: I'll create a summarization prompt with structured system instructions using ## headers, semantic tags for the article content, and clear output format specification. **User**: "My prompt conditionals aren't working" **Agent**: You're likely using Handlebars syntax (`{{#if}}`). Use Liquid.js syntax instead: `{% if condition %}...{% endif %}`. **User**: "The variables in my prompt aren't being replaced" **Agent**: Check that your variables have spaces inside the braces: `{{ variable }}` not `{{variable}}`. Also verify the variable names match what you're passing in the `variables` object. **User**: "How do I safely loop through an array that might be empty?" **Agent**: Always check existence AND size: `{% if items and items.size > 0 %}{% for item in items %}...{% endfor %}{% else %}No items provided.{% endif %}` **User**: "How should I structure my system message?" **Agent**: Use clear markdown headers: `## Role` for persona, `## Task` for objective, `## Methodology` for approach, `## Output Format` for expected response, and `## Constraints` for rules. --- *This agent specializes in Output SDK prompt file creation, syntax, and best practices.*

--- name: output-dev-prompt-file description: Create .prompt files for LLM operations in Output SDK workflows. Use when designing prompts, configuring LLM providers, or using Liquid.js templating. allowed-tools: [Read, Write, Edit] --- # Creating .prompt Files ## Overview This skill documents how to create `.prompt` files for LLM operations in Output SDK workflows. Prompt files use YAML frontmatter for configuration and Liquid.js templating for dynamic content. ## When to Use This Skill - Creating prompts for LLM-powered workflow steps - Configuring LLM provider settings (model, temperature, etc.) - Using template variables in prompts - Troubleshooting prompt formatting issues ## Location Convention Prompt files are stored INSIDE the workflow folder: ``` src/workflows/{workflow-name}/ ├── workflow.ts ├── steps.ts ├── types.ts └── prompts/ ├── analyzeContent@v1.prompt ├── generateSummary@v1.prompt └── extractData@v2.prompt ``` **Important**: Prompts are workflow-specific and live inside the workflow folder, NOT in a shared location. ## File Naming Convention ``` {promptName}@v{version}.prompt ``` Examples: - `generateImageIdeas@v1.prompt` - `analyzeContent@v1.prompt` - `summarizeText@v2.prompt` The version suffix (`@v1`, `@v2`) allows for prompt versioning without breaking existing code. ## Basic Structure > Picking a model? See [`output-dev-model-selection`](../output-dev-model-selection/SKILL.md) for the current decision tree and AI Gateway lookup script. Examples below show concrete IDs as of 2026-05-04 — refresh them with that skill. ``` --- provider: anthropic # current as of 2026-05-04 — run output-dev-model-selection for the latest model: claude-sonnet-4-6 temperature: 0.7 maxTokens: 4096 --- <system> System instructions go here. </system> <user> User message with {{ variable }} placeholders. </user> ``` ## YAML Frontmatter Options ### Required Fields ```yaml --- provider: anthropic # LLM provider: anthropic, openai, vertex # current as of 2026-05-04 — run output-dev-model-selection for the latest model: claude-sonnet-4-6 --- ``` ### Provider Consistency All prompt files in a workflow should use the **same provider** unless the user explicitly requests otherwise. Mixing providers (e.g., some prompts using anthropic and others using openai) requires the user to have API keys for all providers, which causes runtime failures if they don't. When no existing prompts dictate a provider, default to `anthropic`. For the model itself, see [`output-dev-model-selection`](../output-dev-model-selection/SKILL.md) — it walks priority (reasoning/balance/speed/cost), provider lookup, and produces a current model ID. ### Optional Fields ```yaml --- provider: anthropic # current as of 2026-05-04 — run output-dev-model-selection for the latest model: claude-sonnet-4-6 temperature: 0.7 # 0.0 to 1.0, default varies by provider maxTokens: 4096 # Maximum output tokens providerOptions: # Provider-specific options thinking: type: enabled budgetTokens: 2000 --- ``` ### Common Provider Configurations > Each example below pins a model that was current as of 2026-05-04. Run [`output-dev-model-selection`](../output-dev-model-selection/SKILL.md) when picking or refreshing. #### Anthropic (Claude) ```yaml --- provider: anthropic model: claude-sonnet-4-6 temperature: 0.7 maxTokens: 8192 --- ``` #### Anthropic with Extended Thinking ```yaml --- provider: anthropic model: claude-sonnet-4-6 temperature: 0.7 maxTokens: 32000 providerOptions: thinking: type: enabled budgetTokens: 2000 --- ``` #### OpenAI ```yaml --- provider: openai # current as of 2026-05-04 — run output-dev-model-selection for the latest model: gpt-5-5 temperature: 0.7 maxTokens: 4096 --- ``` #### Vertex (Gemini) ```yaml --- provider: vertex # current as of 2026-05-04 — run output-dev-model-selection for the latest model: gemini-3-pro temperature: 0.7 maxTokens: 8192 --- ``` ## Message Blocks Use XML-style tags to define message roles: ### System Message ``` <system> You are an expert at analyzing technical content. Your responses should be clear and structured. </system> ``` ### User Message ``` <user> Please analyze the following content: {{ content }} </user> ``` ### Assistant Message (for few-shot examples) ``` <assistant> I'll analyze this content step by step... </assistant> ``` ## Liquid.js Templating ### Variable Substitution ``` <user> Analyze this content about {{ topic }}: {{ content }} Generate {{ numberOfIdeas }} ideas. </user> ``` ### Conditional Content ``` <system> You are an expert content analyzer. {% if colorPalette %} **Color Palette Constraints:** {{ colorPalette }} {% endif %} {% if artDirection %} **Art Direction Constraints:** {{ artDirection }} {% endif %} </system> ``` ### Loops ``` <user> Analyze each of these items: {% for item in items %} - {{ item.name }}: {{ item.description }} {% endfor %} </user> ``` ### Default Values ``` <user> Generate {{ numberOfIdeas | default: 3 }} ideas for {{ topic }}. </user> ``` ## Complete Example Based on a real prompt file (`generateImageIdeas@v1.prompt`): ``` --- provider: anthropic # current as of 2026-05-04 — run output-dev-model-selection for the latest model: claude-sonnet-4-6 temperature: 0.7 maxTokens: 32000 providerOptions: thinking: type: enabled budgetTokens: 2000 --- <system> You are an expert at creating structured, precise infographic prompts optimized for Gemini's image generation model. Your task is to generate prompts for informational infographics that illustrate key concepts from the provided content. CRITICAL RULES you MUST follow: - Use Markdown dashed lists to specify constraints - Use ALL CAPS for "MUST" requirements to ensure strict adherence - Include specific compositional constraints (e.g., rule of thirds, lighting) - Always include negative constraints to prevent unwanted elements - Keep each infographic focused on ONE clear concept {% if colorPalette %} **Color Palette Constraints:** {{ colorPalette }} {% endif %} {% if artDirection %} **Art Direction Constraints:** {{ artDirection }} {% endif %} </system> <user> Generate {{ numberOfIdeas }} structured infographic prompts based on key topics from this content. <content> {{ content }} </content> Each prompt MUST follow this structure: Create an infographic about [specific topic]. The infographic MUST follow ALL of these constraints: - The infographic MUST use the reference images as a visual style guide - The composition MUST follow the rule of thirds for visual balance - The infographic MUST use clean, minimal design with simple lines and shapes {% if colorPalette %}- The color palette MUST strictly follow: {{ colorPalette }}{% endif %} {% if artDirection %}- The art direction MUST strictly follow: {{ artDirection }}{% endif %} - NEVER include any watermarks, logos, or decorative overlays - NEVER use generic AI art buzzwords like "hyperrealistic" Focus on the most important concepts that would benefit from visual explanation. </user> ``` ## CRITICAL: Variable Type Constraint The `variables` field in `generateText` and `Agent` only accepts **`string | number | boolean`** values. You cannot pass arrays or objects as variables -- TypeScript will reject them. When your step has complex data (arrays, objects), pre-format it into a string before passing it as a variable: ```typescript // WRONG - arrays/objects as variables cause TS2322 const { output } = await generateText( { prompt: 'rank@v1', variables: { stories: storyArray, // Type error: not assignable to string | number | boolean interests: interestArray // Type error: not assignable to string | number | boolean } } ); // CORRECT - pre-format complex data into strings const storiesText = stories.map( s => `- ${s.title} (score: ${s.score}, by: ${s.author})` ).join( '\n' ); const interestsText = interests.join( ', ' ); const { output } = await generateText( { prompt: 'rank@v1', variables: { stories: storiesText, // string - OK interests: interestsText // string - OK } } ); ``` The prompt template then uses the pre-formatted string directly with `{{ stories }}` instead of Liquid loops. This is simpler and avoids the type constraint entirely. ## Using Prompts in Steps ### With generateText and Output.object() ```typescript import { generateText, Output } from '@outputai/llm'; import { z } from '@outputai/core'; const { output } = await generateText( { prompt: 'generateImageIdeas@v1', // References prompts/generateImageIdeas@v1.prompt variables: { content: 'Solar panel technology explained...', numberOfIdeas: 3, colorPalette: 'blue and green tones', artDirection: 'minimalist style' }, output: Output.object( { schema: z.object( { ideas: z.array( z.string() ) } ) } ) } ); // output contains { ideas: [...] } ``` ### With generateText ```typescript import { generateText } from '@outputai/llm'; const { result } = await generateText( { prompt: 'summarize@v1', variables: { content: 'Long article text...', maxLength: 200 } } ); // result contains the generated text string ``` ## Using Skills with Prompts Prompts can load skill files that provide lazy-loaded instructions to the LLM. Skills keep the initial context small while giving the LLM access to deep expertise on demand. See `output-dev-skill-file` for the full guide on creating skill files. The simplest approach is colocated auto-discovery. Place `.md` files in a `skills/` folder next to your prompt file: ``` src/workflows/{workflow-name}/ └── prompts/ ├── writing_assistant@v1.prompt └── skills/ ├── clarity_guidelines.md └── structure_guide.md ``` The prompt file does not need any special configuration. Output auto-discovers the `skills/` directory and injects a `load_skill` tool the LLM can call. Mention `load_skill` in the system message so the LLM knows to use it: ``` <system> You are an expert technical writing assistant. Use load_skill to get the full instructions for any skill before applying it. </system> ``` You can also list skill paths explicitly in frontmatter, or create inline skills in code. See `output-dev-skill-file` for all three methods. ## Using Prompts with Agent Prompts work with both `generateText` and the `Agent` class. Use `Agent` for multi-step tool loops and stateful conversations. See `output-dev-agent-class` for the full guide. ```typescript import { Agent, Output } from '@outputai/llm'; const agent = new Agent( { prompt: 'writing_assistant@v1', variables: { content_type: 'documentation', focus: 'clarity', content: input.content }, output: Output.object( { schema: reviewSchema } ), maxSteps: 5 } ); const { output } = await agent.generate(); ``` ## CRITICAL: Prompts and Structured Output Schemas ### Do Not Duplicate the Schema in the Prompt When a step uses `Output.object()` with `generateText`, the Zod schema is automatically sent to the LLM provider as a tool definition. The LLM already knows the exact JSON shape it must return. **Do not also specify the schema in the prompt.** This is a best practice documented by multiple LLM providers: - **Anthropic**: The schema is sent as a tool definition; `.describe()` on fields is how you guide the model's output. The SDK automatically transforms unsupported constraints into field descriptions. - **Google Vertex AI**: "Only specify the schema in the schema object. Don't also specify the schema in the prompt. Doing both can reduce performance." If you must discuss the schema in the prompt, match the exact field order from the schema. **Why this matters:** 1. **Performance**: Redundant schema instructions can confuse the model and reduce output quality 2. **Maintenance**: When the schema changes, you must update both the schema AND the prompt, or they drift apart 3. **Correctness**: The prompt's JSON examples can contradict the actual schema (wrong field names, missing fields, wrong types) ### What NOT to Include in Prompts When `Output.object()` is used, do not include any of these in the prompt: - `## Output Format` sections describing the JSON shape - JSON examples showing the expected response structure - Field-by-field descriptions that mirror the schema - Instructions like "Return a JSON object with exactly these fields" - Instructions like "Return only the JSON object with no surrounding explanation" ``` <!-- WRONG - prompt duplicates what Output.object() already sends --> <system> ## Output Format Return a JSON object with this shape: { "title": "string", "summary": "string", "tags": ["string"] } </system> ``` ### What TO Include in Prompts Use the prompt for **quality expectations, domain knowledge, and content guidance** -- things the schema cannot express: ``` <!-- CORRECT - prompt focuses on content quality, not structure --> <system> Write a concise, specific title (under 80 characters). The summary should capture the main argument, not just the topic. Choose tags from the reader's domain -- avoid generic terms like "technology". </system> ``` ### Use `.describe()` on Schema Fields Instead The right place to communicate field-level expectations is on the schema itself, using `.describe()`. LLM providers use these descriptions when generating output: ```typescript // In types.ts -- .describe() guides the LLM on each field const ArticleSummarySchema = z.object( { title: z.string().describe( 'Concise title under 80 characters' ), summary: z.string().describe( 'One-sentence summary capturing the main argument' ), tags: z.array( z.string() ).describe( '3-5 domain-specific tags, avoid generic terms' ) } ); ``` The schema handles structure AND field-level guidance; the prompt handles task framing, methodology, and quality standards. ### When the Step Does NOT Use Output.object() If `generateText` is called **without** `Output.object()` (plain text output), then including output format instructions in the prompt is appropriate since no schema is sent to the provider. ## Best Practices ### 1. Be Explicit About Requirements ``` <system> CRITICAL RULES you MUST follow: - Rule 1 - Rule 2 - NEVER do X - ALWAYS do Y </system> ``` ### 2. Use XML Tags for Structure in User Messages ``` <user> Analyze the following: <content> {{ content }} </content> <requirements> {{ requirements }} </requirements> </user> ``` ### 3. Provide Examples (Few-Shot) ``` <system> You analyze sentiment. Return: positive, negative, or neutral. </system> <user> "I love this product!" </user> <assistant> positive </assistant> <user> "{{ text }}" </user> ``` ### 4. Version Your Prompts When making significant changes, create a new version: - `analyzeContent@v1.prompt` - Original - `analyzeContent@v2.prompt` - Improved with better examples Update the step to use the new version: ```typescript prompt: 'analyzeContent@v2' // Changed from v1 ``` ### 5. Handle Optional Variables ``` {% if optionalField %} Additional context: {{ optionalField }} {% endif %} ``` ## Common Patterns > The model lines in the patterns below were current as of 2026-05-04. Refresh via [`output-dev-model-selection`](../output-dev-model-selection/SKILL.md) when copying into a new prompt. ### Classification Prompt ``` --- provider: anthropic model: claude-sonnet-4-6 temperature: 0.3 --- <system> You are a content classifier. Categorize content into exactly one category. Available categories: {{ categories | join: ", " }} </system> <user> Classify this content: {{ content }} </user> ``` ### Extraction Prompt ``` --- provider: anthropic model: claude-sonnet-4-6 temperature: 0.2 --- <system> You extract structured data from text. Be precise and only include information explicitly stated. </system> <user> Extract the following fields from this text: {% for field in fields %} - {{ field }} {% endfor %} Text: {{ text }} </user> ``` ### Generation Prompt ``` --- provider: anthropic # current as of 2026-05-04 — run output-dev-model-selection for the latest model: claude-sonnet-4-6 temperature: 0.8 --- <system> You are a creative writer. Generate engaging content based on the given parameters. </system> <user> Generate {{ count }} {{ type }} about {{ topic }}. Requirements: {{ requirements }} </user> ``` ## Verification Checklist - [ ] File located in `prompts/` folder inside workflow directory - [ ] File named `{promptName}@v{version}.prompt` - [ ] YAML frontmatter includes `provider` and `model` - [ ] Message blocks use proper XML tags (`<system>`, `<user>`, `<assistant>`) - [ ] Variables use `{{ variableName }}` syntax - [ ] Conditionals use `{% if %}...{% endif %}` syntax - [ ] All required variables are documented or have defaults - [ ] Step code references correct prompt name - [ ] No JSON output format instructions when step uses `Output.object()` (schema handles structure) ## Related Skills - `output-dev-skill-file` - Creating skill files for prompts - `output-dev-agent-class` - Using the Agent class with prompts - `output-dev-step-function` - Using prompts in step functions - `output-dev-evaluator-function` - Using prompts in evaluators - `output-dev-folder-structure` - Understanding prompts folder location - `output-dev-workflow-function` - Orchestrating LLM-powered steps - `output-eval-judge-prompt` — Methodology for designing effective LLM judge prompts