awesome-agentic-ai-zh system prompt

# Stage 2 — Prompt Engineering > [繁體中文](./02-prompt-engineering.md) | [简体中文](./02-prompt-engineering.zh-Hans.md) | **English** ⏱ **Time estimate**: 1-2 weeks (~5-12 hours) > 👋 **Coming from [Stage 1](01-llm-basics.en.md)?** Good — you can call an API. The next 5-12 hours: write reusable structured prompts, use few-shot and chain-of-thought for hard reasoning tasks, and quantify prompt improvement with evals. **Jumped straight here?** Make sure you can call an LLM API and estimate cost in tokens — if not, head back to [Stage 1](01-llm-basics.en.md). > 💡 Term-unfamiliar? (prompt / few-shot / CoT / system prompt / …) → see [`resources/glossary.en.md`](../resources/glossary.en.md). ## 📌 Learning Goals After this stage you will be able to: - Write structured prompts (role + task + format + examples) - Apply few-shot prompting and know when it helps - Use chain-of-thought (CoT) for reasoning tasks - Iteratively refine a prompt and measure improvement - Recognize when prompting hits its limit (and you need tools / agents) ## 🚪 Entry Conditions You should already: - Be able to call an LLM API (Stage 1) - Be able to parse / iterate over API responses ## 📚 Required Reading 1. [**anthropics/prompt-eng-interactive-tutorial**](https://github.com/anthropics/prompt-eng-interactive-tutorial) ⭐⭐⭐⭐⭐ ★ 35k+ — **Anthropic's official interactive tutorial**, 9 chapters of Jupyter notebooks (basic / intermediate / advanced + appendix), with playground and answer key. Runs on Claude 3 Haiku (cheapest). **The canonical hands-on resource for Stage 2.** Also packaged as module 2 of the [**anthropics/courses**](https://github.com/anthropics/courses) 5-course umbrella — for broader coverage (API Fundamentals / Real World Prompting / Eval / Tool Use) go straight to the umbrella 2. [**anthropics/courses — Real World Prompting**](https://github.com/anthropics/courses) ⭐⭐⭐⭐ ★ 21k+ — Module 3 of the same umbrella, **"how to actually use prompting in real situations"**: chatbot / legal / financial / coding case walkthroughs. Read #1 first, then this. 3. [**Anthropic Prompt Engineering Guide**](https://docs.anthropic.com/en/docs/build-with-claude/prompt-engineering/overview) — official docs, pairs with #1 3. [**OpenAI Prompt Engineering**](https://platform.openai.com/docs/guides/prompt-engineering) — OpenAI's perspective 4. [**dair-ai Prompt Engineering Guide**](https://www.promptingguide.ai/) — academic-flavored, in-depth 5. [**Anthropic — Prompting Best Practices**](https://docs.anthropic.com/en/docs/build-with-claude/prompt-engineering/be-clear-and-direct) — be clear and direct ## 🛠 Hands-on Exercises > 🦙 **This stage defaults to Ollama gemma4:e4b** (cost-driven; $0/run). Prompt engineering is especially instructive on small models — they are sensitive to prompt quality, so you can clearly see how much each technique (system prompts, few-shot, CoT, refinement) improves output. Every exercise has Path A (Ollama, default) + Path B (Anthropic, optional). > > 💰 **Stage 2 budget estimate** (4 exercises, 3-5 runs each): **all local = $0**, **all haiku ≈ $0.20**, **all sonnet ≈ $0.60**. The few-shot classifier alone is 12 calls × 5 reps ≈ $0.30 haiku / $0.90 sonnet. Full budget: [`examples/README.en.md#recommended-llm-list`](../examples/README.en.md#recommended-llm-list). > > Full three-path trade-off in [`examples/README.en.md`](../examples/README.en.md#three-paths--default-is-ollama-cost-driven). ### Exercise 1: System Prompt Same user message, three different system prompts. Watch the personality / output format change. <details open> <summary>📋 <b>Starter code — Path A (local Ollama gemma4:e4b, default)</b> (copy to <code>practice_1.py</code>)</summary> ```python # Requires: pip install openai # Pre-req: ollama pull gemma4:e4b && ollama serve import sys if hasattr(sys.stdout, "reconfigure"): sys.stdout.reconfigure(encoding="utf-8", errors="replace") from openai import OpenAI client = OpenAI(base_url="http://localhost:11434/v1", api_key="ollama") SYSTEM_PROMPTS = { "Strict lawyer": "You are a precise contract lawyer. Cite statute numbers, avoid subjective adjectives.", "Kindergarten teacher": "You are a kind kindergarten teacher speaking to a 5-year-old. Use analogies, colloquial language, under 80 words.", "JSON machine": "Reply only in JSON. schema: {\"answer\": string, \"confidence\": float}", } USER_MSG = "Explain what a lease agreement is." outputs = {} for label, system in SYSTEM_PROMPTS.items(): # Ollama (OpenAI-compatible) puts system in the messages array (Anthropic uses system=) r = client.chat.completions.create( model="gemma4:e4b", max_tokens=200, messages=[ {"role": "system", "content": system}, {"role": "user", "content": USER_MSG}, ], ) outputs[label] = r.choices[0].message.content print(f"\n--- [{label}] ---") print(outputs[label]) # === Self-check === import json last_text = outputs["JSON machine"] assert "{" in last_text and "}" in last_text, "JSON-machine output should contain JSON braces" try: parsed = json.loads(last_text.strip().split("\n")[-1] if "\n" in last_text else last_text) assert "answer" in parsed, "schema expects an 'answer' field" except json.JSONDecodeError: pass # some models add prose around the JSON; tolerate that print(f"\n✅ Exercise 1 passed — same question, three different personas / formats / tones") ``` </details> <details> <summary>📋 <b>Starter code — Path B (Anthropic API, optional)</b> (copy to <code>practice_1_anthropic.py</code>)</summary> ```python # Requires: pip install anthropic import sys, json if hasattr(sys.stdout, "reconfigure"): sys.stdout.reconfigure(encoding="utf-8", errors="replace") import anthropic client = anthropic.Anthropic() SYSTEM_PROMPTS = { "Strict lawyer": "You are a precise contract lawyer. Cite statute numbers, avoid subjective adjectives.", "Kindergarten teacher": "You are a kind kindergarten teacher speaking to a 5-year-old. Use analogies, colloquial language, under 80 words.", "JSON machine": "Reply only in JSON. schema: {\"answer\": string, \"confidence\": float}", } USER_MSG = "Explain what a lease agreement is." outputs = {} for label, system in SYSTEM_PROMPTS.items(): # Anthropic uses `system=` parameter (not part of messages array) msg = client.messages.create(model="claude-haiku-4-5", max_tokens=200, system=system, messages=[{"role": "user", "content": USER_MSG}]) outputs[label] = msg.content[0].text print(f"\n--- [{label}] ---") print(outputs[label]) # Self-check (same JSON-shape assert; schema is cross-backend) json_output = outputs["JSON machine"] assert "{" in json_output and "}" in json_output print(f"\n✅ Exercise 1 passed (Anthropic)") ``` **Key difference**: Anthropic uses `system=` parameter; OpenAI/Ollama puts system in messages array. Claude follows system prompts more strictly than gemma4:e4b — the "Strict lawyer" persona will actually cite statute numbers. **Cost**: ~$0.003 / 3 personas. </details> ### Exercise 2: Few-Shot Pick a classification task. Run it 0-shot, then 3-shot. Measure accuracy difference. <details open> <summary>📋 <b>Starter code — Path A (local Ollama gemma4:e4b, default)</b> (copy to <code>practice_2.py</code>)</summary> ```python # Requires: pip install openai # Pre-req: ollama pull gemma4:e4b && ollama serve import sys if hasattr(sys.stdout, "reconfigure"): sys.stdout.reconfigure(encoding="utf-8", errors="replace") from openai import OpenAI client = OpenAI(base_url="http://localhost:11434/v1", api_key="ollama") # Sentiment classifier: positive / negative / neutral TEST_SET = [ ("This movie was amazing — I want to watch it again!", "positive"), ("Boring plot, awkward acting.", "negative"), ("This is a 2019 film.", "neutral"), ("Not sure how I feel about it, might think more.", "neutral"), ("Season 1 was great but season 2 fell apart.", "negative"), ("Left in a great mood — recommended!", "positive"), ] FEW_SHOT_EXAMPLES = """Examples: input: The steak at this place made me cry tears of joy. output: positive input: The waiter was rude. Never coming back. output: negative input: This shop is in New Taipei City. output: neutral """ def classify(text: str, *, use_few_shot: bool) -> str: prefix = FEW_SHOT_EXAMPLES + "\n" if use_few_shot else "" prompt = f"{prefix}input: {text}\noutput:" r = client.chat.completions.create( model="gemma4:e4b", max_tokens=10, messages=[{"role": "user", "content": prompt}], ) return r.choices[0].message.content.strip().splitlines()[0] def evaluate(use_few_shot: bool) -> tuple[int, int]: correct = 0 for text, label in TEST_SET: pred = classify(text, use_few_shot=use_few_shot) ok = label in pred print(f" {'✓' if ok else '✗'} [{label}] {text[:30]}... → '{pred}'") if ok: correct += 1 return correct, len(TEST_SET) print("=== 0-shot ===") c0, n = evaluate(use_few_shot=False) print(f"correct {c0}/{n} = {c0/n:.0%}") print("\n=== 3-shot ===") c3, _ = evaluate(use_few_shot=True) print(f"correct {c3}/{n} = {c3/n:.0%}") # === Self-check === print(f"\n✅ Exercise 2 passed — 0-shot {c0}/{n}, 3-shot {c3}/{n}") assert c3 >= c0, f"expected 3-shot ≥ 0-shot, got {c3} < {c0}" ``` </details> <details> <summary>📋 <b>Starter code — Path B (Anthropic API, optional)</b> (copy to <code>practice_2_anthropic.py</code>)</summary> ```python # Requires: pip install anthropic import sys if hasattr(sys.stdout, "reconfigure"): sys.stdout.reconfigure(encoding="utf-8", errors="replace") import anthropic client = anthropic.Anthropic() # Same TEST_SET / FEW_SHOT_EXAMPLES as Path A — only the classify() body changes: def classify(text: str, *, use_few_shot: bool) -> str: prefix = FEW_SHOT_EXAMPLES + "\n" if use_few_shot else "" msg = client.messages.create( model="claude-haiku-4-5", max_tokens=10, messages=[{"role": "user", "content": f"{prefix}input: {text}\noutput:"}], ) return msg.content[0].text.strip().splitlines()[0] # Rest of TEST_SET / FEW_SHOT_EXAMPLES / evaluate() stays identical to Path A ``` **Cost**: 12 calls ≈ $0.005. Claude is usually accurate at 0-shot already, so the few-shot lift is smaller than on gemma4:e4b — that contrast is the actual teaching point. </details> ### Exercise 3: CoT Pick a math word problem. Compare: - Plain prompt - Plain prompt + "Let's think step by step" - Plain prompt + worked example showing CoT <details open> <summary>📋 <b>Starter code — Path A (local Ollama gemma4:e4b, default)</b> (copy to <code>practice_3.py</code>)</summary> ```python # Requires: pip install openai # Pre-req: ollama pull gemma4:e4b && ollama serve import sys, re if hasattr(sys.stdout, "reconfigure"): sys.stdout.reconfigure(encoding="utf-8", errors="replace") from openai import OpenAI client = OpenAI(base_url="http://localhost:11434/v1", api_key="ollama") QUESTION = "Tom has 3 apples. He gives Sarah 1, then mom gives him 5 more, then he eats 2. How many does he have now?" ANSWER = 5 # 3 - 1 + 5 - 2 = 5 COT_EXAMPLE = """Example: Q: A chicken has 2 legs. 3 chickens and 1 person — how many legs total? A: Let me work through this step by step. 3 chickens × 2 legs = 6 legs. 1 person has 2 legs. Total 6 + 2 = 8 legs. The answer is 8. """ def ask(prompt: str) -> str: r = client.chat.completions.create( model="gemma4:e4b", max_tokens=300, messages=[{"role": "user", "content": prompt}], ) return r.choices[0].message.content def extract_number(text: str) -> int | None: """Pull the last number from the response as the answer.""" nums = re.findall(r"-?\d+", text) return int(nums[-1]) if nums else None out_a = ask(QUESTION) ans_a = extract_number(out_a) out_b = ask(QUESTION + "\nLet's think step by step.") ans_b = extract_number(out_b) out_c = ask(COT_EXAMPLE + "\n\nQ: " + QUESTION + "\nA:") ans_c = extract_number(out_c) for label, out, ans in [("A plain", out_a, ans_a), ("B +step-by-step", out_b, ans_b), ("C +CoT example", out_c, ans_c)]: print(f"\n--- [{label}] answer={ans} {'✓' if ans == ANSWER else '✗'} ---") print(out[:200]) # === Self-check === correct = sum(1 for a in (ans_a, ans_b, ans_c) if a == ANSWER) assert correct >= 1, f"at least 1 of 3 prompts should be correct, got {correct}/3" assert ans_b == ANSWER or ans_c == ANSWER, "B (step-by-step) or C (CoT example) must be correct — CoT is non-negotiable for small models" print(f"\n✅ Exercise 3 passed — {correct}/3 correct") ``` </details> <details> <summary>📋 <b>Starter code — Path B (Anthropic API, optional)</b> (copy to <code>practice_3_anthropic.py</code>)</summary> Same logic as Path A, just swap the client and `ask()`: ```python import anthropic client = anthropic.Anthropic() def ask(prompt: str) -> str: msg = client.messages.create(model="claude-haiku-4-5", max_tokens=300, messages=[{"role": "user", "content": prompt}]) return msg.content[0].text # Rest (QUESTION, ANSWER, COT_EXAMPLE, extract_number, 3 calls, assert) stays identical ``` **Claude typically gets 3/3 right** including the plain-prompt baseline — that contrast with gemma4:e4b (where CoT is essential) is the actual teaching point. **Cost**: 3 calls ≈ $0.002. </details> ### Exercise 4: Iterative Refinement Take a vague prompt, refine it 5 times. Track the iterations. Notice what changes improve quality. <details open> <summary>📋 <b>Starter code — Path A (local Ollama gemma4:e4b, default)</b> (copy to <code>practice_4.py</code>) — this exercise has no "right answer"; the point is observing the process</summary> ```python # Requires: pip install openai # Pre-req: ollama pull gemma4:e4b && ollama serve import sys if hasattr(sys.stdout, "reconfigure"): sys.stdout.reconfigure(encoding="utf-8", errors="replace") from openai import OpenAI client = OpenAI(base_url="http://localhost:11434/v1", api_key="ollama") # 5 iterations, each adds one constraint PROMPTS = { "v1 vague": "Write a paragraph about ReAct.", "v2 +audience": "Write a paragraph about ReAct for software engineers who know Python.", "v3 +format": "Write a paragraph about ReAct for software engineers who know Python. Under 100 words, single paragraph.", "v4 +example": "Write a paragraph about ReAct for software engineers who know Python. Under 100 words, single paragraph, ending with a concrete example (e.g. weather lookup).", "v5 +bans": "Write a paragraph about ReAct for software engineers who know Python. Under 100 words, single paragraph, ending with a concrete example (e.g. weather lookup). Avoid words like 'empower', 'leverage', 'intelligent'.", } outputs = {} for label, prompt in PROMPTS.items(): r = client.chat.completions.create( model="gemma4:e4b", max_tokens=200, messages=[{"role": "user", "content": prompt}], ) text = r.choices[0].message.content outputs[label] = text print(f"\n--- [{label}] ({len(text)} chars) ---") print(text) # === Self-check === v1_len, v5_len = len(outputs["v1 vague"]), len(outputs["v5 +bans"]) banned_words = ("empower", "leverage", "intelligent") v5_text_lower = outputs["v5 +bans"].lower() v5_has_banned = any(w in v5_text_lower for w in banned_words) assert v5_len > 0, "v5 must have output" assert not v5_has_banned, f"v5 should avoid banned words; got: {[w for w in banned_words if w in v5_text_lower]}" print(f"\n✅ Exercise 4 passed — v5 length {v5_len}, no banned words") print(f"💡 Observe: v1 ({v1_len} chars) is typically looser than v5 ({v5_len} chars); constraints tighten prompts") print("💡 The 5 dimensions: (1) target audience (2) format (3) length (4) example demand (5) banned words") ``` </details> <details> <summary>📋 <b>Starter code — Path B (Anthropic API, optional)</b> (copy to <code>practice_4_anthropic.py</code>)</summary> Same loop and PROMPTS as Path A, with Anthropic SDK: ```python import anthropic client = anthropic.Anthropic() outputs = {} for label, prompt in PROMPTS.items(): msg = client.messages.create(model="claude-haiku-4-5", max_tokens=200, messages=[{"role": "user", "content": prompt}]) outputs[label] = msg.content[0].text # Rest (length compare, banned-word assert) stays identical ``` **Cost**: 5 calls ≈ $0.002. **Claude's v1 is already coherent** so the v5 lift is smaller; gemma4:e4b makes the lift more dramatic. </details> ## 🎯 Curated Projects 4 categories, 9 projects in one table. **Pick by "Best for"; click through for depth on the repo / site.** | Category | Project | ⭐ | Best for | Why / Notes | |---|---|---|---|---| | **Academic / teaching-style guide**<br>(start here) | [dair-ai/Prompt-Engineering-Guide](https://github.com/dair-ai/Prompt-Engineering-Guide) | ⭐⭐⭐⭐⭐ | Reference book; look up a specific technique | Basics to advanced (CoT / ToT / ReAct / RAG) end to end, ★ 74k+, MIT | | | [PromptingGuide.ai](https://www.promptingguide.ai/) | ⭐⭐⭐⭐ | Phone reading; want runnable examples | Same content as dair-ai GitHub in website form + runnable examples | | | [NirDiamant/Prompt_Engineering](https://github.com/NirDiamant/Prompt_Engineering) | ⭐⭐⭐⭐ | Learn-by-running | 22 techniques (zero-shot → CoT → ReAct → constitutional), each in its own notebook, ★ 7k+. More hands-on than dair-ai (⚠️ NOASSERTION custom terms, research / non-commercial leaning) | | **Official cookbook** | [Anthropic Cookbook — Prompt patterns](https://github.com/anthropics/claude-cookbooks) | ⭐⭐⭐⭐⭐ | Advanced Claude prompting (prompt caching / multimodal) | Introduced in Stage 1; for this stage focus on `misc/prompt_caching.ipynb` and `multimodal/` | | | [GoogleCloudPlatform/generative-ai](https://github.com/GoogleCloudPlatform/generative-ai) | ⭐⭐⭐ | Google stack (PaLM / Gemini) users | Google Cloud's prompting cookbook; cross-vendor perspective | | **Inspiration collection**<br>(steal patterns, don't copy)| [f/awesome-chatgpt-prompts](https://github.com/f/awesome-chatgpt-prompts) | ⭐⭐⭐ | When you're stuck for ideas | Hundreds of "Act as a [role]..." prompts, ★ 162k+, CC0. **Take the pattern, rewrite — don't copy verbatim** | | **Production management**<br>(scale up)| [microsoft/prompt-engine](https://github.com/microsoft/prompt-engine) | ⭐⭐⭐ | Managing many prompts in production | TypeScript library, template + dialogue history management | | | [microsoft/promptflow](https://github.com/microsoft/promptflow) | ⭐⭐⭐ | Team apps needing eval | Visual prompt design + eval tooling, ★ 11k+ | | | [stanfordnlp/dspy](https://github.com/stanfordnlp/dspy) ⭐ **Stage 2 → 3 bridge** | ⭐⭐⭐⭐⭐ | After dair-ai, want to scale prompts | Treat prompts as code — define signature / module, compiler auto-optimizes, ★ 34k+, MIT. **A framework, not a tutorial; higher entry barrier; pair with dspy.ai official tutorial** | > 💡 **Suggested reading order**: dair-ai guide for theory → Anthropic Cookbook for Claude implementation → NirDiamant for hands-on → dspy when going to production. ## 🔭 Advanced: The Three Layers of Prompt → Context → Harness Engineering Engineering practice for LLM-powered systems can be divided into **three stack layers**. This is not about "one call vs. many calls." Each layer engineers a different object: - **Prompt Engineering** (this stage) = engineering **the string sent into the model** - **Context Engineering** (Stage 6) = engineering **what information goes into the context window on each call** — dynamically assembling RAG retrieval results, memory, tool definitions, and conversation history - **Harness Engineering** (Stage 7) = engineering **the execution and control layer around the model** — agent loops, retry, sandboxing, observability, deployment, and all other non-LLM code → The three layers are **orthogonal**: a one-call RAG app is still doing context engineering (the point is assembling context, not how many calls happen); a 50-call chatbot with no retrieval is still only doing prompt engineering. **Full three-layer lineage in this roadmap**: | Discipline | What is being engineered | Where this roadmap teaches it fully | |---|---|---| | **1. Prompt Engineering** | The string sent into the LLM itself (system prompt / few-shot / format) | **This stage (Stage 2)** | | **2. Context Engineering** | What information goes into the context window (RAG / memory / tool defs / history) | [Stage 6 — Context Engineering: RAG and Memory](06-memory-rag.en.md) | | **3. Harness Engineering** | The execution and control layer around the model (agent loop / retry / sandbox / observability) | [Stage 7 — Multi-Agent · Productionization](07-multi-agent-production.en.md) | > 💡 **Karpathy 2025-06**: context engineering is the delicate art of putting information that is **just useful for the next step** into the context window. > > 💡 **Simon Willison / Addy Osmani**: "coding agent = LLM + harness"; a harness is all the code that is not the model itself. [OpenAI also used the term "Harness Engineering" in February 2026](https://openai.com/index/harness-engineering). **You do not need to finish the latter two layers in this stage**. This section only gives you the direction so that Stage 6 / 7 feel like a continuation of the same lineage. Further reading (optional, for when you want to dig deeper): - [`Meirtz/Awesome-Context-Engineering`](https://github.com/Meirtz/Awesome-Context-Engineering) (★ 3k+) — comprehensive survey from prompt engineering to production agents - [`Windy3f3f3f3f/how-claude-code-works`](https://github.com/Windy3f3f3f3f/how-claude-code-works) (★ 2.4k+) — Claude Code internals, includes a context-engineering chapter ## ✅ Self-Check Before Stage 3 Can you: - [ ] Write a prompt with system message + user message + 3 example messages (few-shot) - [ ] Demonstrate CoT improving accuracy on a reasoning task - [ ] Iteratively refine a prompt 5 times tracking each version - [ ] Identify when prompting is the wrong tool (and tool use is needed) If yes → proceed to [Stage 3 — Tool Use & Agent Intro](03-tool-use-and-hello-agent.en.md). This is the most important stage — don't rush past prompts but also don't get stuck here.