Blog Post #37: The Self-Improving Agent: Implementing a “Critic” Tool for Self-Correction

by

No writer publishes their first draft. They write, they pause, and they critique their own work: “Is this sentence clear? Is the tone right? Could this be more persuasive?” This iterative loop of creation and self-criticism is fundamental to producing high-quality work.

What if we could teach our AI agents to do the same?

So far, our agents execute a plan and assume the result is correct. A ReAct agent might self-correct if a tool fails with an error, but it doesn’t question whether a successful output was actually good enough. A Plan-and-Execute agent just follows its initial plan, even if that plan was suboptimal.

To build truly advanced agents, we can implement a powerful pattern where the agent can critique and refine its own work before finishing. We do this by creating a Critic Tool.

The Theory: Creator vs. Critic Personas

A critic tool is a special kind of tool that doesn’t interact with the outside world (like a search engine or API). Instead, it calls back to an LLM, but with a different, specialized prompt or “persona.” This forces the LLM to switch “hats.”

  1. The “Creator” Persona: This is the agent’s default mode. When prompted to generate a plan or a piece of text, it’s optimized for fluency, creativity, and completing the task.
  2. The “Critic” Persona: When the agent calls the critic tool, the tool wraps the agent’s work in a new, highly analytical prompt. This prompt instructs the LLM to act as a skeptical expert, whose only job is to find flaws, identify weaknesses, and suggest concrete improvements based on a set of principles.

By separating these two modes of thinking, the agent can produce a more robust and refined final output, much like a human would.

Building a Self-Correcting Agent

Let’s build an agent whose job is to draft a marketing email. It will have two tools: one to write the first draft, and a second to critique that draft.

Step 1: The “Creator” and “Critic” Tools

We’ll create two custom tools. For this example, both will call an LLM internally, but with very different instructions.

# tools.py
from langchain_openai import ChatOpenAI
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.output_parsers import StrOutputParser
from langchain.tools import tool

# Initialize a single LLM for both tools to use
llm = ChatOpenAI(model="gpt-4o", temperature=0.7)

@tool
def email_drafting_tool(topic: str, audience: str) -> str:
    """
    Generates a compelling, first-draft of a marketing email on a given topic
    for a specific audience.
    """
    print("--- DRAFTING EMAIL ---")
    prompt = ChatPromptTemplate.from_template(
        "You are a marketing assistant. Write a short, engaging first draft for a marketing "
        "email about '{topic}' to an audience of '{audience}'. Keep it under 100 words."
    )
    chain = prompt | llm | StrOutputParser()
    return chain.invoke({"topic": topic, "audience": audience})

@tool
def email_critic_tool(draft_email: str) -> str:
    """
    Analyzes a draft email and provides a critique with concrete, actionable
    suggestions for improvement. It must check for clarity, tone, and the
    strength of the call-to-action.
    """
    print("--- CRITIQUING EMAIL ---")
    prompt = ChatPromptTemplate.from_messages([
        ("system", "You are a world-class marketing copy editor. Your feedback is sharp, "
                   "brutally honest, but always constructive. Your goal is to provide a numbered "
                   "list of actionable suggestions to improve the draft you are given. Focus on "
                   "clarity, engagement, and the call-to-action."),
        ("human", "Please critique the following draft email:\n\n---\n{draft}\n---")
    ])
    chain = prompt | llm | StrOutputParser()
    return chain.invoke({"draft": draft_email})

Step 2: The Plan-and-Execute Agent

This self-correction workflow is a perfect fit for a Plan-and-Execute agent, as it follows a clear, predictable sequence of steps.

A user query like, “Draft and refine an email about our new AI-powered calendar app for busy professionals,” would cause our planner to generate a plan similar to this:

  1. Step 1: Use the email_drafting_tool to generate a first draft about the new AI calendar app for busy professionals.
  2. Step 2: Pass the draft from Step 1 to the email_critic_tool to get feedback and suggestions for improvement.
  3. Step 3: Analyze the critique from Step 2 and the original draft, then rewrite the email to create a final, improved version.
  4. Step 4: Present the final, revised email as the answer.

Here’s how we’d implement the agent:

# main.py
from langchain_experimental.plan_and_execute import PlanAndExecute, load_agent_executor, load_chat_planner
from langchain_openai import ChatOpenAI
from dotenv import load_dotenv
from tools import email_drafting_tool, email_critic_tool

# --- SETUP ---
load_dotenv()
tools = [email_drafting_tool, email_critic_tool]
model = ChatOpenAI(model="gpt-4o", temperature=0)

# --- THE PLAN-AND-EXECUTE AGENT ---
planner = load_chat_planner(model)
executor = load_agent_executor(model, tools, verbose=True)
agent = PlanAndExecute(planner=planner, executor=executor, verbose=True)

# --- RUN THE AGENT ---
if __name__ == "__main__":
    query = "Draft and refine a marketing email about our new AI-powered calendar app called 'ChronosAI' for an audience of busy professionals."
    
    print(f"Running self-correcting agent with query:\n'{query}'")
    agent.invoke(query)

Seeing it in Action

When you run this code, the verbose=True output will be fascinating. You will see:

  1. The Plan being created, with steps for drafting and critiquing.
  2. The Executor calling the email_drafting_tool.
  3. The Observation of the first draft.
  4. The Executor then calling the email_critic_tool with that draft as input.
  5. The Observation of the critique (e.g., “1. The opening is weak. 2. The call-to-action is not specific enough…”).
  6. The agent’s final reasoning step, where it combines the draft and the critique to generate a much-improved final version.

Conclusion

The self-correction pattern is a powerful leap forward in agent design. By creating a “critic” tool, you give your agent the ability to reflect on its own work, identify weaknesses, and make targeted improvements.

This leads to:

  • Significantly higher-quality and more reliable outputs.
  • More robust reasoning in complex, multi-step tasks.
  • A workflow that more closely mimics a human expert’s creative process.

By building agents that can critique themselves, we are teaching them not just to do, but to do well. This is a crucial step on the path toward more autonomous, capable, and trustworthy AI systems.

Author

Debjeet Bhowmik

Experienced Cloud & DevOps Engineer with hands-on experience in AWS, GCP, Terraform, Ansible, ELK, Docker, Git, GitLab, Python, PowerShell, Shell, and theoretical knowledge on Azure, Kubernetes & Jenkins. In my free time, I write blogs on ckdbtech.com

Leave a Comment