AI Agents with Python 2026 (Part 2): How AI Agents Work — Models, Tools, Memory and Reasoning

AI Agents with Python 2026 (Part 2): How AI Agents Work — Models, Tools, Memory and Reasoning

In Part 1 of AI Agents with Python 2026, we answered a fundamental question:

What are AI agents?

We explored the difference between traditional software, chatbots, Generative AI, and goal-driven AI systems. We also introduced the major components of an AI agent, including models, tools, memory, knowledge, actions, and guardrails.

Now it’s time to look under the hood.

How does an AI agent actually work?

How does it decide what to do?

How does it know which tool to use?

How does it remember information?

And how does it determine when a task is complete?

Welcome to Part 2 of AI Agents with Python 2026 — From Automation to Autonomous Systems.

Code Repository

The concepts in this series are supported by practical Python projects.

You can find the source code for Part 2 and the other projects in the AI Agents with Python 2026 GitHub repository.

View the GitHub Repository

For this article, see the:

02-how-ai-agents-work folder.

The Architecture of an AI Agent

At a high level, an AI agent can be thought of as a system that combines several components:

Goal → Model → Decision → Tool → Observation → Next Decision → Result

A simplified architecture looks like this:

USER / APPLICATION
                        │
                        ▼
                 ┌──────────────┐
                 │     GOAL     │
                 └──────┬───────┘
                        │
                        ▼
                 ┌──────────────┐
                 │  AI MODEL    │
                 │ LLM / Model  │
                 └──────┬───────┘
                        │
                        ▼
                 ┌──────────────┐
                 │   DECISION   │
                 │ / PLANNING   │
                 └──────┬───────┘
                        │
              ┌─────────┼─────────┐
              ▼         ▼         ▼
           TOOL 1    TOOL 2    TOOL 3
              │         │         │
              └─────────┼─────────┘
                        │
                        ▼
                  OBSERVATION
                        │
                        ▼
                 ┌──────────────┐
                 │ NEXT ACTION? │
                 └──────┬───────┘
                        │
                 ┌──────┴──────┐
                 │             │
                YES            NO
                 │             │
                 ▼             ▼
              Continue       Result

This is a simplified representation.

Real-world agent architectures can be considerably more sophisticated, but understanding this basic structure gives us a strong foundation.

1. The Goal

Everything begins with a goal.

An agent needs to know what it is trying to accomplish.

For example:

“Analyze this month’s sales data and identify the three products with the largest decline.”

The goal provides direction.

Without a clear objective, an agent may generate irrelevant responses or perform unnecessary actions.

A good goal should define:

• What needs to be accomplished
• What information is relevant
• What output is expected
• What constraints apply

For example:

Poor goal:

“Analyze the data.”

Better goal:

“Analyze the sales dataset, identify products whose monthly sales declined by more than 10%, and produce a summary table.”

The second instruction gives the agent a much clearer target.

2. The AI Model

The model is the reasoning and language component of many modern AI agents.

Large Language Models (LLMs) are particularly useful because they can:

• Interpret natural language
• Understand instructions
• Analyze context
• Generate structured responses
• Select among available actions
• Produce tool arguments

However, the model is only one part of the agent.

Think of it like the brain of a larger system.

A brain without eyes, hands, memory, or access to the outside world has limited ability to interact with its environment.

Similarly, an LLM becomes much more useful when connected to appropriate tools and data.

3. Instructions and Context

The model needs information about its role and the task.

This may include:

• System instructions
• User instructions
• Available tools
• Previous messages
• Retrieved information
• Current task state
• Rules and constraints

For example, an agent designed to assist a company’s IT department might receive instructions such as:

You are an IT support assistant.

Your responsibilities:
1. Diagnose common technical issues.
2. Search the approved knowledge base.
3. Provide troubleshooting instructions.
4. Escalate high-risk issues to a human technician.

Do not modify production systems without authorization.

These instructions help define the agent’s behavior.

4. Tools

An AI model can generate information, but tools allow an agent to interact with external systems.

Tools can include:

• Web search
• APIs
• Databases
• File systems
• Calculators
• Python functions
• Email systems
• Calendar systems
• Business applications

Imagine an AI assistant that needs to determine the current weather.

The model’s internal knowledge may not provide the current conditions.

Instead, the agent can call an appropriate weather service.

The workflow becomes:

User Request
     ↓
AI Model
     ↓
Weather Tool
     ↓
Current Weather Data
     ↓
AI Model
     ↓
Response

The tool provides information the model could not reliably obtain on its own.

Tool Calling

One of the important concepts in agentic AI is tool calling.

Instead of simply generating a text response, the model can determine that a particular tool is required.

For example:

tools = [
    "search_database",
    "calculate",
    "get_weather"
]

Suppose the user asks:

“What is the weather in Accra?”

The model may determine that the get_weather tool is appropriate.

The application then executes the function and provides the result back to the model.

A simplified Python representation could look like this:

def get_weather(city):
    # Call an approved weather service
    return weather_data

The important principle is that the model decides what it needs, while the application controls what it is actually allowed to execute.

That distinction is critical for security.

Models Shouldn’t Have Unlimited Access

Imagine giving an AI model unrestricted access to:

• Your database
• Your email
• Your file system
• Your financial accounts
• Your operating system

That would create enormous risks.

Instead, agents should receive only the permissions they require.

This follows a familiar cybersecurity principle:

Principle of Least Privilege

Give a system the minimum permissions necessary to perform its intended function.

For example:

An AI customer-support agent may need permission to read customer information.

It may not need permission to delete customer records.

This principle will become especially important when we discuss agent security later in the series.

5. Memory

Memory allows an agent to retain relevant information.

Without memory, every interaction can appear to be independent.

Consider a personal AI assistant.

You tell it:

“My preferred programming language is Python.”

Later you ask:

“Recommend a programming project for me.”

A system with appropriate memory could use your previous preference to make the response more relevant.

Memory can exist at different levels.

Short-Term Memory

Short-term memory refers to information available within the current interaction or task.

For example:

User:
My budget is GHS 5,000.

User:
Show me laptops.

Agent:
I'll focus on options around your GHS 5,000 budget.

The agent needs the earlier information to interpret the second request correctly.

Long-Term Memory

Long-term memory involves retaining information beyond a single interaction.

For example:

User Preference:
Prefers Python tutorials.

Previous Project:
Built an AI study assistant.

Current Goal:
Learning AI agents.

An application can store such information in an appropriate database or memory system.

However, storing personal information introduces privacy considerations.

Developers need to determine:

• What should be remembered?
• How long should it be stored?
• Where should it be stored?
• Who can access it?
• How can users delete it?

Memory should be designed intentionally.

6. Knowledge

Memory and knowledge are related but not identical.

Memory often refers to information about the interaction, user, or task state.

Knowledge retrieval refers to obtaining relevant information from external sources.

For example, an AI agent for a company might need access to:

• Employee policies
• Product documentation
• Technical manuals
• Internal procedures
• Frequently asked questions

Instead of placing every document directly into the model’s prompt, the system can retrieve relevant information when needed.

This leads us to an important AI architecture:

Retrieval-Augmented Generation (RAG)

A simplified RAG workflow looks like:

User Question
      ↓
Retrieve Relevant Information
      ↓
Knowledge Source
      ↓
Relevant Context
      ↓
AI Model
      ↓
Answer

RAG becomes particularly powerful when combined with agents.

An agent can determine that it needs information, retrieve the appropriate content, and then use that information to continue the task.

We’ll explore this in detail in Part 6.

7. Reasoning and Planning

One of the most interesting aspects of AI agents is their ability to break a larger task into smaller actions.

Suppose you ask:

“Prepare a report comparing three cloud platforms for a small business.”

The task could involve:

1. Identify the platforms.
2. Gather pricing information.
3. Compare features.
4. Analyze advantages and disadvantages.
5. Organize the findings.
6. Produce the report.

A system may need to determine which step should happen next based on the information currently available.

This is where planning becomes important.

Planning Does Not Always Mean Complex Reasoning

It is important to avoid thinking that every agent needs an elaborate autonomous planning system.

Sometimes the workflow can be simple:

Input
 ↓
Call API
 ↓
Format Result
 ↓
Return Response

Other tasks may require:

Goal
 ↓
Plan
 ↓
Research
 ↓
Analyze
 ↓
Verify
 ↓
Generate
 ↓
Review
 ↓
Complete

The appropriate architecture depends on the problem.

Again:

Use the simplest architecture that can reliably solve the problem.

8. Observation

After an agent performs an action, it needs to know what happened.

This is the observation stage.

For example:

Agent:
Search for information about Python.

Tool:
Returns 20 search results.

Agent:
Analyze the results and determine which are relevant.

The returned information becomes an observation that the agent can use to determine its next step.

This creates a loop:

Think → Act → Observe → Decide → Act Again

The Agent Loop

Let’s put everything together.

┌──────────────┐
             │     GOAL     │
             └──────┬───────┘
                    ↓
             ┌──────────────┐
             │   CONTEXT    │
             └──────┬───────┘
                    ↓
             ┌──────────────┐
             │  AI MODEL    │
             └──────┬───────┘
                    ↓
             ┌──────────────┐
             │   DECISION   │
             └──────┬───────┘
                    ↓
             ┌──────────────┐
             │     TOOL     │
             └──────┬───────┘
                    ↓
             ┌──────────────┐
             │  OBSERVATION │
             └──────┬───────┘
                    ↓
             ┌──────────────┐
             │   COMPLETE?  │
             └───┬──────┬───┘
                 │      │
                NO     YES
                 │      │
                 ↓      ↓
              Continue  Result

This loop is central to understanding agentic systems.

A Practical Example: Research Agent

Let’s design a simple research agent.

The user says:

“Research the benefits of solar energy for small businesses in Ghana and prepare a summary.”

The agent might perform the following workflow.

Step 1 — Understand

Identify:

• Topic
• Geographic focus
• Target audience
• Required output

Step 2 — Plan

Determine what information is needed.

Step 3 — Retrieve

Use approved information sources.

Step 4 — Analyze

Review the retrieved information.

Step 5 — Organize

Group findings into categories.

Step 6 — Generate

Produce the requested summary.

Step 7 — Review

Check whether the response satisfies the original goal.

Step 8 — Return

Provide the final result.

This is an example of a goal-driven workflow.

What Happens When a Tool Fails?

Real systems fail.

An API may be unavailable.

A database may timeout.

A search may return poor results.

A tool may return unexpected data.

A professional agent needs to handle these situations.

For example:

try:
    result = get_data()
except Exception as error:
    print("Tool failed:", error)

In production systems, error handling should be much more sophisticated.

The agent may:

• Retry the operation
• Use another approved tool
• Ask the user for clarification
• Escalate to a human
• Stop safely

Failure handling is part of agent design.

Human-in-the-Loop

Not every decision should be automated.

Some actions require human approval.

For example:

AI Agent
   ↓
Prepare financial transaction
   ↓
Human Approval
   ↓
Execute Transaction

This approach is known as Human-in-the-Loop (HITL).

It is particularly useful when an agent can perform high-impact actions.

Examples include:

• Financial transactions
• Deleting data
• Sending sensitive communications
• Changing production infrastructure
• Approving important decisions

Human oversight can significantly reduce the consequences of agent errors.

Deterministic vs Agentic Workflows

There is another important distinction.

A deterministic workflow follows predefined steps:

Step 1 → Step 2 → Step 3 → Step 4

An agentic workflow can make decisions about what to do next:

Goal
 ↓
Decision
 ↓
Action
 ↓
Observation
 ↓
Next Decision

Neither approach is automatically better.

For predictable tasks, deterministic workflows can be easier to test, monitor, and secure.

For tasks involving uncertainty and changing conditions, an agentic approach may provide greater flexibility.

Good engineering means choosing appropriately.

Where Python Fits

Python can act as the orchestration layer connecting the components of an AI agent.

A simplified architecture could look like:

Python Application
       │
       ├── AI Model
       │
       ├── Tools
       │
       ├── APIs
       │
       ├── Database
       │
       ├── Memory
       │
       └── RAG / Knowledge Base

Python can coordinate these components and implement the business logic around them.

This is one reason Python is such a useful language for AI development.

A Simple Python Agent Architecture

We can represent a very basic agent conceptually like this:

class SimpleAgent:

    def __init__(self, model, tools):
        self.model = model
        self.tools = tools

    def run(self, goal):
        context = goal

        while True:
            decision = self.model.decide(
                context,
                self.tools
            )

            if decision["action"] == "finish":
                return decision["result"]

            tool = self.tools[decision["tool"]]

            result = tool(**decision["arguments"])

            context = {
                "goal": goal,
                "previous_result": result
            }

This is intentionally simplified.

A production-grade agent would need much more:

• Validation
• Authentication
• Error handling
• Logging
• Tool permissions
• State management
• Security controls
• Observability
• Cost management
• Loop limits

But the example illustrates the basic idea:

Receive a goal → decide → use a tool → observe → continue or finish.

Why Agent Loops Need Limits

Imagine an agent that continuously decides it needs another action.

It could potentially keep running indefinitely.

That can lead to:

• Unnecessary API costs
• Slow responses
• Resource consumption
• Repeated actions
• Unexpected behavior

Developers should therefore consider controls such as:

• Maximum number of iterations
• Execution time limits
• Tool-call limits
• Budget limits
• Approval requirements

For example:

MAX_STEPS = 10

The system can stop the agent if it exceeds the permitted number of steps.

Simple safeguards like this can prevent runaway workflows.

Security Is Part of the Architecture

Agent security should not be added at the end.

It should be considered from the beginning.

Important areas include:

Authentication

Who is allowed to use the agent?

Authorization

What is the agent allowed to access?

Input Validation

What information can users provide?

Tool Permissions

Which functions can the agent execute?

Data Protection

What sensitive information can the agent access?

Logging

What actions did the agent perform?

Human Approval

Which actions require a person to approve them?

These questions become increasingly important as agents become more capable.

The Agent Stack

You can think of an AI agent as a technology stack:

┌──────────────────────────────┐
│          USER / GOAL         │
├──────────────────────────────┤
│        AGENT LOGIC           │
├──────────────────────────────┤
│       AI MODEL / LLM         │
├──────────────────────────────┤
│       TOOLS & FUNCTIONS      │
├──────────────────────────────┤
│       MEMORY & STATE         │
├──────────────────────────────┤
│       KNOWLEDGE / RAG        │
├──────────────────────────────┤
│       APIs & DATABASES       │
├──────────────────────────────┤
│ SECURITY / GUARDRAILS / LOGS │
└──────────────────────────────┘

Each layer has a purpose.

Understanding these layers will make it easier to design and troubleshoot AI agents.

A Beginner’s Mental Model

If you’re just starting with AI agents, remember these six questions:

1. What is the goal?

What should the system accomplish?

2. What does the model need to understand?

What context and instructions are required?

3. What tools are available?

What external capabilities can the agent use?

4. What information does it need?

Where will the knowledge come from?

5. What actions can it take?

What is the agent actually allowed to do?

6. What happens if something goes wrong?

How will the system handle failure?

If you can answer these questions, you’re already thinking like an AI agent developer.

Your Practical Exercise

Before moving to the next part, design an agent on paper.

Choose a problem such as:

AI Study Assistant

Then define:

Goal:
Help students understand study materials.

Model:
Language model.

Tools:
Document reader and calculator.

Knowledge:
Course materials.

Memory:
Current study session.

Actions:
Generate explanations and practice questions.

Guardrails:
Do not fabricate information from unavailable course materials.

Human Oversight:
Student reviews generated material.

You have now created a basic agent architecture.

What’s Coming in Part 3?

Now that we understand the architecture, it’s time to write some code.

In Part 3, we’ll build:

Your First Simple AI Agent with Python

We’ll move from diagrams and concepts to implementation.

You’ll learn how to:

• Set up an AI agent project
• Connect Python to an AI model
• Define an agent goal
• Create a simple tool
• Allow the agent to use the tool
• Process the result
• Return a final response

We’ll keep the first implementation intentionally simple.

The objective is to understand how the pieces connect, not to build a complicated production system immediately.

Final Thoughts

An AI agent is not simply a chatbot with a fancy name.

It is a system designed around a goal and equipped with capabilities that allow it to interact with information and tools.

The basic architecture can be summarized as:

Goal → Model → Decision → Tool → Observation → Next Action → Result

Around that core architecture, we add:

Memory + Knowledge + Security + Guardrails + Human Oversight

Once you understand these components, agentic AI becomes much less mysterious.

It becomes a software engineering problem.

And that’s exactly where Python becomes powerful.

We’ve learned what AI agents are.

Now we understand how they work.

Next, we’re going to build one.

See you in Part 3.

AI Agents with Python 2026

Wednesdays → AI Agents with Python 2026

Mondays → Technology, AI, cybersecurity, broadcasting technology, innovation, and other tech-related articles

Next:
Part 3 — Your First Simple AI Agent with Python