Getting Started with AgentCore Observability¶

Amazon Bedrock AgentCore Observability helps you trace, debug, and monitor agent performance in production environments. This guide will help you get started with implementing observability features in your agent applications.

What is AgentCore Observability?¶

AgentCore Observability provides:

Detailed visualizations of each step in the agent workflow
Real-time visibility into operational performance through CloudWatch dashboards
Telemetry for key metrics such as session count, latency, duration, token usage, and error rates
Rich metadata tagging and filtering for issue investigation
Standardized OpenTelemetry (OTEL)-compatible format for easy integration with existing monitoring stacks
Flexibility to be used with all AI agent frameworks and any large language model

Prerequisites¶

Before starting, make sure you have:

AWS Account with credentials configured (aws configure) with model access enabled to the Foundation Model you would like to use.
Python 3.10+ installed
Enable transaction search on Amazon CloudWatch. First-time users must enable CloudWatch Transaction Search to view Bedrock AgentCore spans and traces.
Add the OpenTelemetry library Include aws-opentelemetry-distro (ADOT) in your requirements.txt file.
Ensure that your framework is configured to emit traces (eg. strands-agents[otel] package), you may sometimes need to include <your-agent-framework-auto-instrumentor> # e.g., opentelemetry-instrumentation-langchain

AgentCore Observability offers two ways to configure monitoring to match different infrastructure needs: 1. AgentCore Runtime-hosted agents 2. Non-runtime hosted agents

Enabling Observability for AgentCore-Hosted Agents¶

AgentCore Runtime-hosted agents are deployed and executed directly within the AgentCore environment, providing automatic instrumentation with minimal configuration. This approach offers the fastest path to deployment and is ideal for rapid development and testing.

For a complete example please refer to this notebook

Step 1 : Create your Agent, shown below is an example with Strands Agents SDK:¶

To enable OTEL exporting, please note to install Strands Agents with otel extra dependencies:

pip install 'strands-agents[otel]'

Highlighted below are the steps to host a strands agent on AgentCore Runtime to get started:

##  Save this as strands_claude.py
from strands import Agent, tool
from strands_tools import calculator # Import the calculator tool
import argparse
import json
from bedrock_agentcore.runtime import BedrockAgentCoreApp
from strands.models import BedrockModel

app = BedrockAgentCoreApp()

# Create a custom tool
@tool
def weather():
    """ Get weather """ # Dummy implementation
    return "sunny"


model_id = "us.anthropic.claude-3-7-sonnet-20250219-v1:0"
model = BedrockModel(
    model_id=model_id,
)
agent = Agent(
    model=model,
    tools=[calculator, weather],
    system_prompt="You're a helpful assistant. You can do simple math calculation, and tell the weather."
)

@app.entrypoint
def strands_agent_bedrock(payload):
    """
    Invoke the agent with a payload
    """
    user_input = payload.get("prompt")
    print("User input:", user_input)
    response = agent(user_input)
    return response.message['content'][0]['text']

if __name__ == "__main__":
    app.run()

Step 2 : Deploy and invoke your Agent on AgentCore Runtime¶

Now that you created an agent ready to be hosted on AgentCore runtime, you can easily deploy it using the bedrock_agentcore_starter_toolkit package as shown below :

from bedrock_agentcore_starter_toolkit import Runtime
from boto3.session import Session
boto_session = Session()
region = boto_session.region_name

agentcore_runtime = Runtime()
agent_name = "strands_claude_getting_started"
response = agentcore_runtime.configure(
    entrypoint="strands_claude.py", # file created in Step 1
    auto_create_execution_role=True,
    auto_create_ecr=True,
    requirements_file="requirements.txt", # ensure aws-opentelemetry-distro exists along with your libraries required to run your agent
    region=region,
    agent_name=agent_name
)

launch_result = agentcore_runtime.launch()
launch_result

In these simple steps you deployed your strands agent on runtime with the Bedrock agentcore starter toolkit that automaticcally instruments your agent invocation using Open Telemetry. Now, you can invoke your agent using the command shown below and see the Traces, sessions and metrics on GenAI Obsrvability dashboard on Amazon Cloudwatch.

invoke_response = agentcore_runtime.invoke({"prompt": "How is the weather now?"})
invoke_response

Enabling Observability for Non-AgentCore-Hosted Agents¶

For agents running outside of the AgentCore runtime, deliver the same monitoring capabilities for agents deployed on your own infrastructure, allowing consistent observability regardless of where your agents run. Additionally, you would need to follow the steps below to configure the environment variables needed to observe your agents.

For a complete example please refer to this notebook

Step 1: Configure AWS Environment Variables¶

export AWS_ACCOUNT_ID=<account id>
export AWS_DEFAULT_REGION=<default region>
export AWS_REGION=<region>
export AWS_ACCESS_KEY_ID=<access key id>
export AWS_SECRET_ACCESS_KEY=<secret key>

Step 2: Configure CloudWatch logging:¶

Create a log group and log stream for your agent in Amazon CloudWatch which you can use to configure below environment variables.

Step 3: Configure OpenTelemetry Environment Variables¶

export AGENT_OBSERVABILITY_ENABLED=true # Activates the ADOT pipeline
export OTEL_PYTHON_DISTRO=aws_distro # Uses AWS Distro for OpenTelemetry
export OTEL_PYTHON_CONFIGURATOR=aws_configurator # Sets AWS configurator for ADOT SDK
export OTEL_EXPORTER_OTLP_PROTOCOL=http/protobuf # Configures export protocol
export  OTEL_EXPORTER_OTLP_LOGS_HEADERS=x-aws-log-group=<YOUR-LOG-GROUP>,x-aws-log-stream=<YOUR-LOG-STREAM>,x-aws- metric-namespace=<YOUR-NAMESPACE>
# Directs logs to CloudWatch groups
export OTEL_RESOURCE_ATTRIBUTES=service.name=<YOUR-AGENT-NAME> # Identifies your agent in observability data

Replace <YOUR-AGENT-NAME> with a unique name to identify this agent in the GenAI Observability dashboard and logs.

Step 4: Create an agent locally¶

# Create agent.py -  Strands agent that is a weather assistant
from strands import Agent
from strands_tools import http_request

# Define a weather-focused system prompt
WEATHER_SYSTEM_PROMPT = """You are a weather assistant with HTTP capabilities. You can:

1. Make HTTP requests to the National Weather Service API
2. Process and display weather forecast data
3. Provide weather information for locations in the United States

When retrieving weather information:
1. First get the coordinates or grid information using https://api.weather.gov/points/{latitude},{longitude} or https://api.weather.gov/points/{zipcode}
2. Then use the returned forecast URL to get the actual forecast

When displaying responses:
- Format weather data in a human-readable way
- Highlight important information like temperature, precipitation, and alerts
- Handle errors appropriately
- Convert technical terms to user-friendly language

Always explain the weather conditions clearly and provide context for the forecast.
"""

# Create an agent with HTTP capabilities
weather_agent = Agent(
    system_prompt=WEATHER_SYSTEM_PROMPT,
    tools=[http_request],  # Explicitly enable http_request tool
)

response = weather_agent("What's the weather like in Seattle?")
print(response)

Step 5: Run your agent with automatic instrumentation command¶

With aws-opetelemetry-distro in your requirements.txt, opentelemetry-instrument command will:

Load your OTEL configuration from your environment variables
Automatically instrument Strands, Amazon Bedrock calls, agent tool and databases, and other requests made by agent
Send traces to CloudWatch
Enable you to visualize the agent's decision-making process in the GenAI Observability dashboard

opentelemetry-instrument python agent.py

You can now view your traces, sessions and metrics on GenAI Observability Dashboard on Amazon CloudWatch with the help of YOUR-AGENT-NAME that you configured in your environment variable.

To correlate traces across multiple agent runs, you can associate a session ID with your telemetry data using OpenTelemetry baggage:

from opentelemetry import baggage, context
ctx = baggage.set_baggage("session.id", session_id)

Run the session-enabled version following command, complete implementation provided in the notebook:

opentelemetry-instrument python strands_travel_agent_with_session.py --session-id "user-session-123"

AgentCore Observability on GenAI Observability on Amazon CloudWatch¶

After implementing observability, you can view the collected data in CloudWatch:

Bedrock AgentCore Overview on GenAI Observability dashboard¶

Open the GenAI Observability on CloudWatch console
You are able to view the data related to model invocations and agents on Bedrock AgentCore on the dashboard.
In the Bedrock Agentcore tab you are able to see Agents View, Sessions View and Traces View.
Agents View lists all your Agents that are on and not on runtime, you can also click on the agent and view further details like runtime metrics, sessions and traces specific to an agent.
In the Sessions View tab, you can navigate across all the sessions associated with agents.
In the Trace View tab, you can look into the traces and span information for agents. Also explore the trace trajectory and timeline by clicking on a trace.

View Logs in CloudWatch¶

Open the CloudWatch console
In the left navigation pane, expand Logs and select Log groups
Search for your agent's log group:
Standard logs (stdout/stderr) Location: /aws/bedrock-agentcore/runtimes/<agent_id>-<endpoint_name>/[runtime-logs] <UUID>
OTEL structured logs: /aws/bedrock-agentcore/runtimes/<agent_id>-<endpoint_name>/runtime-logs

View Traces and Spans¶

Open the CloudWatch console
Select Transaction Search from the left navigation
Location: /aws/spans/default
Filter by service name or other criteria
Select a trace to view the detailed execution graph

View Metrics¶

Open the CloudWatch console
Select Metrics from the left navigation
Browse to the bedrock-agentcore namespace
Explore the available metrics

Best Practices¶

Start Simple, Then Expand - The default observability provided by AgentCore captures most critical metrics automatically, including model calls, token usage, and tool execution.
Configure for Development Stage - Tailor your observability configuration to match your current development phase and progressively adjust.
Use Consistent Naming - Establish naming conventions for services, spans, and attributes from the start
Filter Sensitive Data - Prevent exposure of confidential information by filtering sensitive data from observability attributes and payloads.
Set up alerts - Configure CloudWatch alarms to notify you of potential issues before they impact users