AWS CloudTrail vs. AWS CloudWatch: A Defender's Guide

An S3 bucket holding customer data goes public at 02:14. The on-call engineer opens CloudWatch, finds clean dashboards, no alarms, normal latency, and concludes nothing happened. The bucket is still public. CloudWatch was never going to catch it, because making a bucket public is not a performance event. It is an API call, PutBucketAcl, and the only AWS service that recorded it, with the identity that made it, the source IP, and the timestamp, is CloudTrail. The engineer was reading the right console for the wrong question.

That mix-up is the reason this comparison matters. AWS CloudTrail and Amazon CloudWatch sound interchangeable and are constantly confused, but they answer different questions. CloudTrail answers "who did what to my account?" CloudWatch answers "how are my resources behaving?" One is your audit log, the other is your telemetry. This guide covers what each service actually records, how their pricing models differ, what their retention windows are, where they overlap, and how a SOC wires them together so an investigation has both the action and its effect.

What is AWS CloudTrail?

AWS CloudTrail records the API activity in your AWS account. Every action a user, role, or AWS service takes, through the console, the CLI, an SDK, or a direct API call, is captured as an event that names the identity, the action, the affected resource, the source IP, and the time. If something changed in your account, CloudTrail is the service that has the record of who changed it.

CloudTrail records events in three forms, and the distinction drives both cost and coverage:

• Event history gives you a viewable, searchable, immutable record of the past 90 days of management events in each Region, available the moment you open an account, at no charge. This is the default that most teams forget they already have.
• Trails deliver an ongoing copy of your events to an Amazon S3 bucket, with optional forwarding to CloudWatch Logs and Amazon EventBridge. A trail is how you keep events past 90 days and feed them into a SIEM or query them with Amazon Athena. One copy of management events to S3 is free from CloudTrail; you pay only for S3 storage.
• CloudTrail Lake is a managed data lake that stores events in columnar ORC format inside event data stores you can query with SQL, with retention configurable up to about 10 years. Ingestion, storage, and queries are billed.

The events themselves come in categories. Management events record control-plane operations: creating an IAM user, launching an EC2 instance, modifying a security group. These are free in one trail copy. Data events record data-plane operations at high volume: S3 object-level GetObject and PutObject, Lambda Invoke, DynamoDB item access. These are disabled by default and billed when enabled, because the volume is enormous. Network activity events capture VPC endpoint API activity. CloudTrail Insights is the one category that does analysis: it watches your management events and flags anomalies in API call rate and error rate, the kind of spike that accompanies a runaway script or an attacker enumerating an account. Insights is also billed.

The practical limit to remember: CloudTrail tells you an action happened. It does not, on its own, alert you in real time, build a dashboard, or measure whether the action degraded anything. For that you forward it onward, usually to CloudWatch.

What is Amazon CloudWatch?

Amazon CloudWatch monitors how your AWS resources and applications behave in real time. Where CloudTrail records actions, CloudWatch records state and performance: CPU utilization climbing on an EC2 instance, a Lambda function's error rate, request latency, queue depth, free disk space. It is the operational nervous system of an AWS environment, and it is built from a few distinct pieces.

• Metrics are the time-series numbers, CPU, network throughput, invocation counts, that AWS services publish automatically and that your own applications can publish as custom metrics.
• Alarms watch a metric against a threshold and act: notify through SNS, trigger Auto Scaling, run a Lambda. This is the real-time alerting CloudTrail lacks.
• Dashboards visualize metrics and logs in one place, sharable across accounts and Regions.
• CloudWatch Logs ingests, stores, and queries log data in log groups, with CloudWatch Logs Insights for fast interactive queries. This is where forwarded CloudTrail events often land so they can be alarmed on.
• The CloudWatch agent pulls system-level metrics and logs off EC2 instances and on-premises servers that AWS does not see by default, like memory and disk usage.

CloudWatch collects metrics at different resolutions, and the resolution is the thing teams get wrong. Basic monitoring publishes data every 5 minutes and is free for most services. Detailed monitoring publishes every 1 minute and is billed. High-resolution custom metrics go down to 1-second granularity. A 5-minute blind spot is fine for a batch job and far too coarse for a latency-sensitive API, so the choice of resolution is an actual design decision, not a default to ignore.

CloudWatch's blind spot is the mirror image of CloudTrail's. It sees that CPU spiked or an error rate jumped, but not the API call or the human identity behind the change that caused it. It measures effect, not cause.

CloudTrail vs. CloudWatch: the comparison

The cleanest way to hold the difference: CloudTrail is the account's flight recorder, CloudWatch is its instrument panel. One is consulted after the fact to reconstruct what was done; the other is watched live to see how things are running.

Two cells deserve emphasis. First, the free tiers are not symmetric. CloudTrail's 90-day event history is on automatically, so an investigation always has at least 90 days of management-event history even if nobody configured a trail. CloudWatch's free basic monitoring is coarse 5-minute data; the 1-minute and 1-second resolutions cost money. Second, the retention defaults run opposite ways and both surprise people. CloudWatch Logs keep data forever unless you set a retention policy, which is a quiet way to grow a bill. CloudTrail event history is only 90 days, so anything you need for a longer compliance window or a slow-burn investigation has to be captured by a trail before the window closes.

How retention actually works

Retention is where most "the logs aren't there" incidents start, and the two services follow different rules.

CloudTrail event history holds 90 days of management events, free, and that is the hard ceiling on the console search. Data events and Insights events never appear in event history at all. To keep anything beyond 90 days, or to keep data events, you need a trail writing to S3, where the events live as long as your S3 lifecycle policy allows, or a CloudTrail Lake event data store, which retains up to roughly 10 years. The failure mode is predictable: a breach is discovered four months after the initial access, the responder opens event history, and the founding events have already aged out because no trail was ever configured.

CloudWatch splits retention by data type. CloudWatch Logs default to "Never Expire," meaning a log group keeps every event indefinitely until you set a retention policy between one day and ten years. Metrics follow a rolling schedule: 1-minute data points are kept 15 days, 5-minute points for 63 days, and 1-hour points for 455 days (about 15 months), after which the metric expires if no new data arrives. So a metric you want to compare year over year has to be exported, and a log group you never set a policy on quietly accumulates cost.

The defender's takeaway is to set both deliberately. Turn on a trail to S3 (or Lake) on day one so management events survive past 90 days, and set explicit retention on CloudWatch log groups so they neither vanish too early nor bill forever.

Using CloudTrail and CloudWatch together

The two are not a choice. A real AWS deployment runs both, and the standard pattern wires them into one pipeline so a single event carries both its cause and its effect.

The mechanics are direct. You configure a CloudTrail trail to forward its events into a CloudWatch Logs log group. Once CloudTrail events are in CloudWatch Logs, you build metric filters that match specific patterns, an AuthorizationError, a root-account login, a security-group change, and turn each match into a CloudWatch metric. Then you set a CloudWatch alarm on that metric, so a sensitive API call moves from a passive record to a real-time page. This is exactly how the AWS-published CIS Benchmark alarms work: CloudTrail supplies the events, CloudWatch supplies the alerting CloudTrail lacks.

The combination is what closes the gap each service has alone. When that public S3 bucket from the opening goes live, CloudTrail holds the PutBucketAcl event with the identity and source IP that made the change, and a CloudWatch metric filter plus alarm on that event turns it into an alert the on-call actually sees, instead of a record nobody reads until the breach is public. CloudTrail without CloudWatch is a logbook no one opens until it is too late. CloudWatch without CloudTrail sees the symptom and never the cause. Run together, you get the action, the actor, and the alert in one chain, which is what makes the difference between cloud security you can audit and a dashboard that looked fine while the account was being taken apart. The same forwarded events feed your wider SIEM and turn quiet AWS misconfigurations into detections.

The bottom line

CloudTrail and CloudWatch get confused because both produce logs, but they answer opposite questions. CloudTrail is the audit record of API activity: who did what to the account, captured as immutable events with identity, action, resource, and time, kept 90 days free in event history and longer through trails or CloudTrail Lake. CloudWatch is the real-time view of how resources behave: metrics, logs, dashboards, and alarms, with the threshold-based alerting CloudTrail does not have.

Neither is sufficient alone. CloudWatch will tell you a resource is on fire and never who lit it; CloudTrail will tell you who lit it in a record nobody reads until the fire is out. The working pattern is to forward CloudTrail into CloudWatch Logs, turn the sensitive events into metric filters, and alarm on them, so a single pipeline carries the action, the actor, and the alert. For a defender, that pairing is the difference between reconstructing an incident from evidence and guessing at it from a dashboard that looked fine the whole time.