What Is Log Rotation? How It Works and Why It Matters

A disk fills up at 2 a.m. The web server stops serving. The auth daemon cannot write its log, so it stops logging. For the next four hours, until someone clears space, there is no record of who hit the box or who logged in. When the responder shows up the next morning, the most important window of the incident is a hole. The cause was not the attacker. It was a log file that grew without limit because nobody set up log rotation.

Log rotation is the routine that keeps a single log file from growing forever. When the file hits a size or age threshold, the system renames it, starts a fresh file under the original name, and then compresses, archives, or deletes the old one on a schedule. It is plumbing, not a feature anyone gets excited about, and that is exactly why it gets skipped until a full disk takes a service down or a forensic timeline turns up empty. This guide covers what log rotation is, how it works, the rotation strategies, the tools that do it, what to do with the old files, and why it is a blue-team concern and not just a sysadmin chore.

What is log rotation?

Log rotation is the process of controlling the size and number of log files by periodically closing the active file, starting a new one, and managing the old copies. The active log, say /var/log/syslog, keeps growing as the system appends to it. Rotation breaks that single ever-growing file into a sequence of bounded files: the current one plus a fixed number of older, dated or numbered copies.

The trigger is a threshold. Rotation fires when the file crosses a configured limit: a maximum size (rotate at 100 MB), an age (rotate daily, weekly, or monthly), or sometimes a record count. When the threshold is hit, the rotation routine renames the current file and opens a new empty one with the original name, so the application keeps logging to the same path without interruption. The rename, the new-file creation, and the handoff happen in seconds.

Two naming conventions dominate. The first appends a timestamp: mylogfile.log becomes mylogfile-20260620.log and a fresh mylogfile.log takes its place. The second uses a numeric suffix and shifts the numbers on each rotation: the current log rotates to .1, the old .1 becomes .2, and so on. Microsoft SQL Server uses this numeric scheme for its error log, cycling ERRORLOG, ERRORLOG.1, ERRORLOG.2, and up. Either way, the result is the same: a finite set of files instead of one that grows until the disk is gone.

Why log rotation matters

Three things break when logs are not rotated, and all three land on a defender's desk.

A full disk takes the service down and the logging with it. An unbounded log file will eventually consume every free byte on its volume. When the disk fills, the service that writes the log usually fails, and so does anything else sharing that volume. Worse for an investigator, logging itself stops. The window where the disk was full is a blind spot in every log on that host, and it tends to happen during exactly the kind of high-volume event, a flood of failed logins, a scan, a crash loop, that you most want a record of. Log rotation caps the total footprint so a noisy day cannot fill the disk.

Huge files are slow to work with. A 40 GB single log file is painful. A grep across it during triage takes minutes, tail and head pull large chunks into memory, and an editor may refuse to open it at all. Bounded, rotated files are quick to search and easy to ship. Smaller files also process faster on ingestion, which matters when a SIEM is parsing them into alerts; a backlog of oversized files slows the path between an event and the alert that fires on it.

It is the front half of your retention story. Rotation decides how logs are bounded and named on the host; retention decides how long you keep them and where. The two are linked: a rotation policy that deletes after seven days quietly sets a seven-day retention limit, and if a compliance regime or an incident needs ninety days, that data is already gone. Rotation is where retention policy is enforced in practice, which is why getting the log retention window right starts with the rotation config.

How log rotation works

The mechanics are the same across tools. A rotation job runs on a schedule, checks each managed log against its trigger condition, and acts only on the files that have crossed it.

1. Check the trigger. Compare the file against its configured threshold: size, age, or both. Skip files that have not crossed it.
2. Rotate the file. Rename the current log (timestamp or numeric suffix) and open a fresh file at the original path so the application keeps writing without a gap.
3. Signal the writer if needed. A process holding the file open by its file descriptor will keep writing to the old, renamed file unless it is told to reopen. The rotation tool either sends the process a signal to reopen its log, or uses a copy-truncate approach (below).
4. Post-process the old file. Compress it, and once it falls past the retention count or age, delete or archive it.

Step 3 is the one that bites people. On Linux, a daemon opens its log file and holds the descriptor. If you just rename the file, the daemon does not notice; it keeps appending to the now-renamed file, and your shiny new log stays empty. There are two ways around it. The clean way is create mode: rotate the file, then signal the process (for example postrotate running systemctl reload or sending SIGHUP) so it reopens and writes to the new file. The other way is copytruncate: copy the current file's contents to the rotated name, then truncate the original to zero bytes in place, so the descriptor the process still holds now points at an empty file. Copytruncate needs no signal and works with software that cannot reopen on demand, but it has a small race window where lines written between the copy and the truncate are lost. Use create plus a reload where you can; use copytruncate only when the application cannot be signaled.

Rotation strategies and what to do with old files

Two decisions define a rotation policy: when to rotate, and what to do with the file after.

When to rotate is the trigger:

• Size-based. Rotate when the file exceeds a set size (for example size 100M). Predictable on disk use, but the time each file covers varies with traffic.
• Time-based. Rotate on a fixed interval, daily, weekly, or monthly. Predictable time windows, easy to reason about in an investigation ("pull Tuesday's file"), but a busy day can still produce a very large single file.
• Hybrid. Rotate on time but also if size is exceeded first (daily plus maxsize). The common production default, because it bounds both the time window and the worst-case file size.

What to do with the rotated file is the second decision, and it is where defenders have a stake:

These are not exclusive. A normal setup compresses on rotation, keeps a few weeks locally, ships everything to a central store, and deletes the local copy once it has aged out. The one rule that matters: never let the "what to do" step silently set your retention shorter than compliance or incident response requires. If you must keep ninety days and rotation deletes at fourteen, you have a policy gap, not a logging system.

A point worth stating plainly: log rotation is not a substitute for a log management system. Rotation bounds files on a single host. It does not give you search across hosts, correlation, long-term retention, or alerting. Forwarding rotated logs into centralized logging and a SIEM is what turns scattered text files into something you can hunt in. Rotation keeps the host healthy so those logs exist to forward in the first place.

Tools that do log rotation

You rarely write rotation yourself. The platform provides it.

logrotate (Linux). The standard on most Linux distributions. Driven by /etc/logrotate.conf and per-application drop-ins in /etc/logrotate.d/, it is run on a schedule (typically a daily cron or systemd timer) and supports size and time triggers, compress, rotate N to cap the number of kept files, create versus copytruncate, and postrotate scripts to signal services. A minimal stanza tells the whole story:

/var/log/myapp/*.log { daily rotate 14 compress missingok notifempty create copytruncate }

That rotates daily, keeps 14 compressed days, tolerates a missing or empty file, and uses copy-truncate for an app that cannot reopen its log.

systemd journald (Linux). The journal rotates itself by size and time through SystemMaxUse, SystemMaxFileSize, and MaxRetentionSec in journald.conf, rather than through logrotate.

Windows Event Log. The Windows event logs rotate by maximum log size, with a per-log policy to overwrite oldest events, archive when full, or stop logging. "Stop logging when full" is a trap: it preserves old events at the cost of dropping new ones, creating the same blind spot as a full disk.

Application-managed. Many applications and logging libraries rotate their own files (SQL Server's ERRORLOG.N, Nginx and Apache via their own directives or a logrotate drop-in, and library appenders like Logback or log4j2 with size and time-based rolling policies).

Frequently Asked Questions

What is log rotation?

Log rotation is the process of controlling log file size by periodically closing the active log, opening a new one under the same name, and managing the old copies. When a file crosses a size or age threshold, the system renames it (with a timestamp or numeric suffix), starts a fresh file, and then compresses, archives, or deletes the older files on a schedule. It keeps a single log from growing until it fills the disk.

What is the difference between log rotation and log retention?

Log rotation is the mechanism that bounds and cycles log files on a host: when to start a new file and what to do with the old ones. Log retention is the policy for how long logs are kept and where. They are linked because the rotation settings (how many files to keep, when to delete) enforce the retention window in practice. A rotation policy that deletes after seven days sets a seven-day retention limit whether or not that meets your requirements.

What is the difference between create and copytruncate in logrotate?

With create, logrotate renames the file and opens a new one, then a postrotate script signals the writing process to reopen its log so it writes to the new file. With copytruncate, logrotate copies the current file to the rotated name and truncates the original to zero bytes in place, so a process holding the file open keeps writing to the same (now empty) descriptor. Use create plus a reload when the application can reopen on a signal; use copytruncate only when it cannot, accepting a small risk of losing lines written during the copy.

Why is log rotation important for security?

Without rotation, a log file grows until it fills the disk, which crashes the service and stops all logging on that host, creating a blind spot during the exact high-activity events an investigator needs to see. Rotation caps the footprint, keeps files small enough to search and ship quickly, and speeds ingestion into a SIEM. It also enforces the retention window, so it directly determines whether the logs an incident needs still exist.

Should rotated logs be deleted, compressed, or archived?

Deletion frees space but destroys evidence and can violate compliance, so it should only happen after logs have aged past the required retention window. Compression (gzip) is the common default on Linux because it keeps logs local and searchable while cutting storage. Archival moves old logs off-host to a backup server, object storage, or tape for long-term history. Most setups combine them: compress on rotation, ship to a central store, and delete the local copy only after it has aged out.

Does log rotation replace a log management system?

No. Rotation only bounds files on a single host. It does not provide cross-host search, correlation, long-term retention, or alerting. A log management system or SIEM does that. The two are complementary: rotation keeps each host healthy and its logs bounded, while the central platform aggregates the forwarded logs into something you can hunt and alert on.

The bottom line

Log rotation closes the active log file when it crosses a size or age threshold, opens a fresh one under the same name, and then compresses, archives, or deletes the old copies on a schedule. On Linux that is usually logrotate; journald, Windows Event Log, and individual applications do their own. The mechanics are simple, but two details decide whether it helps or hurts: signal the writer (or use copytruncate) so new logs actually land in the new file, and never let the cleanup step delete logs faster than your retention requirement.

For a defender, rotation is not a sysadmin afterthought. An unrotated log fills a disk and blinds you during an incident; an over-aggressive rotation deletes the evidence before you can pull it. Set the trigger, keep enough compressed history locally, forward everything to a central store, and align the delete step with the retention you are actually required to hold. Done right, rotation is invisible. Done wrong, it is the reason the most important hour of an incident is missing from every log on the box.