How to rotate TYPO3 log files

Logging is an essential part of the life cycle of an application. When errors occur, they are a way to be notified when something went wrong and to understand why it happened.

By default TYPO3 writes errors in a log file and deprecation notices — which should be disabled on production systems. Third-party extensions can also write additional log files. These log files are stored in the directory var/log/ in the TYPO3 installation. They can become quite large over time — depending on the configured log level and the number of log entries:

ls -l /var/www/html/var/log/
(out)-rw-rw-r-- 1 www-data www-data  2291246 Mar 21 10:35 typo3_705faa28a2.log
(out)-rw-rw-r-- 1 www-data www-data 23203989 Mar 21 10:32 typo3_deprecations_705faa28a2.log

While this is no big deal in a development environment (you can simply delete the files at startup or manually) large files should be avoided on production systems:

  • Errors and warnings from a few weeks ago are usually no longer of interest.
  • If you are investigating an error, it can be useful to search in a log file from one day or a small file.
  • Log files can consume a lot of disk space.

This article shows you how to automatically rotate and delete log files with logrotate, a tool for UNIX systems. Log files can be rotated hourly, daily, weekly, monthly, yearly, or depending on their size. The first version of the package dates back to 1996, so it is considered a bulletproof solution.

Installation of logrotate

In most cases the web server will run under Linux with the TYPO3 installation. On Linux systems (and most other Unix derivates) the logrotate package is available — log files from Apache, MySQL and other services are also rotated with this tool. You can test if logrotate is available with:

which logrotate
(out)/usr/sbin/logrotate

If it is not installed, this is the way to go on Debian-based systems:

apt install logrotate

Or on RedHat Enterprise Linux/CentOS:

yum install logrotate

When installing using the Linux package managers, the required cron jobs are also activated.

Configuration

The configuration files for logrotate are stored in /etc/logrotate.d/. Let's have a look into this folder (it may look different depending on your installation):

ls -l /etc/logrotate.d/
(out)-rw-r--r-- 1 root root 173 May 28  2019 apt
(out)-rw-r--r-- 1 root root 112 Apr 19  2019 dpkg
(out)-rw-r--r-- 1 root root 351 Aug 13  2019 nginx
(out)-rw-r--r-- 1 root root 155 Feb 24 19:39 php7.4-fpm

Take a look into these configuration files. The next step is to create a custom logrotate configuration for a TYPO3 installation. Below you can see the logrotate configuration for my TYPO3 installations:

/var/www/html/var/log/*.log {
    daily
    missingok
    rotate 7
    compress
    delaycompress
    notifempty
    create 664 www-data www-data
    sharedscripts
}
/etc/logrotate.d/typo3

The format looks quite simple. The first line defines the path to the files and the file name. The following keywords specify the behavior when rotating:

  • The "daily" keyword defines the frequency of the rotation. Other configurations could be "hourly", "weekly" or "monthly". You can also rotate depending on the size of the log file.
  • "missingok" ignores missing log files.
  • "rotate 7" says that the rotated log files should be kept for 7 days, then they will be deleted.
  • "compress" and "delaycompress": Rotated log files should be compressed, but not the previous day's log files.
  • "notifempty": If the log file is empty, it is not rotated.
  • "create 664 www-data www-data": The rotated log file is created with the given permissions.
  • "sharedscripts": The scripts will only be executed once for the specified file pattern.

There are some more configuration options, have a look into the logrotate manual.

After 7 days the TYPO3 log directory looks like this:

ls -l /var/www/html/var/log/
(out)-rw-rw-r--+ 1 www-data www-data  405078 Mar 28 11:35 typo3_705faa28a2.log
(out)-rw-rw-r--+ 1 www-data www-data 2859566 Mar 28 03:15 typo3_705faa28a2.log.1
(out)-rw-rw-r--+ 1 www-data www-data  149251 Mar 27 03:15 typo3_705faa28a2.log.2.gz
(out)-rw-rw-r--+ 1 www-data www-data  158489 Mar 26 03:15 typo3_705faa28a2.log.3.gz
(out)-rw-rw-r--+ 1 www-data www-data  101047 Mar 25 03:15 typo3_705faa28a2.log.4.gz
(out)-rw-rw-r--+ 1 www-data www-data   65603 Mar 24 03:15 typo3_705faa28a2.log.5.gz
(out)-rw-rw-r--+ 1 www-data www-data   44760 Mar 23 03:15 typo3_705faa28a2.log.6.gz
(out)-rw-rw-r--+ 1 www-data www-data   54469 Mar 22 03:15 typo3_705faa28a2.log.7.gz
(out)-rw-rw-r--+ 1 www-data www-data 1009940 Mar 28 11:35 typo3_deprecations_705faa28a2.log
(out)-rw-rw-r--+ 1 www-data www-data 9870595 Mar 28 03:15 typo3_deprecations_705faa28a2.log.1
(out)-rw-rw-r--+ 1 www-data www-data  253630 Mar 27 03:15 typo3_deprecations_705faa28a2.log.2.gz
(out)-rw-rw-r--+ 1 www-data www-data  292400 Mar 26 03:15 typo3_deprecations_705faa28a2.log.3.gz
(out)-rw-rw-r--+ 1 www-data www-data  171451 Mar 25 03:15 typo3_deprecations_705faa28a2.log.4.gz
(out)-rw-rw-r--+ 1 www-data www-data  119368 Mar 24 03:15 typo3_deprecations_705faa28a2.log.5.gz
(out)-rw-rw-r--+ 1 www-data www-data   76274 Mar 23 03:15 typo3_deprecations_705faa28a2.log.6.gz
(out)-rw-rw-r--+ 1 www-data www-data   86522 Mar 22 03:15 typo3_deprecations_705faa28a2.log.7.gz