Apache Security - Ivan Ristic [103]
Table 8-3. Commonly used log formats
Name
LogFormat string
common (the default)
%h %l %u %t "%r" %>s %b
combined
%h %l %u %t "%r" %>s %b "%{Referer}i" "%{User-Agent}i"
vcommon
%v %h %l %u %t "%r" %>s %b
vcombined
%v %h %l %u %t "%r" %>s %b "%{Referer}i" "%{User-Agent}i"
Though you can create your own log format, you will most likely use one of the formats above since that is what web server log analyzers support. Nevertheless, the ability to create logs with a custom format is convenient for advanced uses, as we shall see later in this chapter.
TransferLog
TransferLog is the basic request logging directive, which creates an access log with the given filename:
TransferLog /var/www/logs/access_log
The filename can be given with an absolute path, as above; if a relative filename is supplied, Apache will create the full path by pre-pending the server home directory (e.g. /usr/local/apache).
By default, the TransferLog directive uses the Common Log Format (CLF), which logs every request on a single line with information formatted (as shown in Section 8.1.1.1). Here is an example of what such a line looks like:
81.137.203.242 - - [29/Jun/2004:14:36:04 +0100] "POST /upload.php
HTTP/1.1" 200 3229
However, if a LogFormat directive has been used earlier in the configuration file, the TransferLog directive will use the format it defined and not the CLF. This is unexpected and can lead to errors since changing the order in which formats are defined can lead to a different format being used for the log files. I prefer not to use TransferLog, and instead use the CustomLog directive (which forces me to explicitly define the log format).
CustomLog
The real power comes from using the CustomLog directive. The equivalent to the TransferLog usage described above looks like this:
CustomLog /var/www/logs/access_log custom
The explicit naming of the log format helps us avoid mistakes. I like this directive because of its conditional logging features. Have a look at the following configuration fragment:
# determine which requests are static - you may need to
# adjust the regular expression to exclude other files, such
# as PDF documents, or archives
SetEnvIfNoCase REQUEST_URI "\.(gif|png|jpg)$" static_request
# only log dynamic requests
CustomLog /var/www/logs/application_log combined env=!static_request
The conditional logging opens the door to many interesting logging opportunities, which really helps in real life. Most commonly, you will use mod_setenvif or mod_rewrite (which can also set environment variables) to determine what gets logged.
I mentioned that, by default, Apache uses the CLF, which does not record many request parameters. At the very least you should change the configuration to use the combined format, which includes the UserAgent and the Referer fields.
Looking at the log format string table shown in the LogFormat section, you can see over twenty different format strings, so even the use of a combined format results in loss of information. Create your own log format based on your information requirements. A nice example can be found at:
"Profiling LAMP Applications with Apache's Blackbox Logs" by Chris Josephes (http://www.onlamp.com/pub/a/apache/2004/04/22/blackbox_logs.html)
In the article, Chris makes a case for a log format that allows for web serving troubleshooting and performance management. At the end, he names the resulting log format Blackbox.
Error Logging
The Apache error log contains error messages and information about events unrelated to request serving. In short, the error log contains everything the access log doesn't:
Startup and shutdown messages
Various informational messages
Errors that occurred during request serving (i.e., status codes 400-503)
Critical events
Standard error output (stderr)
The format of the error log is fixed. Each line essentially contains only three fields: the time, the error level, and the message.