Squid_ The Definitive Guide - Duane Wessels [173]
Segmentation violation errors are, unfortunately, more common and sometimes harder to fix. A "SEGV" usually occurs when the process tries to access an invalid memory area. It might be a NULL pointer or a memory address outside the scope of the process. They are particularly difficult to track down when the cause (the bug) and effect (the SEGV) are separated in time.
By default, Squid traps bus errors and segmentation violations, and attempts a clean shutdown when they occur. You'll see something like this in cache.log:
FATAL: Received Bus Error...dying.
2003/09/29 23:18:01| storeDirWriteCleanLogs: Starting...
In most cases, Squid is able to write clean versions of the swap.state files. Just before exiting, Squid calls abort( ) to create a core file. The core file may help you, or other developers, track down and fix the bug.
A core file is generally more useful when it is created immediately following the error, rather than calling the clean shutdown procedure first. You can tell Squid not to trap bus errors and segmentation violations with the -C command line option:
% squid -C ...
Note that some operating systems use the filename core, while others prepend the process name (i.e., squid.core). Once you have the core file, use a debugger to get a stack trace. gdb is the GNU debugger—a companion to the GNU C compiler. If you don't have gdb, try running dbx or adb instead. Here's how you can use gdb to get a stack trace:
% gdb /usr/local/squid/sbin/squid /path/to/squid.core
...
Core was generated by 'squid'.
Program terminated with signal 6, Abort trap.
...
Then, type where to print the stack trace:
(gdb) where
#0 0x28168b54 in kill ( ) from /usr/lib/libc.so.4
#1 0x281aa0ce in abort ( ) from /usr/lib/libc.so.4
#2 0x80a2316 in death (sig=10) at tools.c:301
#3 0xbfbfffac in ?? ( )
#4 0x80abe0a in storeDiskdSend (mtype=4, sd=0x82101e0, id=1214000,
sio=0x9e90a10, size=4096, offset=-1, shm_offset=0)
at diskd/store_io_diskd.c:485
#5 0x80ab726 in storeDiskdWrite (SD=0x82101e0, sio=0x9e90a10,
buf=0x13e94000 "...", size=4096, offset=-1, free_func=0)
at diskd/store_io_diskd.c:251
#6 0x809d2fb in storeWrite (sio=0x9e90a10, buf=0x13e94000 "...",
size=4096, offset=-1, free_func=0) at store_io.c:89
#7 0x80a1c2d in storeSwapOut (e=0xc5a7800) at store_swapout.c:259
#8 0x809b667 in storeAppend (e=0xc5a7800, buf=0x810f9a0 "...", len=57344)
at store.c:533
#9 0x807873b in httpReadReply (fd=134, data=0xc343590) at http.c:642
#10 0x806492f in comm_poll (msec=10) at comm_select.c:445
#11 0x8084404 in main (argc=2, argv=0xbfbffa8c) at main.c:742
#12 0x804a465 in _start ( )
As you can see, the stack trace prints the name of each function, its arguments, and the source code filenames and line numbers. This information is extremely useful when tracking down bugs. In some cases, however, it isn't sufficient. You might be asked to execute additional commands in the debugger, such as printing the value of a variable from within a certain function:
(gdb) frame 4
#4 0x80abe0a in storeDiskdSend (mtype=4, sd=0x82101e0, id=1214000,
sio=0x9e90a10, size=4096, offset=-1, shm_offset=0)
at diskd/store_io_diskd.c:485
485 x = msgsnd(diskdinfo->smsgid, &M,
msg_snd_rcv_sz, IPC_NOWAIT);
(gdb) set print pretty
(gdb) print M
$2 = {
mtype = 4,
id = 1214000,
seq_no = 7203103,
callback_data = 0x9e90a10,
size = 4096,
offset = -1,
status = -1,
shm_offset = 0
}
After you've reported a bug, try to keep the core file around for a few days, in case you need additional information from it.
Can't Find the Core File?
core files are written in the process' current directory. By default, Squid doesn't change its current directory at startup. Thus, your core file, if any, should be written in the directory in which Squid was started. You won't find a core file if the filesystem doesn't have enough free space or if the process owner doesn't have write permission in the directory. You can use the coredump_dir directive to make