[ новости /+++ | форум | теги | ]

NAME

NJAMD - Not Just Another Malloc Debugger
 

SYNOPSIS

#include <stdlib.h>

void *calloc(size_t nmemb, size_t size);
void *malloc(size_t size);
void free(void *ptr);
void *realloc(void *ptr, size_t size);

new, new[], delete, delete[]

#include <string.h>

char *strdup(const char *s);

And so much more...

export LD_PRELOAD=libnjamd.so
export NJAMD_PROT=val
export NJAMD_CHK_FREE=val
export NJAMD_NO_FREE_INFO=1
export NJAMD_ALIGN=num
export NJAMD_DUMP_LEAKS_ON_EXIT=num
export NJAMD_DUMP_STATS_ON_EXIT=1
export NJAMD_DUMP_CORE=soft,hard
export NJAMD_PERSISTENT_HEAP=1
export NJAMD_TRACE_LIBS=1
export NJAMD_NO_TRACE=1
export NJAMD_ALLOW_READ=1
export NJAMD_ALLOW_FREE_0=1
export NJAMD_ALLOW_MALLOC_0=1
kill -USR1 <pid>

DESCRIPTION

NJAMD is a full featured malloc debugger. That is, it protects against all common dynamic memory bugs, including overflow, underflow, writes to freed memory, and memory leaks, all without recompiling or even relinking your executable. In addition, it is able to trace memory leaks even through arbitrary library functions that wrap malloc(3), such as strdup(3), GUI widget allocators, and even C++ new and delete.

Normally, when a program does something illegal with its dynamic memory (such as writing past the end of a buffer returned by malloc(3), ie: an overflow), its execution may not immediately terminate. Instead, bizzarre and unexpected results can occur later on during program execution. This is due to the fact that malloc implementations store book keeping information before and after allocated segments. So overwriting these regions won't cause your program to crash right away, but will cause chaos during subsequent malloc requests, or even during usuage of memory returned from otherwise seemingly valid malloc()

NJAMD changes all this. It provides immediate notification (through segmentation fault) if you do anything illegal with your memory. Using your favorite debugger, you can pinpoint the source of error to the line, and even to the assembly instruction.  

USAGE

Overview

Environment Variables

LD_PRELOAD=library

This environment variable instructs the dynamic linker to override system supplied functions with those in a specified library, namely ours. To use LIBNJAMD to debug your programs, enter the equivalent to

export LD_PRELOAD=libnjamd.so

into your shell. All subsequent programs run from that shell will then use LIBNJAMD's allocator routines instead of those in the standard libc.

Alternatively, to debug only one command, enter

LD_PRELOAD=libnjamd.so <program>

NOTE on Irix systems, the variable is _RLD_LIST and the syntax is _RLD_LIST=libnjamd.so:DEFAULT. Otherwise the behavior is the exact same.

NJAMD_PROT=val

To use LIBNJAMD to protect against overflows (accessing memory past the end of an allocated buffer), enter the equivalent to

export NJAMD_PROT=overflow

into your shell. This is the default mode of operation.

There are two ways to protect against underflows (accessing memory before an allocated buffer), strict and weak. Weak is considerably faster than strict, and uses half as much memory. However, weak will only catch underflows greater than 4 or 8 bytes, depending on your archetecture.

To protect against ALL underflows, enter

export NJAMD_PROT=strict

To protect against most larger underflows, enter

export NJAMD_PROT=underflow

For memory leak checking only, enter

export NJAMD_PROT=none

This option uses standard libc malloc, and is thus is much faster and lighter than the other options, for people who just want memory leak accounting. Note that it is unavailable on platforms that don't support dlopen(2). Also, this option will misreport leaked memory by one malloc on some platforms (GNU/Linux w/ glibc 2.1), becuase malloc calls itself to set up some data structures.

Do note that each version of the library performs consistancy checks so that you know if the opposite error occured when you try to free that block. For example, when you free a buffer, the overflow version checks to make sure that the data before your buffer hasn't changed, and the underflow versions checks to make sure that the data after your buffer hasn't changed. So at the worst, you always know of a memory error by the time you free the memory. This even applies to the "none" option.

NJAMD_CHK_FREE=val

There are various methods of handling accesses to freed memory, and each has its advantages and disadvantages. If you select NJAMD_PROT=none this setting has no effect (NJAMD operates as if you selected no free protection).

export NJAMD_CHK_FREE=segv

The default method is to protect freed memory. A double free will yield a segmentation fault and no error message, and any access to a memory region freed by free or realloc will cause a segmentation fault.

export NJAMD_CHK_FREE=error

This option both protects freed memory and provides you with some sort of notification when you try to free a chunk twice. While this causes no physical memory loss, it does pollute the address space a bit, and can bog down the operating system kernel with excessive mappings to keep track of. The BSD's especially are hurt by this option, and Linux has a limit of 65536 mappings, which can be used up pretty quickly.

export NJAMD_CHK_FREE=none

This method provides no protection of freed memory. Writes to freed memory may produce the same bizzare and unpredicatble results as when using a normal malloc implementation. In addition, this option is 10 times faster than the other error checking schemes due to a free-list cache put in NJAMD > 0.9.0 (In fact, NJAMD_PROT=over with no free checking is FASTER than NJAMD_PROT=none for the allocfree benchmark/test!)

Use of this option is recommended when allocation intensive progams run out of memory under either of the preceding options, or if NJAMD is prohibively slow. Do note that by default, some OS's do not allow you to map the entire address space. You must use sysctl(2) to allow this. In Linux, for example, you must issue

sysctl -w vm.overcommit_memory=1

to use the entire address space. Try doing this instead of turning off the checking of freed memory. Also, look into the kernel_mods patches that came with the source distribution.

In addition, if you are trying to use this option because you are running out of ADDRESS space, you might want to try the next option: (NJAMD_NO_FREE_INFO)

export NJAMD_CHK_FREE=nofree

Setting this option will cause NJAMD to neglect to free ANY memory. I can't think of any circumstances where this would be useful, but it was very easy to implement :)

NJAMD_NO_FREE_INFO=1

Setting this option will cause NJAMD to internally conserve heap table space by not recording information about freed segments. Translation, things will go quicker and take less memory in some cases, but you will lose information about where segments are freed. This option may be necessary to debug HUGE applications on 64bit procs, when using NJAMD_PROT=none

NJAMD_ALIGN=num

In order for programs to work correctly under certain archetectures (ie, sparc and most other RISC CPUs), malloc must return memory aligned to a certain number of bytes if you want that section of memory to contain pointers and floating point values. The alignment of your archetecture is detected automatically when you install NJAMD. However, note that aligning memory to n bytes will cause the overflow detection to miss overflows of up to n bytes. If you are on a RISC CPU, but know that alignment is not an issue in your program (ie, if it only deals with strings), then

export NJAMD_ALIGN=1 to set alignment to 1 byte.

NJAMD_DUMP_LEAKS_ON_EXIT=num

Setting this option instructs LIBNJAMD to dump memory leak diagnostics to the front end (or standard error when running standalone) upon program termination. num levels of stack trace are provided for each malloc and free (the default max is 3, and can be set at NJAMD compile time in ./include/lib/njamd.h, via the TRACE_DEPTH define)

Do note that it is common practice for short-lived programs such as ls(1) to simply exit without freeing memory.

export NJAMD_DUMP_LEAKS_ON_EXIT=3

NJAMD_DUMP_STATS_ON_EXIT=1

Setting this environment variable instructs NJAMD to dump a short summary of memory usage versus address space usage. This option will help you figure out how much overhead is being used by NJAMD, and how much address space in total was needed to debug your application. In other words, it will either give you an excuse to buy a 64 bit CPU, or a few more RAM chips ;)

NJAMD_DUMP_CORE=hard,soft

By default, NJAMD will catch all deadly signals in order to perform cleanup, provide statistics, and give its own backtrace of when the fault occurred. However, when you are using a debugger, this behavior is not always desirable.

If you would like NJAMD to perform cleanup and statistics information, but would also like a core file, then

export NJAMD_DUMP_CORE=soft

Using soft core dumping will cause the return address information INSIDE the core file to make no sense, but it will allow NJAMD to provide a call stack dump upon exit. Using softcore also limits the coresize to 4megs (defined through NJ_LIMIT_SOFTCORE in the source).

If you would like the core file to be perfectly valid and complete at the expense of statistics and post-mortem heap integrity, then

export NJAMD_DUMP_CORE=hard

NJAMD_PERSISTENT_HEAP=1

Setting this option will cause the program heap to remain after execution in a file forrmatted ./njamd-<pid>-heap. Unlike other malloc debuggers, saving the heap in NJAMD imposes no extra performance overhead on the system. The heap is always mapped to a file in tmp, but this file only persists if this option is set. To utilize this file, use the njamd post-mortem utility njamdpm(1). Be advised that the heap can take up as much as 8 megs on 32 bit systems. ls may report it as 8 megs due to the lseek. The actual size is much less. Use du(1)

export NJAMD_PERSISTENT_HEAP=1

NJAMD_TRACE_LIBS=1

NJAMD's default action is to ignore shared library return addresses and only give you return addresses in the statically linked portion of the program. This can be a problem if your program consists of a large amount of supporting libraries, and the static section is simply a main loop. So instead, to provide return addresses in the libraries, set

export NJAMD_TRACE_LIBS=1

NJAMD_NO_TRACE=1

When debugging programs that use libraries compiled with optimization greater than -O2 or with -fomit-frame-pointer, you must disable tracing, or NJAMD will segfault. This isn't really NJAMD's fault. Gcc (see 'info gcc') claims that if __builtin_return_address is unavailable it will just return NULL. Instead it segmentation faults. I've notified the gcc team, but received no response.

export NJAMD_NO_TRACE=1

NJAMD_ALLOW_READ=1

Glibc versions up to and including 2.1.2 had a bug relating to the sscanf(3) code that was tripped by this library. As much as we would like to, unfortunately, we cannot take credit for being the first to discover this bug. The glibc folk found and fixed the bug in version 2.1.3. At any rate, this option is provided as a workaround. It allows reads past the end of a buffer to not segfault your program. The default is to forbid read, write, and execute attempts.

export NJAMD_ALLOW_READ=1

NJAMD_ALLOW_MALLOC_0=1, NJAMD_ALLOW_FREE_0=1

ANSI C specifies that malloc's of 0 are legal, and that free's of NULL are illegal. However, NJAMD's default action is to warn when either takes place, as either usually indicates a bug (or at the very least, an unneeded call). To turn this behavior off for either free or malloc, issue

export NJAMD_ALLOW_FREE_0=1, or
export NJAMD_ALLOW_MALLOC_0=1 respectively.

Dumping memory information

njamd: Memory leak of XX bytes allocated at address 0xXXXXXXX.

The address given is the actual address of the corresponding memory allocation in your code. In future versions of NJAMD, scripts or runtime functionality will be provided to translate these addresses on the fly to functions and line numbers. Until then, read on to find out how to use gdb(1) to translate them for you.

Debugging from within gdb

set env LD_PRELOAD=libnjamd.so

from INSIDE gdb. Issuing an LD_PRELOAD command to your shell before starting gdb causes gdb to use that library as well, which means gdb would be using NJAMD, and unless you're on the gdb development team, you're probably not interested in debugging gdb :)

All other options can be set in the same mannor from with gdb, or in the shell's environment outside gdb.

You can obtain memory leak information at any point by setting a break point, and then issuing signal SIGUSR1 at that breakpoint. This will provide a memory leak dump as described above. In addition, so will
call __nj_dump_leaks(__nj_display_depth)

You can also get information about any address used by njamd by issuing call __nj_ptr_info(address)

This will dump out a call stack of the malloc or free that contains that address. This is very useful for gaining information about a segmentation fault. Ie if the segfault occurs on a line that does buf[i] = 2, issue
call __nj_ptr_info(&buf[i]) to gdb.

To get gdb to translate these return addresses into something meaningful, issue

info line *0xaddress to obtain the line number of the allocation request, or

list *0xaddress to see the adjacent code as well.

Another neat trick you may find handy for tracking down things like free(NULL) and malloc(0) and other behavior that produces warnings is to set a breakpoint at __nj_eprintf so that you can determine the location of the offending instruction. __nj_eprintf is NJAMD's general purpose error function. It is called to print out any NJAMD warning or error message you see. Note that if you are using LD_PRELOAD, you may have to set a breakpoint in main and start the program before setting this breakpoint for gdb to know that __nj_eprintf is a valid symbol.

BUGS

PERFORMANCE

export NJAMD_CHK_FREE=none

Setting the checking of freed memory to none allows for an optimization that can speed up NJAMD 10 fold, making it comparable to libc malloc in some usage cases!

Furturemore, by default NJAMD uses the minimum alignment possible that won't cause a segfault. Increasing this alignment may increase performance on some archetectures, because you won't lose cycles for unaligned memory accesses. Try setting

export NJAMD_ALIGN=4 or 8

Also, there is a define at the top of ./include/lib/njamd.h in the NJAMD source tree, TRACE_DEPTH, that allows you to set how large of a stack trace is recorded. Lowering this to 1 (default is 3) may speed things up a bit, as well as cut the size of the heap file almost in half.  

NOTES

Core dump sizes are limited to 4 megs (and are probably next to useless) because of the huge amount of mapping that takes place. Some OS's (again, those evil BSD's ;) actually zero-fill mapped but unfaulted memory as it is dumped to disk, causing a core dump to take a horrendous amout of time and disk space.

For information on how the system works, read the programmers documentation in the source tree, and check out my (admittantly incomplete) Shared Memory HOWTO: http://fscked.org/writings/SHM/shm.html  

AUTHORS

Mike Perry <mikepery@fscked.org> - libnjamd
Steve Engelhardt <sengelha@yahoo.com> - Front End

SEE ALSO

njamdpm(1), efence(3), malloc(3), mmap(2), mprotect(2)

Index

NAME

SYNOPSIS

DESCRIPTION

USAGE

Overview

Environment Variables

Dumping memory information

Debugging from within gdb

BUGS

PERFORMANCE

NOTES

AUTHORS

SEE ALSO

Партнёры:

Хостинг: