Core Dump
What is Core Dump?
A core dump is a technical term for a file containing detailed information about the state of an application at a given time, in a form which developers can make use of. They are particularly helpful for tracking down the reasons for application crashes, which is why many systems can be told to create one automatically when an application crash occurs.
The default action of certain signals is to cause a process to terminate and produce a core dump file, a disk file containing an image of the process's memory at the time of termination. This image can be used in a debugger (e.g., gdb) to inspect the state of the program at the time that it terminated. A list of the signals which cause a process to dump core can be found in signal.
How to create a core dump on Linux
A core dump is very helpful for helping us tracking down crashes of VirtualBox.
To create a core dump, start VirtualBox from a command line (e.g. xterm):
$ ulimit -c unlimited $ echo -n 1 | sudo tee /proc/sys/kernel/core_uses_pid $ echo -n 1 | sudo tee /proc/sys/fs/suid_dumpable $ VirtualBox
NOTE: > and >> are Bash built-in redirection commands, after sudo only echo has root privilege so sudo echo -n 1 > /proc/sys/kernel/core_uses_pid will result in permission denied error.
If possible, start the virtual machine directly:
$ ulimit -c unlimited $ echo -n 1 | sudo tee /proc/sys/kernel/core_uses_pid $ echo -n 1 | sudo tee /proc/sys/fs/suid_dumpable $ /usr/lib/virtualbox/VirtualBox -startvm VM_NAME
IMPORTANT: Ensure that NO startup script (~/.bashrc, ~/.bash_profile, ~/.profile) contains an instruction like ulimit -c 0 as the limit cannot be increased once it was set to zero.
NOTE: Starting with version 2.0.0, the VirtualBox processes are started suid root, that is, with permissions to do things that "normal" applications cannot.
VirtualBox is (as of version 2.0.0) a system application which can be started from a user account without special privileges - a so-called "setuid" application. As core dumps are usually not allowed for these sensitive applications you have to explicitly allow them after starting your computer but before the crash which you want a core dump from. This can be done by typing the following command at the command line.
$ echo -n 1 | sudo tee /proc/sys/fs/suid_dumpable
When VirtualBox or one of its processes crashes, a file core.<pid> is created in the current directory. Be aware that core dumps can be very huge. Please compress the file before submitting it to a bug report. Or better don't attach the file to a report. Note that this core dump can contain a memory dump of your guest which can include sensitive information.
If several core files are created, you can check which process created them using the command
file core.<pid>
to be sure of the right one to send.
Core dump on Ubuntu
The procedure for actually creating a core dump is similar on most Linux systems, but on Ubuntu it is slightly different due to the Ubuntu crash reporting tool (Apport).
Core Pattern for Ubuntu kernels
cat /proc/sys/kernel/core_pattern |/usr/share/apport/apport %p %s %c
To create a core dump on an Ubuntu system, create a file called ~/.config/apport/settings in your home directory with the contents
[main] unpackaged=true
and run VirtualBox. Any crashes of VirtualBox after you have created the file will cause a crash report to be created in /var/crash with a name like "_usr_lib_virtualbox_VirtualBox.1000.crash".
Extract the core file using the apport-unpack tool
apport-unpack <report > <target_directory>
The core dump file can be found in target directory, named CoreDump.
To stop crash files being created for non-system applications delete the file that you created again.
How to create dumps on Mac OS X
To create a core dump on Mac OS X, start VirtualBox from a command line:
ulimit -c unlimited VirtualBox
or better start the VM directly:
ulimit -c unlimited /Applications/VirtualBox.app/Contents/MacOS/VirtualBox -startvm VM_NAME
Ensure that NO startup script (~/.bashrc, ~/.bash_profile, ~/.profile) contains an instruction like ulimit -c 0 as the limit cannot be increased once it was set to zero.
Alternatively (for Finder & launchpad)
$ launchctl limit core unlimited
or for permanent effect the same could be specified in $HOME/.launchd.conf or /etc/launchd.conf.
The core files can be found in /cores directory.
How to create dumps on Solaris
To get core on Solaris host run the following command as root
coreadm -g /var/cores/core.%f.%p -i core.%f.%p -e global -e process -e global-setid -e proc-setid -e log
The cores will now be placed in /var/cores/ folder with the global dumps will go into /var/crash/<hostname>
System core dumps need to be enabled via dumpadm. The important thing is to have "Savecore enabled" to "yes" (use dumpadm -y). The config. should look something like this:
dumpadm Dump content: kernel pages Dump device: /dev/zvol/dsk/rpool/dump (dedicated) Savecore directory: /var/crash/myhostname Savecore enabled: yes Save compressed: on
Forcing VirtualBox to terminate with a core dump
Sometimes it is required to force a VirtualBox process to terminate, for example, a VM hangs for some unknown reason. This can be done as follows:
On Linux
$ ulimit -c unlimited $ echo -n 1 | sudo tee /proc/sys/fs/suid_dumpable $ /usr/lib/virtualbox/VirtualBox -startvm VM_NAME & $ pidof VirtualBox 7145 $ kill -4 7145
As an alternative to kill you can do
$ pidof VirtualBox 7145 $ gcore 7145
NOTE: gcore is a shell script calling gdb. Some Linux distributions do NOT have it in the GDB package.
or in GDB
# Attach to the virtualbox process using PID (gdb) attach 641 Attaching to process 641 # Generate core dump (gdb) gcore Saved corefile core.641 # detach the virtualbox process previously attached (gdb) detach Detaching from program: /usr/lib/virtualbox/VirtualBox, process 641
On Mac OS X
$ ulimit -c unlimited $ /Applications/VirtualBox.app/Contents/MacOS/VirtualBox -startvm VM_NAME & $ ps aux | grep VirtualBox ... 7145 ... VirtualBox ... $ kill -4 7145
On Solaris
# ulimit -c unlimited # /opt/VirtualBox/amd64/bin/VirtualBox -startvm VM_NAME & # ps -ef|grep VirtualBox ... 7145 ... VirtualBox ... # kill -4 7145
You can find result core file according location specified in coreadm
# coreadm
NOTE: Passing the signal number 4 (SIGILL) is essential! The same applies to the alternative frontends VBoxHeadless and VBoxSDL.
Linux with systemd
For Linux distributions with systemd (Arch Linux, Fedora, OpenSUSE etc...), core dumps are stored in systemd journal.
cat /proc/sys/kernelcore_pattern |/usr/lib/systemd/systemd-coredump %p %u %g %s %t %e
The core dumps can be accessed using systemd-coredumpctl
For example
systemd-coredumpctl list
TIME PID UID GID SIG EXE
Thu 2013-02-21 20:54:30 EST 283 1000 1000 11 /usr/bin/gsetti
Sat 2013-02-23 16:11:49 EST 249 1000 1000 11 /usr/bin/gsetti
Thu 2013-03-14 11:40:08 EST 1706 0 0 4 /usr/sbin/httpd
Thu 2013-03-14 13:32:01 EST 2105 0 0 11 /tmp/segfault
(END)
Send core dump for PID 2105 to GDB
systemd-coredumpctl gdb 2105
This behaviour can be disabled with a simple "hack":
$ ln -s /dev/null /etc/sysctl.d/coredump.conf $ /lib/systemd/systemd-sysctl # or possibly a reboot
As always, the size of core dumps has to be equal or higher than the size of the core that is being dumped, as done by for example ulimit -c unlimited.
More on core dumps for Linux
/proc/sys/kernel/core_pattern
By default, a core dump file is named core, but the /proc/sys/kernel/core_pattern file (since Linux 2.6 and 2.4.21) can be set to define a template that is used to name core dump files.
The template can contain % specifiers which are substituted by the following values when a core file is created:
- %% a single % character
- %p PID of dumped process
- %u (numeric) real UID of dumped process
- %g (numeric) real GID of dumped process
- %s number of signal causing dump
- %t time of dump, expressed as seconds since the Epoch, 1970-01-01 00:00:00 +0000 (UTC)
- %h hostname (same as nodename returned by uname(2))
- %e executable filename (without path prefix)
- %E pathname of executable, with slashes ('/') replaced by exclama‐tion marks ('!').
- %c core file size soft resource limit of crashing process (since Linux 2.6.24)
/proc/sys/kernel/core_uses_pid
Since version 2.4, Linux has also provided a more primitive method of controlling the name of the core dump file. If the /proc/sys/kernel/core_uses_pid file contains the value 0, then a core dump file is simply named core.
sudo echo 1 > /proc/sys/kernel/core_uses_pid
If this file contains a nonzero value, then the core dump file includes the process ID in a name of the form core.PID
/proc/sys/fs/suid_dumpable
The value in this file determines whether core dump files are produced for set-user-ID or otherwise protected/tainted binaries. Three different integer values can be specified:
- 0 (default) This provides the traditional (pre-Linux 2.6.13) behavior. A core dump will not be produced for a process which has changed credentials (by calling seteuid(2), setgid(2), or similar, or by executing a set-user-ID or set-group-ID program) or whose binary does not have read permission enabled.
- 1 ("debug") All processes dump core when possible. The core dump is owned by the file system user ID of the dumping process and no security is applied. This is intended for system debugging situations only. Ptrace is unchecked.
- 2 ("suidsafe") Any binary which normally would not be dumped (see "0" above) is dumped readable by root only. This allows the user to remove the core dump file but not to read it. For security reasons core dumps in this mode will not overwrite one another or other files. This mode is appropriate when administrators are attempting to debug problems in a normal environment.