aoqi@0:
aoqi@0: The HotSpot Serviceability Agent (SA) is a set of Java APIs which aoqi@0: mirror the internal APIs of the HotSpot VM and which can be used to aoqi@0: examine the state of a HotSpot VM. aoqi@0:
aoqi@0: aoqi@0:aoqi@0: The system understands the layout of certain VM data structures and is aoqi@0: able to traverse these structures in an examination-only fashion; that aoqi@0: is, it does not rely on being able to run code in the target VM. For aoqi@0: this reason it transparently works with either a running VM or a core aoqi@0: file. aoqi@0:
aoqi@0: aoqi@0:aoqi@0: The system can reconstruct information about Java frames on the stack aoqi@0: and objects in the heap. Many of the important data structures in the aoqi@0: VM like the CodeCache, Universe, StubQueue, Frames, and VFrames have aoqi@0: been exposed and have relatively complete (or at least useful) aoqi@0: implementations. aoqi@0:
aoqi@0: aoqi@0:aoqi@0: A small graphical debugger called HSDB (the "HotSpot Debugger") has aoqi@0: been written using these APIs. It provides stack memory dumps aoqi@0: annotated with method invocations, compiled-code inlining (if aoqi@0: present), interpreted vs. compiled code, interpreter codelets (if aoqi@0: interpreted), and live oops from oop-map information. It also provides aoqi@0: a tree-based oop inspector. More information will be added as aoqi@0: necessary; please send aoqi@0: email with suggestions on what would be useful. aoqi@0:
aoqi@0: aoqi@0:aoqi@0: The SA currently only works on Solaris. It uses dbx to connect to the aoqi@0: remote process or core file and communicates with a small piece of aoqi@0: code (an "import module") loaded into the debugger. aoqi@0:
aoqi@0: aoqi@0:aoqi@0: The Java-side source code, which is the bulk of the SA, is in aoqi@0: src/share/vm/agent. The organization of the sun.jvm.hotspot package aoqi@0: hierarchy mirrors the organization in the VM. This should allow aoqi@0: engineers familiar with the HotSpot sources to quickly understand how aoqi@0: the SA works and to make modifications if necessary. To build these aoqi@0: sources, cd to src/share/vm/agent and type "make". aoqi@0:
aoqi@0: aoqi@0:aoqi@0: aoqi@0: The SA on Solaris works by communicating with a small piece of code aoqi@0: (an "import module") loaded into dbx. The source code for this import aoqi@0: module is in src/os/solaris/agent. To build this library, cd to aoqi@0: src/os/solaris/agent and type "make 32bit" or "make 64bit". The aoqi@0: distinction is necessary because the SPARC version of dbx ships with aoqi@0: two versions of its executable, and depending on which architecture aoqi@0: (v8 or v9) the debugger is running on selects the appropriate aoqi@0: executable. The SA tries the v8 version first, but if you are running aoqi@0: on a v9 machine you must provide both versions to the SA. aoqi@0:
aoqi@0: aoqi@0:aoqi@0: aoqi@0: The system is currently hardwired to look on jano for its dbx aoqi@0: executable and import module. The relevant directory structure looks aoqi@0: like this: aoqi@0: aoqi@0:
aoqi@0: The code which builds up path names to these executables is contained aoqi@0: in sun.jvm.hotspot.HotSpotAgent.java. There are hardcoded paths in aoqi@0: this file to jano, but the rest of the system is isolated from this. aoqi@0:
aoqi@0: aoqi@0:aoqi@0: (It would be nice to have a panel in the debugger allowing aoqi@0: configuration of all of the known operating systems' options; right aoqi@0: now Solaris is the only supported OS, making that easier.) aoqi@0:
aoqi@0: aoqi@0:aoqi@0: An installation of HSDB has been placed on jano. To access it, add the aoqi@0: following directory to your PATH: aoqi@0:
aoqi@0: aoqi@0:aoqi@0:
aoqi@0: /net/jano/export/disk05/hotspot/sa/bin/common aoqi@0:aoqi@0: aoqi@0: aoqi@0:
aoqi@0: To start the debugger, type "hsdb". aoqi@0:
aoqi@0: aoqi@0:aoqi@0: Alternatively, you can start a local build of the debugger by building aoqi@0: the sources in src/share/vm/agent, cd'ing to that directory, and aoqi@0: typing "java sun.jvm.hotspot.HSDB". aoqi@0:
aoqi@0: aoqi@0:aoqi@0: There are three modes for the debugger: attaching to a local process, aoqi@0: opening a core file, and attaching to a remote "debug server". The aoqi@0: remote case requires two programs to be running on the remote machine: aoqi@0: the rmiregistry (see the script "start-rmiregistry" in this directory; aoqi@0: run this in the background) and the debug server (see the script aoqi@0: "start-debug-server"), in that order. start-rmiregistry takes no aoqi@0: arguments; start-debug-server takes as argument the process ID or the aoqi@0: executable and core file names to allow remote debugging of. Make sure aoqi@0: you do NOT have a CLASSPATH environment variable set when you run aoqi@0: these scripts. (The classes put into the rmiregistry are in sun.*, and aoqi@0: there are permissions problems if they aren't placed on the boot aoqi@0: classpath.) aoqi@0:
aoqi@0: aoqi@0:aoqi@0: NOTE that the SA currently only works against VMs on Solaris/SPARC. aoqi@0: Remote debugging of Solaris/SPARC VMs on arbitrary platforms is aoqi@0: possible using the debug server; select "Connect to debug server..." aoqi@0: in HSDB. aoqi@0:
aoqi@0: aoqi@0:aoqi@0: Once the debugger has been launched, the threads list is displayed. aoqi@0: The current set of functionality allows: aoqi@0:
aoqi@0: aoqi@0:aoqi@0: More functionality is planned. Please send email with suggestions on what aoqi@0: would be useful, with any questions or comments, or if the debugger aoqi@0: crashes. aoqi@0:
aoqi@0: aoqi@0:aoqi@0: HSDB does not suspend the system at a safepoint, but at an arbitrary aoqi@0: point. This means that many of the invariants in the VM's code are not aoqi@0: followed. aoqi@0:
aoqi@0: aoqi@0:aoqi@0: As it happens, most of the places where the code ported over from the aoqi@0: VM has failed involve the topmost frame on the stack. Some aoqi@0: modifications have been made to allow the system to recognize aoqi@0: problematic situations. aoqi@0:
aoqi@0: aoqi@0:aoqi@0: Certainly, not all of the failure modes of the debugger have been aoqi@0: found. Please send email if aoqi@0: HSDB throws an exception. The best debugging aid in these situations aoqi@0: is a core file since it is a static view of the VM to which we can aoqi@0: then adapt the debugger code, as opposed to having to try to suspend aoqi@0: the VM over and over to reproduce the failure. gcore (1) is a useful aoqi@0: tool. (NOTE: do not try gcore with any application using the DGA X aoqi@0: server extension (example: Java2Demo); the kernel will panic. See bug aoqi@0: 4343237.) aoqi@0:
aoqi@0: aoqi@0: aoqi@0: