Why long way for thread dump on UNIX/Linux machines? Easy Steps!

jps - Java process list : Java Utilitycommand 

I was looking for diagnostic ways with freely available tools from JDK then I found the great valuable command tool 'jps'. The jps command can be used with three options(l, v, m). it can work for any Java - JEE Servers such as WebLogic, WebSphere, JBoss, Tomcat. In other words, any app server that uses the latest JDK 5+ version (version 6, 7, 8 also supports). Let me walk-through those interesting options in Win and *nix platforms. Before you execute this command tool, make sure that JAVA_HOME\bin is in the PATH setting.

Java Command tool : jps options

jps Option : jps -l

this -l option will give the full Java package name that invoked the Java process.

C:\Users\pavanbsd>jps -l
9116 sun.tools.jps.Jps
8812 weblogic.Server

Filtering the weblogic instance with find command on jps will give you the desired outcome.

C:\Users\pavanbsd>jps -l |find "weblogic"
8812 weblogic.Server

jps -v

this -v option is most important for tracking WebLogic, this option gives you the JVM aguments used to invoke the JAVA process (weblogic.Server). Usually for our WebLogic what we set in the setDomainEnv.sh will appear with this option.

C:\Users\pavanbsd>jps -v |find "weblogic"
8812 Server -Xms256m -Xmx512m -XX:MaxPermSize=128m -Dweblogic.Name=adm_clstr_dom -Djava.security.policy=C:\Oracle\MIDDLE
~1\WLSERV~1.3\server\lib\weblogic.policy -Dweblogic.ProductionModeEnabled=true -da -Dplatform.home=C:\Oracle\MIDDLE~1\WL
SERV~1.3 -Dwls.home=C:\Oracle\MIDDLE~1\WLSERV~1.3\server -Dweblogic.home=C:\Oracle\MIDDLE~1\WLSERV~1.3\server -Dweblogic
.management.discover=true -Dwlw.iterativeDev=false -Dwlw.testConsole=false -Dwlw.logErrorsToConsole=false -Dweblogic.ext
.dirs=C:\Oracle\MIDDLE~1\patch_wls1036\profiles\default\sysext_manifest_classpath

jps -m

this -m option will gives you the Java main() programs command line arguments that used by Java process.

 
C:\Users\pavanbsd>jps -m
8812 Server
8984 Jps -m

The problem of finding Java PIDs

Of course, you can kill the instance in the emergency in the same way this is quicker than regular ps command. Usually, WLA uses the following command for finding the WebLogic managed server process:

ps -fu username

Scanning that process list with your eyes for WebLogic start process, then its child process finally java process in that process list. the whole process looks hectic when there are many WebLogic instances running on the same machine. Sometimes killing wrong process makes nonsense/messy. To avoid such collisions in the decision of killing process best way is using 'jps'.

Try it and know about it then you can enjoy working with it...

Another problem area that will do pain for Middleware admins is that Find is the NodeManager is running or not on a machine (Win/*nix)? the best option I have simply types 'jps' it will give the desired result. Here I found the simple way for taking thread dump with JDK tools which work on the UNIX machine and also in Windows. Assuming that you WLA(WebLogic Admin) are aware of which WebLogic instance needs to take Thread-dump for analysis of an issue. The best option which I use that makes it very easy to do so is the java process in a UNIX box is :

$ jps -lv |grep WL_instancename

This will give you the process id and long view of JAVA_OPTIONS assigned to that instance. Pick that first number (Java PID) and use your regular kill -3 on it :)

The jps command as Troubleshooting helper

The jps command can be used to pipe out the PID for thread-dumps and memory/heap-dumps or Generation wise footprint with the following JDK commands utilities:

• jstat 
• jstack
• jhat 
• jdb
• jmap
• jinfo
• jvisualvm
• jconsole

Count WebLogic Servers using jps

Basically while working on Produciton environments, there would be WebLogic server bounce required while doing Application release or some patching happen on some of the middleware architecture components. Sometimes you may need to know how many WebLogic instances are running in that Unix machine or how many left for bounce. Simply you can use our free 'jps' command as shown below

 $jps |wc -l

If you wish to see only java process you can use simply jps command.

$jps

We can use for the WebLogic 9.x 10, 11g versions managed instances for thread dumps or terminating the process.

The same trick will work for all Oracle Fusion Middleware Infrastructure using services such as Oracle Analytics Server (OAS) domain, Oracle Data Integrator (ODI) domain, Oracle Business Intelligence Enterprise Edition (OBIEE), Oracle APEX domain, Oracle SOA Suite, Oracle Service Bus (OSB) domain, OUD, OIM and may more.

Note:
1. This command is available in JDK version 1.5 onwards only.
2. Careful while using long view it shows full details.

Heap dump on a Unix machine

After exploring the Jay/Faisal's blog on 'jmap' java utility usage and Heap dump. It is really great work by Jay/Faisal. I just thought that similar kind of experiment we did long back on UNIX machines. I am glad to sharing that with you guys.

Last year, we were struggling to overcome the OutOfMemoryError, which would effect the most of productive hours. In this assignment I need to figure out what process is causing the low memory in the environment is identified by searching a all log files in the machine. Assume that all the WebLogic instance log files are collected into common directory structure, each of them are stored respective instance named folder.

Script 1:

After identifying the impacted instances, I need to take the heap dump of that particular instance with corresponding process id.

#==============================================================
# File Name  : CheckLogs.sh
# Author  : Pavan Devarakonda
# Purpose  : Script for searching all WebLogic instances logs in a box
#==============================================================
instances=`ls /path/instances|grep web`
phrase=$1
date
for x in $instances
do
        echo 'Checking in the :' $x
        cd /path/instances/$x/logs
        egrep -i $phrase instance*log
        egrep -i $phrase instance*out
done
 
# Know the CPU load average at this time
date
uptime

-->

Script 2: Automatic script for Heap dump, here you need to provide the managed server name at the command-line argument. 

#!/bin/bash

#=======================================================
# Name    : InstanceHeapdump.sh
# Purpose : This script will takes instance name as input
# Takes the thread dump and also takes heap dump
#=======================================================
 
if [ "$1" = "" ]; then
        echo "Usage : $0 "
        exit
else
        instance="$1"
        user=$LOGNAME
        ppid=`ps -fu $user|grep $instance|grep startMan|grep -v grep|awk '{print $2}'`
        wpid=`ps -fu $user|grep $ppid|grep startWebLogic.sh|awk '{print $2}'`
        jpid=`ps -fu $user|grep $wpid|grep java|awk '{print $2}'`
        echo 'The Java PID:' $jpid
        kill -3 $jpid
        if [ $jpid = "" ]; then
                echo "Invalid instance input..."
                exit
        else
               jmap -dump:live,format=b,file=heap.bin $jpid
                mv heap.bin $instance_heap.bin
        fi
fi

This could give you one more way of finding a java process in UNIX machine. You can use jps command instead of three lines of awk filters. In this same script to make hold the java process not to crash, we can call a WLST script to suspend the instance and then you can happily take the heap dump.

What to do after heapdump creation?
Use jhat command to run the analyzer to show the outcome on a browser.
Follow the Faisal tips to find memory leaks, use eclipse MAT that is comfortable for your system.