Java parameters for WebLogic Server

There are many burning issues on going WebLogic based environments due to this Java parameters

• JAVA_OPTIONS, l
• USER_MEM_ARGS, 
• PRE_CLASSPATH, 
• POST_CLASSPATH. 

Heap size in MEM

The USER_MEM_ARGS can be defined for WebLogic as follows as minimum requirement :

-Xms1024m The minimum size of JVM
-Xmx1024m The maximum size of JVM keeping the same as minimum will avoid number of GCs.
-XX:NewSize=512m The Young Generation (YG) space size
-XX:MaxNewSize=512m The maximum sise for Young Generation space
-XX:SurvivorRatio=6 There will be two (from, to) survivor spaces both will equal in space and one will be always empty for swapping which will be used after GC.

-Xnoclassgc

Perm space 

-XX:PermSize=128m The permanant space for static classes they will be loaded only once.
-XX:MaxPermSize=128m There could be chance of increase of classes as the application functionality increased widely.

Troubleshoot OOM errors
In production or Development environment if there is issue with memory we need to have a snapshot of how the JVM is occupied this we can get by command line with passing WebLogic process id. Other option is that we can have it in the JAVA_OPTION because it will capture the heap dump automatically as on when there is   memory issue.

-XX:+HeapDumpOnOutOfMemoryError This parameter will be used whenever OOME encountered by instances heap dump will be take automatically
-XX:+DisableExplicitGC This parameter will restrict the external Garbage collection programs from explicit

GC Algorithms

We have three options of GC algorithms Serial, Parallel, Concurrent Mark Sweep. Most of WebLogic production environments prefers CMS.

-XX:+UseConcMarkSweepGC The GC algorithm you can choose other ways as per your application need.
-verbose:gc The GC functionality will be traced with this parameter

While debugging the GC issue, you might need to know that how frequently GC happen for WebLogic instance. the following options is going to help you.

-XX:+PrintGCTimeStamps This will print the timestamp of minor GC happen

-XX:+PrintGCDetails This will write to GC logs about Eden, Young, Perm space details
-XX:+PrintGCApplicationStoppedTime Whenever Full GC happen there will be pause for the application execution that will be written to logs
-XX:+PrintHeapAtGC Current JVM space details after Full GC.

We can collect the GC logs in a separate file by this argument it takes absolute path.
-Xloggc:/$HOME/logs/instance_GCoutput.log GarbageCollection log path can be specified by this.

-Xint interpreter-only mode for JIT this is introduced in mixed mode JVMs.

References:

1. http://java.sun.com/javase/technologies/hotspot/vmoptions.jsp
2. IBM DeveloperWorks :
3. http://olex.openlogic.com/wazi/2009/how-to-fix-memory-leaks-in-java/
4. http://java.sun.com/javase/6/webnotes/trouble/TSG-VM/html/tooldescr.html#gblfj
5. JVM Options -Xint

JVM monitoring with jstat Shell Script

The Oracle WebLogic Server instances are nothing but each one running on a single JVM, when this JVM crashes there could be one of the reasons as overworking of Garbage Collection or not working at all (waiting continuously).  

It is good practice that monitoring GC in JVM with detailed statistics will give you a clear idea to analysis at what circumstances happening wrong. And the best way to look in deeper levels of garbage collection also like Young Generation (Eden space, Survivor spaces S0, S1) Old Generation (tenured Generation), and Perm Generation (Statistic Objects/Classes).

JDK 1.5 and latest providing excellent JDK command utilities for interrogate the current running Java Process and look inside of JVM take snap with following:
1. jps (Java Process)
2. jstat (JVM status)

The 'jps' command with -lv options gives you complete detailed java process arguments for MEM settings and relavent WebLogic Server instances name. We have already discussed about this command utility in another post.

The jstat command with -gc option will produce the Garbage Collection of given java process id.

GC Monitoring shell script with jstat tool

I have modified according to my convenience and requirements as a shell script that will take the argument as WebLogic Server instance name and give us the output as statistics of Young, Old, Perm Generations Current, Used Memory details in MB conversion function.

##############################################################
# Purpose:  To display current, used spaces for an instance 
#   in the JVM Eden, Old, Perm sizes
# Author:  Pavan Devarakonda
##############################################################
# Function for converting the Kilo bytes to Mega Bytes.
toMb()
{
        echo 'scale=0;'$1'/1024' | bc; 
} 

#=============================================
# JVM status Function starts here
#=============================================
instJvm()
{
        DATA=`$JAVA_HOME/bin/jstat -gc $PID | grep -v S0C`
         
        EC=`echo $DATA | awk '{print $5}'`
        EU=`echo $DATA | awk '{print $6}'`
        OC=`echo $DATA | awk '{print $7}'`
        OU=`echo $DATA | awk '{print $8}'`
        PC=`echo $DATA | awk '{print $9}'`
        PU=`echo $DATA | awk '{print $10}'`
        echo -e  "$2\t|" `toMb $EC`"M   "`toMb $EU`"M\t| "`toMb $OC`"M   "`toMb $OU`"M\t| "`toMb $PC`"M   "`toMb $PU`"M"
}
#=============================================
# main starts here
#=============================================
 
 
echo -e "=============================================================="
echo -e " Instance\t |    Eden    \t|    Old Gen   \t|    Perm Gen"
echo -e "         \t |Current Used\t| Current Used \t| Current Used"
echo -e "=============================================================="
 
DOMAIN_HOME=$1

for i in ` ls $DOMAIN_HOME/servers/|grep -v bkp`
do 
        PID=`$JAVA_HOME/bin/jps -lv | grep $i | awk '{print $1}'`
        if [ "$PID" != ""  ]
        then    
                instJvm $PID $i
        fi      
done
echo -e "=============================================================="

Example execution:

  ./jstat_mon.sh  /u01/domains/apex_domain
  

The output as follows:

The jstat command using script

        

Taking Advantage of Naming Conventions

In most of the Middleware environments, the WebLogic domain will have admin servers, managed servers use naming convention construct that will have some common identity for each domain unique. In the above example, The domain uses naming convention with the letter 'c', This can be any word or letter that is used in your environment. To make your script you need to replace 'c' with your enviroment key word/letter. (Thanks Raman Jain for your valuable comment)

Let me show you the sample Output:

Looking ahead to your valuable comments and suggestions to improve our scripting tricks for better WebLogic Administration.

The jstat Command reference:

http://roundoverlinux.blogspot.in/2013/03/jstat-command-is-java-virtual-machine.html