Single click installation of WebLogic 14c|12c in Linux

Current IT Trending towards devOps automation in every Software product development. I am also part of  this devOps culture. I had a thought that, How much I can leverage my regular task by making simple shell script for Installation task.

As every Oracle WebLogic system administrator would pass through this situation but only few of them who have "AUTOMATION IN MINDS" could attempt to resolve this with a nicely blended Shell script.

Here I would like to target two types of installations with two different Shell scripts:

1. Fusion Middleware 12.2.1.2.0 WebLogic Server installation
2. Fusion Middleware 12.2.1.2.0 Infrastructure installation

Objective of this script is to get single hit would do multiple tasks such as:

• JAVA installation
• setting the JAVA_HOME in shell profile
• using Java run the WebLogic installer in silent mode

YOu can use the same logic in WebLogic 14c the latest version as well.
Assumptions:

• Downloaded Software will be stored in common shared location. 
• Installation of software such as JDK, WebLogic will be same mount point and directory structure
• Linux Operating System user have bash shell!   

Knowledge base:

• Silent Mode installation of WebLogic 12c
• WebLogic domain with WLST

Auto Install pre-requisite for Application - Script 

wls.cfg

JDK_INSTALLER=jdk-8u*-linux-x64.tar.gz
JDK_INSTALLER_PATH=/u01/app/software/jdk
INSTAL_LOC=/u01/app/oracle
WLS_INSTALLER=fmw_12.*_wls.jar
WLS_INSTALLER_PATH=/u01/app/software/FMW_wls
SILENT_DIR=/u01/app/software/silent

Script name: install_wls.sh

#!/bin/bash
# This script will be installing JDK
# installing WebLogic in silent mode
#
#- Info 
#-    Author   : Pavan Devarakonda
#-    Date     : 30-March-2017
##############################################

. wls.cfg
echo "installing JDK ${JDK_INSTALLER}"
cd $INSTAL_LOC
tar -zxvf $JDK_INSTALLER_PATH/$JDK_INSTALLER

ls -l
echo confirm JDK installation
cd $INSTAL_LOC/jdk*

JAVA_HOME=`pwd`
echo "export JAVA_HOME=$JAVA_HOME">>$HOME/.bashrc
echo "export PATH=$JAVA_HOME/bin:$PATH">>$HOME/.bashrc

. $HOME/.bashrc

echo "Check for java version"
java -version

echo Installing WebLogic now...
echo .

java -jar $WLS_INSTALLER_PATH/$WLS_INSTALLER -silent -invPtrLoc $SILENT_DIR/oraInst.loc -responseFile $SILENT_DIR/wls_install.rsp

echo "WebLogic $WLS_INSTALLER installed successfully...."
echo .
echo setting MW_HOME ...
MW_HOME=`grep ORACLE_HOME $SILENT_DIR/wls_install.rsp |cut -d'=' -f2`
echo "export MW_HOME=$MW_HOME">>$HOME/.bashrc
echo "export WL_HOME=$MW_HOME/wlserver">>$HOME/.bashrc
echo "alias wlst=$MW_HOME/oracle_common/common/bin/wlst.sh">>$HOME/.bashrc
. $HOME/.bashrc
echo "Use wlst command to test everything working..."

Oracle Fusion Middleware Infrastructure installation Shell Script

Here the script is little bit improved the installer normally coming as zip file which can be uncompressed by script and also silent file will be using

INSTALL_TYPE=Fusion Middleware Infrastructure With Examples

Now the script for Infrastructure will be look like this.

#!/bin/bash
# This script will be installing JDK
# installing WebLogic in silent mode
#
JDK_INSTALLER=jdk-8u121-linux-x64.tar.gz
JDK_INSTALLER_PATH=/u01/app/software/jdk
INSTAL_LOC=/u01/app/oracle

echo "installing JDK ${JDK_INSTALLER}"
cd $INSTAL_LOC
tar -zxvf $JDK_INSTALLER_PATH/$JDK_INSTALLER

ls -l
echo confirm JDK installation
cd $INSTAL_LOC/jdk*

JAVA_HOME=`pwd`
echo "export JAVA_HOME=$JAVA_HOME">>$HOME/.bashrc
echo "export PATH=$JAVA_HOME/bin:$PATH">>$HOME/.bashrc

. $HOME/.bashrc

echo "Check for java version"
java -version

echo Installing WebLogic now...
echo .

WLS_INSTALLER=fmw_12.2.1.2.0_infrastructure_Disk1_1of1.zip
WLS_INSTALLER_PATH=/u01/app/software/FMW_wls
cd $WLS_INSTALLER_PATH
unzip $WLS_INSTALLER_PATH/$WLS_INSTALLER

INFRA_JAR=fmw_12.2.1.2.0_infrastructure.jar

SILENT_DIR=/u01/app/software/silent

java -jar $WLS_INSTALLER_PATH/$INFRA_JAR -silent -invPtrLoc $SILENT_DIR/oraInst.loc -responseFile $SILENT_DIR/fmw12.2.1_infra_install.rsp

echo "FMW Infrastructure $WLS_INSTALLER installed successfully...."
echo .
echo setting MW_HOME ...
MW_HOME=`grep ORACLE_HOME $SILENT_DIR/$SILENT_DIR/fmw12.2.1_infra_install.rsp |cut -d'=' -f2`
echo "export MW_HOME=$MW_HOME">>$HOME/.bashrc
echo "export WL_HOME=$MW_HOME/wlserver">>$HOME/.bashrc
echo "alias wlst=$MW_HOME/oracle_common/common/bin/wlst.sh">>$HOME/.bashrc
. $HOME/.bashrc
echo "Use wlst command to test everything working..."

Note: If you are running other than Linux platform you can use ksh shell .profile.

keep in touch with your comment and suggestions for improve more effective way.

cheers!!!

Looking into Middleware server logs made simple

Middleware Admin means the day runs with Emergency calls or On-calls, supporting team task always under high pressure cooker -- many of them feel tension tension due to heat in the head many got into hair-fall!!! ...

Its a challenging task, only those who can handle the jobs under pressure can do this with tricks. But When you don't have calls you might think to reduce your stress in the work with smart automation scripting to make life more easier.

Best Practices for WebLogic Logs

If the environment is shorter let say it is size of 10 - 20 managed servers, then the best idea is to implement is that all these managed server logs, JMS stores must shared a common mount point. When application support team run into issues, this will give you the flexibility to search the logs easily and faster way to get rid of issues with this unique log location.

In most of your project environments you might find the multiple logs such as application reated, JMS message stores or some transactional logs. In some critical situations, where you might need to look-up for a particular phrase of text in the WebLogic managed server generated logs. If you already follows best practices for collecting logs all together. It is easy to search in the sub sequant sub directories.

#FileName: CheckLogs.sh
#This script is used to find the given pharse in all logs available in the machine

INST_HOME="/home/wlusr/instances"
INSTANCES=`ls $INST_HOME|grep c`
phrase=$1
for instance in $INSTANCES
do
        echo 'Checking for the :' $instance
        cd $INST_HOME/$instance/logs
        egrep -i $phrase c*log*
        egrep -i $phrase c*out
done

If you keep above script in the general accessible path, such a way that it can be run from any directory on your machine. The Best practices for every WebLogic Middleware server running machines uses JAVA_HOME in their PATH. The best option to store this script is $JAVA_HOME/bin you can run from any path in your user.

Suppose you want to find all the WebLogic Servers which got RUNNING state on a machine. The search pattern you can give as bea-000360.

$ logverify bea-000360
Checking for the : cmgdserver01
 
      

Multi server code Backup

If you digest Operating Systems basics or SHELL basics, you might think much effectively and you can make  more optimized ways to leverage your work, you can design your script with enhanced ability with productivity. Once you taste that flavor in my scripts, you can feel that and understand that why did I told those exaggerated words. Let me share my experience on this.

In every J2EE production environment is going to have an application code (most of the developer say it the composition of 'dist' and 'properties'), which could be no-stage mode deployment style. Where you need to have a copy of code on every WebLogic server running machine. For every production release there could be success or fail. A wise WebLogic Administrator always takes the backup of the existing code and then go further to do fresh code deployment.

According to the best practices of WebLogic deployment process, First time you will do the sample test on the production environment by making single site available for test. That is if your application having EJB-Tier, Web-tier then, start each one server in the tier. Test the basic flow test, which makes you know wheather your new code works or not. Most of the cases the code come to production move is already checked on Staging environment which is the simulated environment of production environment.
It is always good to do double check your work before you do something on Production environments.

Here the important point is that to overcome failure situation, when a new changes in the code fails , such cases alternate solution is to revert back the old code.

How to do this?
Think!! Think wisely, Think for optimizing, Think for better outputs...

We can do simple copy command but what makes error-less work
Take backup with date and time stamp.
There could be multiple application codes on the same machine.
By accomplishing these above task with the following script on a machine.

clear
if [ $# -gt 0 ]; then
 today=`date "+%b%d_%y"`
 APPNAME=$1
 SRC=$HOME/$APPNAME
 BKP=$HOME/backups/$APPNAME
 mkdir -p $BKP
 BKP=$HOME/backups/"$APPNAME"_"$today"
 echo $SRC
 echo $BKP
  
 echo '1. Code backup [dist]only'
 echo '2. Full backup [includes dist, properties]'
 read -p "Please enter your option [1/2]:"  opt
       if [ $opt -eq 1 ] || [ $opt -eq 2 ]; then 
  case "$opt" in
  
 1)  echo "copying dist ..."
  cp -rp $SRC/dist $BKP
     ;;
 2)  echo  "copying all including properties..."
  cp -rp $SRC/dist $BKP
  cp -rp $SRC/properties $BKP
     ;;
 esac
        else
                echo "  invalid option "
        fi
else
    echo "Please enter application name in command line arguments"
fi

Multiple site backup
The above backup is required for every site (multiple machines). If you have already used ssh key generated and all sites are accessible without password. In Ramayan Ram's single arrow hits seven trees in a row. Similarly I want to make the backup script run on one machine and hit on every machine where the application is running. But to make this idea come true I took the help of LinkedIn discussion. Mr. John suggested to keep CMD and execute ssh command.

clear
today=`date "+%b%d_%y"`
if [ $# -gt 0 ]
then
        APPNAME=$1
        SRC=$HOME/$APPNAME
        BKP=$HOME/backups/"$APPNAME"_"$today"
        echo '1. Code backup only'
        echo '2. Full backup [includes properties]'
        read -p "Please enter your option [1/2]:" opt
        if [ $opt -eq 1 ] || [ $opt -eq 2 ]
        then
                CMD="mkdir -p $BKP"
                case "$opt" in
                1 )
                        echo "Will copy only dist."
                        CMD="$CMD ;    cp -rp $SRC/dist $BKP ;     ls -lrtR $BKP ;   exit"
                        ;;
                  2 )
                        echo "Will copy dist and properties."
                        CMD="$CMD ;    cp -rp $SRC/dist $SRC/properties $BKP ;    ls -lrtR $BKP ;   exit"
                        ;;
                 esac
  # serverlist is the text file
                for i in `cat serverlist`
                do
                        echo "Connecting to $i."
                        ssh -a $i $CMD
                done
                echo "done !"
           else
                echo " invalid option "
          fi
else
        echo "Please enter application name in command line arguments"
fi

This script worked... awesome

Copying to multiple remote machines

Here is another interesting story of WLA (of-course mine), When I visited US in 2007 there was lot of restrictions in work places. "Hey its production you know what happen if you touch it??" "Don't open this files", "Don't enter into that folders", it will be dangerous... I know that very well what is missing in the system, where it is required a change but my hands kept criss cross!!

Days passed I got opportunity to come again on long term. Now, the whole new System is going shapeup with my hands. The system is awaiting for me since long days. :) All those sparkling colorful ideas running around my mind, got chance to flow onto the system to form various automated scripts, which are having little in size with greater capabilities.

Whenever there is a application version release the archive files(.jar, .war, .ear) need to copied to all over the remote machines. In olden days we were using 'sftp' command and its related 'put', 'mput', 'get' and 'mget' commands to complete the task. Manually double checking wheather the copying is done correct or not, by verifying in each machine content by comparing the each file sizes. Here I found a flaw that there could be chance of human error. While understanding 'Six Sigma Course', where I learnt about human errors makes greater defect to many customer's business. To avoid this better option is automation of the task as possible as much.I remembered Mr. Neil Gogte words about "Cyber Coolie". The software engineer who works as per his contractor asked him to do only those things he will do. Never think other than the work which is assigned to him. My soul shouts out 'NO!!', I cannot be a Cyber coolie any more !!

My beautiful sparkling colorful ideas SSH password less connection to multiple remote machines, powerful bright idea of 'scp' command usage with verification option built within a script come out as a wonderful shell script, which had mighty power of built-in checking with no chances of human error. When I show the execution of this script to my teammates they are very happy and appreciated me. Many productive hours are saved though this activity was disturging to other regular job. The script made almost hands free task!! Finally, That's the way team turn happy ever by using the easy script.

Script is :
TADAA!!!!!!!!!!

# Define variables values
src=
target=
hostlist=
user=
Logfile=

#=== script logic starts here ====
if [ -d $src ]
then
   echo "Code folder found, preparing to transfer\n"
   while read server
   do
 result=scp -r $src $user@${server}:$target
 if [ $result -eq 0 ] 
 then
  echo $server transfer done >> $Logfile
 else
  echo $server transfer failed.
  exit
 fi
   done < $hostlist
else
   echo "Code folder \"$src\" not found\n"
fi

JVM monitoring with jstat Shell Script

The Oracle WebLogic Server instances are nothing but each one running on a single JVM, when this JVM crashes there could be one of the reasons as overworking of Garbage Collection or not working at all (waiting continuously).  

It is good practice that monitoring GC in JVM with detailed statistics will give you a clear idea to analysis at what circumstances happening wrong. And the best way to look in deeper levels of garbage collection also like Young Generation (Eden space, Survivor spaces S0, S1) Old Generation (tenured Generation), and Perm Generation (Statistic Objects/Classes).

JDK 1.5 and latest providing excellent JDK command utilities for interrogate the current running Java Process and look inside of JVM take snap with following:
1. jps (Java Process)
2. jstat (JVM status)

The 'jps' command with -lv options gives you complete detailed java process arguments for MEM settings and relavent WebLogic Server instances name. We have already discussed about this command utility in another post.

The jstat command with -gc option will produce the Garbage Collection of given java process id.

GC Monitoring shell script with jstat tool

I have modified according to my convenience and requirements as a shell script that will take the argument as WebLogic Server instance name and give us the output as statistics of Young, Old, Perm Generations Current, Used Memory details in MB conversion function.

##############################################################
# Purpose:  To display current, used spaces for an instance 
#   in the JVM Eden, Old, Perm sizes
# Author:  Pavan Devarakonda
##############################################################
# Function for converting the Kilo bytes to Mega Bytes.
toMb()
{
        echo 'scale=0;'$1'/1024' | bc; 
} 

#=============================================
# JVM status Function starts here
#=============================================
instJvm()
{
        DATA=`$JAVA_HOME/bin/jstat -gc $PID | grep -v S0C`
         
        EC=`echo $DATA | awk '{print $5}'`
        EU=`echo $DATA | awk '{print $6}'`
        OC=`echo $DATA | awk '{print $7}'`
        OU=`echo $DATA | awk '{print $8}'`
        PC=`echo $DATA | awk '{print $9}'`
        PU=`echo $DATA | awk '{print $10}'`
        echo -e  "$2\t|" `toMb $EC`"M   "`toMb $EU`"M\t| "`toMb $OC`"M   "`toMb $OU`"M\t| "`toMb $PC`"M   "`toMb $PU`"M"
}
#=============================================
# main starts here
#=============================================
 
 
echo -e "=============================================================="
echo -e " Instance\t |    Eden    \t|    Old Gen   \t|    Perm Gen"
echo -e "         \t |Current Used\t| Current Used \t| Current Used"
echo -e "=============================================================="
 
DOMAIN_HOME=$1

for i in ` ls $DOMAIN_HOME/servers/|grep -v bkp`
do 
        PID=`$JAVA_HOME/bin/jps -lv | grep $i | awk '{print $1}'`
        if [ "$PID" != ""  ]
        then    
                instJvm $PID $i
        fi      
done
echo -e "=============================================================="

Example execution:

  ./jstat_mon.sh  /u01/domains/apex_domain
  

The output as follows:

The jstat command using script

        

Taking Advantage of Naming Conventions

In most of the Middleware environments, the WebLogic domain will have admin servers, managed servers use naming convention construct that will have some common identity for each domain unique. In the above example, The domain uses naming convention with the letter 'c', This can be any word or letter that is used in your environment. To make your script you need to replace 'c' with your enviroment key word/letter. (Thanks Raman Jain for your valuable comment)

Let me show you the sample Output:

Looking ahead to your valuable comments and suggestions to improve our scripting tricks for better WebLogic Administration.

The jstat Command reference:

http://roundoverlinux.blogspot.in/2013/03/jstat-command-is-java-virtual-machine.html