Monday, March 17, 2014

How to disable and enable CRS services in Oracle RAC 10g

This will perform in  one of the cases of OS Raid backup ,for that we need to disable the CRS so that OS Raid backup will be taken and after that we need to enable CRS services ...The flow will be like below

Disable CRS

1)Pre health checks
2) Shutdown MTservices and DB services including Apps/DB listeners
3) Stop TEST and the cluster stack
logon on DB hosts host1 and host2 as root

To become 'root' on the servers, log in as 'custuser' and then su - You are then prompted for a password  - enter the 'root passwd' password again.    

4) Check CRS status:

$ /oracrs/oracle/product/102/bin/crs_stat -t

[custuser@host2 ~]$ su -
Password:

[root@host2 ~]# /oracrs/oracle/product/102/bin/crs_stat -t

Name           Type           Target    State     Host       
------------------------------------------------------------

ora....I1.inst application    OFFLINE   OFFLINE              

ora....I2.inst application    OFFLINE   OFFLINE              

ora.TEST.db  application    OFFLINE   OFFLINE              

ora....I1.lsnr application    ONLINE    OFFLINE              

ora....450.gsd application    ONLINE    ONLINE    host1 

ora....450.ons application    ONLINE    OFFLINE              

ora....450.vip application    ONLINE    ONLINE    host1 

ora....I2.lsnr application    ONLINE    OFFLINE              

ora....451.gsd application    ONLINE    ONLINE    host2 

ora....451.ons application    OFFLINE   OFFLINE              

ora....451.vip application    ONLINE    ONLINE    host2 
[root@host2 ~]#

[root@host1 ~]# /oracrs/oracle/product/102/bin/crs_stat -t

Name           Type           Target    State     Host       
------------------------------------------------------------

ora....I1.inst application    OFFLINE   OFFLINE              

ora....I2.inst application    OFFLINE   OFFLINE              

ora.TEST.db  application    OFFLINE   OFFLINE              

ora....I1.lsnr application    ONLINE    OFFLINE              

ora....450.gsd application    ONLINE    ONLINE    host1 

ora....450.ons application    ONLINE    OFFLINE              

ora....450.vip application    ONLINE    ONLINE    host1 

ora....I2.lsnr application    ONLINE    OFFLINE              

ora....451.gsd application    ONLINE    ONLINE    host2 

ora....451.ons application    OFFLINE   OFFLINE              

ora....451.vip application    ONLINE    ONLINE    host2 
[root@host1 ~]#

$ /oracrs/oracle/product/102/bin/crsctl check crs

[root@host2 ~]# /oracrs/oracle/product/102/bin/crsctl check crs
CSS appears healthy
CRS appears healthy
EVM appears healthy
[root@host2 ~]#

[root@host1 ~]# /oracrs/oracle/product/102/bin/crsctl check crs
CSS appears healthy
CRS appears healthy
EVM appears healthy
[root@host1 ~]#

5) Stop CRS:

$ /oracrs/oracle/product/102/bin/crsctl stop crs

[root@host2 ~]# /oracrs/oracle/product/102/bin/crsctl stop crs
Stopping resources. This could take several minutes.
Successfully stopped CRS resources.
Stopping CSSD.
Shutting down CSS daemon.
Shutdown request successfully issued.

[root@host2 ~]# /oracrs/oracle/product/102/bin/crsctl check crs
Failure 1 contacting CSS daemon
Cannot communicate with CRS
Cannot communicate with EVM
[root@host2 ~]#

[root@host1 ~]# /oracrs/oracle/product/102/bin/crsctl stop crs
Stopping resources. This could take several minutes.
Successfully stopped CRS resources.
Stopping CSSD.
Shutting down CSS daemon.
Shutdown request successfully issued.

[root@host1 ~]# /oracrs/oracle/product/102/bin/crsctl check crs
Failure 1 contacting CSS daemon
Cannot communicate with CRS
Cannot communicate with EVM
[root@host1 ~]#

6) Disable CRS

$ /oracrs/oracle/product/102/bin/crsctl disable crs

[root@host2 ~]# /oracrs/oracle/product/102/bin/crsctl disable crs
[root@host2 ~]#

[root@host1 ~]# /oracrs/oracle/product/102/bin/crsctl disable crs
[root@host1 ~]#

Check there is no CRS processe running:

$ ps -ef | grep crs

[root@host2 ~]# ps -ef | grep crs

root     32000     1  0 06:14 ?        00:00:00 /bin/sh /etc/init.d/init.crsd run
root       329 30870  0 06:15 pts/3    00:00:00 grep crs
[root@host2 ~]#

[root@host1 ~]# ps -ef | grep crs

root     13160     1  0 06:14 ?        00:00:00 /bin/sh /etc/init.d/init.crsd run
root     14256 12229  0 06:15 pts/2    00:00:00 grep crs
[root@host1 ~]#

[root@host2 ~]#ps -ef|grep d.bin

Enable CRS

1) CRS (Cluster stack) has to be started before DBs and MTs startup

logon on DB hosts 450/451 as root To become 'root' on the servers, log in as 'custuser' and then su -
You are then prompted for a password  - enter the 'root password' password again.

2) Enable CRS

$ /oracrs/oracle/product/102/bin/crsctl enable crs

3) startup CRS

$ /oracrs/oracle/product/102/bin/crsctl start crs

4) Check if CRS was properly enabled and started

$ ps -ef | grep crs

$ /oracrs/oracle/product/102/bin/crsctl check crs

$ /oracrs/oracle/product/102/bin/crs_stat -t

$ ps -ef|grep d.bin

5) startup RAC database & listeners

6) startup MTs services

Example

[root@host1 ~]# /oracrs/oracle/product/102/bin/crsctl enable crs

[root@host1 ~]# /oracrs/oracle/product/102/bin/crsctl start crs 

Attempting to start CRS stack

[root@host1 ~]# ps -ef|grep crs

root      5228     1  0 10:37 ?        00:00:00 /bin/sh /etc/init.d/init.crsd run

root     11831  5226  0 16:49 ?        00:00:00 /bin/su -l ororacrs -c sh -c 'ulimit -c unlimited;
cd /oracrs/oracle/product/102/log/host1/evmd; exec /oracrs/oracle/product/102/bin/evmd '

ororacrs 11836 11831  0 16:49 ?        00:00:00 /oracrs/oracle/product/102/bin/evmd.bin

root     11897  5228  0 16:49 ?        00:00:01 /oracrs/oracle/product/102/bin/crsd.bin reboot

root     12249 11909  0 16:49 ?        00:00:00 /oracrs/oracle/product/102/bin/oprocd.bin run -t
1000 -m 10000 -hsi 5:10:50:75:90 -f

root     12256 11927  0 16:49 ?        00:00:00 /sbin/runuser -l ororacrs -c /bin/sh -c 'cd
/oracrs/oracle/product/102/log/host1/cssd/oclsomon; ulimit -c unlimited;
/oracrs/oracle/product/102/bin/oclsomon  || exit $?'

ororacrs 12257 12256  0 16:49 ?        00:00:00 /bin/sh -c cd
/oracrs/oracle/product/102/log/host1/cssd/oclsomon; ulimit -c unlimited;
/oracrs/oracle/product/102/bin/oclsomon  || exit $?

ororacrs 12279 12257  0 16:49 ?        00:00:00 /oracrs/oracle/product/102/bin/oclsomon.bin

ororacrs 12415 11971  0 16:49 ?        00:00:00 /oracrs/oracle/product/102/bin/ocssd.bin

ororacrs 12577 11836  0 16:49 ?        00:00:00 /oracrs/oracle/product/102/bin/evmlogger.bin -o
/oracrs/oracle/product/102/evm/log/evmlogger.info -l
/oracrs/oracle/product/102/evm/log/evmlogger.log

root     17757 10745  0 16:59 pts/1    00:00:00 grep crs

[root@host1 ~]# /oracrs/oracle/product/102/bin/crsctl check crs
CSS appears healthy
CRS appears healthy
EVM appears healthy

[root@host1 ~]# /oracrs/oracle/product/102/bin/crs_stat -t

Name           Type           Target    State     Host       
-----------------------------------------------------------

ora....I1.inst application    ONLINE    ONLINE    host1 

ora....I2.inst application    ONLINE    ONLINE    host2 

ora.TEST.db  application    ONLINE    ONLINE    host1 

ora....I1.lsnr application    ONLINE    ONLINE    host1 

ora....450.gsd application    ONLINE    ONLINE    host1 

ora....450.ons application    ONLINE    OFFLINE              

ora....450.vip application    ONLINE    ONLINE    host1 

ora....I2.lsnr application    ONLINE    ONLINE    host2 

ora....451.gsd application    ONLINE    ONLINE    host2 

ora....451.ons application    ONLINE    OFFLINE              

ora....451.vip application    ONLINE    ONLINE    host2 
[root@host1 ~]#

Trouble shooting of ONS

ons services are showing OFFLINE,can we ignore this?

[root@host1 ~]#  /oracrs/oracle/product/102/bin/crs_stat -t

Name           Type           Target    State     Host       
------------------------------------------------------------

ora....I1.inst application    ONLINE    ONLINE    host1 

ora....I2.inst application    ONLINE    ONLINE    host2 

ora.TEST.db  application    ONLINE    ONLINE    host1 

ora....I1.lsnr application    ONLINE    ONLINE    host1 

ora....450.gsd application    ONLINE    ONLINE    host1 

ora....450.ons application    ONLINE    OFFLINE              

ora....450.vip application    ONLINE    ONLINE    host1 

ora....I2.lsnr application    ONLINE    ONLINE    host2 

ora....451.gsd application    ONLINE    ONLINE    host2 

ora....451.ons application    ONLINE    OFFLINE              

ora....451.vip application    ONLINE    ONLINE    host2 
[root@host1 ~]#

[root@host1 pserv19]# /oracrs/oracle/product/102/bin/onsctl ping
ons is not running ...

[root@host1 pserv19]# env | grep LD_LIBRARY_PATH
[root@host1 pserv19]# env | grep LIBHOME
[root@host1 pserv19]# echo $CRS_HOME

[root@host1 102]# pwd
/oracrs/oracle/product/102
[root@host1 102]# cd opmn

[root@host1 opmn]# ls
bin  conf  lib  logs  mesg
[root@host1 opmn]# cd conf

[root@host1 conf]# ls
ons.config.backup.10203  ons.config.orig  ons.config.tmp
[root@host1 conf]#

[root@host1 conf]# pwd
/oracrs/oracle/product/102/opmn/conf
[root@host1 conf]# ls -ltr

total 12
-rw-rw-r--  1 ororacrs dboracrs 71 Feb 21  2006 ons.config.tmp
-rw-rw-r--  1 ororacrs dboracrs 76 Jan  4  2007 ons.config.orig
-rw-------  1 ororacrs dboracrs 76 Jan  4  2007 ons.config.backup.10203
[root@host1 conf]#

Analysis

ONS.CONF doesn't exist
ONS is not configured

No comments:

Post a Comment