UDE-00008: operation generated ORACLE error 3113

Issue :- 

Export backup is failed due to below error.

Error:-

UDE-00008: operation generated ORACLE error 3113

ORA-03113: end-of-file on communication channel

Cause:-

The initialization parameter AQ_TM_PROCESSES is set to 0
Datapump uses  Advanced Queueing and needs this parameter to be set higher then 0.

Solution:-

Need to set advance queuing parameter to 1.

Also Make sure that you have enough RAM and paging space on your system

In my case , I had ran export using parallel=3 & faced above issue

if possible , Do not run export using parallel process.

Current Value for :-

NAME               TYPE        VALUE

------------------ ----------- -------

aq_tm_processes    integer     0

Updated Value:-

NAME               TYPE        VALUE

------------------ ----------- -------

aq_tm_processes    integer     1

[SRVM(1337)]CRS-10051: CVU found following errors with Clusterware setup : PRVF-5301 : Failed to locate CRS home

Error:-

I was working on issue where cluster is not coming up on node 1 but it is running on node 2.
Following error has been recorded.

Go to    Cd /rdbms/app/oracle/diag/crs/SERVER1/crs/trace
 do      vi alert.log

2017-07-11 14:44:39.105 [CLSECHO(9310)]CRS-10001: 11-Jul-17 14:44 AFD-9204: false
2017-07-11 20:43:52.618 [SRVM(1337)]CRS-10051: CVU found following errors with Clusterware setup : PRVF-5301 : Failed to locate CRS home
PRVF-7573 : Sufficient swap size is not available on node "INP44XPDB1283" [Required = 15.5079GB (1.6261252E7KB) ; Found = 7.8125GB (8191996.0KB)]
PRVG-2033 : Permissions of file "/etc/oracle/olr.loc" did not match the expected octal value on node "INP44XPDB1283". [Expected = "0640" ; Found = "0644"]

PRVG-2031 : Owner of file "/rdbms/app/12.1.0.2/grid/cdata/SERVER1.olr" did not match the expected value on node "INP44XPDB1283". [Expected = "ora12c" ; Found = "root"]

Also we suspect that this may be due DB patching . If you apply patches on RAC1 then in some cases RAC2 cluster may not come up due different patch level.
Error:-
CRS-6706: Oracle Clusterware Release patch level ('1009910258') does not match Software patch level ('93699375'). Oracle Clusterware cannot be started.
CRS-4000: Command Start failed, or completed with errors.

Refer below link for Patch level issue

 CRS-6706: Oracle Clusterware Release patch level ('mmmmm') does not match Software patch level ('nnnnn'). Oracle Clusterware cannot be started

Solution:-

Before performing below steps make sure that you have proper DB backup in case of recovery.

Step1 :Stopped cluster on SERVER2        //Running Node

Step2 :  Run on problematic node 

[ora12c@SERVER1 ~]$
[ora12c@SERVER1 install]$ pwd                            // Go to crs path 
/rdbms/app/12.1.0.2/grid/crs/install

[ora12c@SERVER1 install]$ ls -lrt root*
-rwxr-xr-x. 1 root oinstall  9467 Mar 10  2015 roothas.pl
-rwxr-xr-x. 1 root oinstall 17971 Mar 10  2015 rootcrs.pl
-rwxr-x---. 1 root oinstall   822 May  3 21:32 roothas.sh
-rwxr-x---. 1 root oinstall   822 May  3 21:32 rootcrs.sh

[ora12c@SERVER1 install]$ pwd
/rdbms/app/12.1.0.2/grid/crs/install

[ora12c@SERVER1 install]$ sudo /rdbms/app/12.1.0.2/grid/crs/install/rootcrs.sh -deconfig -force                          // To deconfigure grid
Using configuration parameter file: /rdbms/app/12.1.0.2/grid/crs/install/crsconfig_params
Usage: srvctl <command> <object> [<options>]
    commands: enable|disable|start|stop|status|add|remove|modify|update|getenv|setenv|unsetenv|config|upgrade|downgrade
    objects: database|service|asm|diskgroup|listener|home|ons
For detailed help on each command and object and its options use:
  srvctl <command> -help [-compatible] or
  srvctl <command> <object> -help [-compatible]
PRKO-2012 : nodeapps object is not supported in Oracle Restart
CRS-4013: This command is not supported in a single-node configuration.
CRS-4000: Command Stop failed, or completed with errors.
2017/08/10 12:55:40 CLSRSC-463: The deconfiguration or downgrade script could not stop current Oracle Clusterware stack.

2017/08/10 12:55:41 CLSRSC-557: Oracle Clusterware stack on this node has been successfully deconfigured. There were some errors which can be ignored.

[ora12c@SERVER1 install]$ cd ..
[ora12c@SERVER1 crs]$ cd ..
[ora12c@SERVER1 grid]$ ls -lrt root.sh
-rwxr-x---. 1 ora12c oinstall 495 May  3 21:29 root.sh
[ora12c@SERVER1 grid]$ pwd                                //Go to grid path
/rdbms/app/12.1.0.2/grid                                                      
[ora12c@SERVER1 grid]$ sudo /rdbms/app/12.1.0.2/grid/root.sh
Check /rdbms/app/12.1.0.2/grid/install/root_SERVER1.vsnl.co.in_2017-08-10_12-55-56.log for the output of root script
[ora12c@SERVER1 grid]$

ASM & DB will automatically start on SERVER1

Step 3 :- Start cluster on SERVER2

OUTPUT:- 

Output of /rdbms/app/12.1.0.2/grid/install/root_SERVER1.vsnl.co.in_2017-08-10_12-55-56.log file

[ora12c@SERVER1 ~]$ tail -f /rdbms/app/12.1.0.2/grid/install/root_SERVER1.vsnl.co.in_2017-08-10_13-01-13.log

   Copying coraenv to /usr/local/bin ...

Entries will be added to the /etc/oratab file as needed by

Database Configuration Assistant when a database is created

Finished running generic part of root script.

Now product-specific root actions will be performed.

Relinking oracle with rac_on option

Using configuration parameter file: /rdbms/app/12.1.0.2/grid/crs/install/crsconfig_params

2017/08/10 13:01:14 CLSRSC-4001: Installing Oracle Trace File Analyzer (TFA) Collector.

2017/08/10 13:03:00 CLSRSC-4002: Successfully installed Oracle Trace File Analyzer (TFA) Collector.

2017/08/10 13:03:01 CLSRSC-363: User ignored prerequisites during installation

2017/08/10 13:04:06 CLSRSC-330: Adding Clusterware entries to file 'oracle-ohasd.service'

CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'SERVER1'

CRS-2673: Attempting to stop 'ora.gpnpd' on 'SERVER1'

CRS-2673: Attempting to stop 'ora.mdnsd' on 'SERVER1'

CRS-2673: Attempting to stop 'ora.gipcd' on 'SERVER1'

CRS-2673: Attempting to stop 'ora.evmd' on 'SERVER1'

CRS-2677: Stop of 'ora.gpnpd' on 'SERVER1' succeeded

CRS-2677: Stop of 'ora.mdnsd' on 'SERVER1' succeeded

CRS-2677: Stop of 'ora.evmd' on 'SERVER1' succeeded

CRS-2677: Stop of 'ora.gipcd' on 'SERVER1' succeeded

CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'SERVER1' has completed

CRS-4133: Oracle High Availability Services has been stopped.

CRS-4123: Oracle High Availability Services has been started.

CRS-2672: Attempting to start 'ora.evmd' on 'SERVER1'

CRS-2672: Attempting to start 'ora.mdnsd' on 'SERVER1'

CRS-2676: Start of 'ora.evmd' on 'SERVER1' succeeded

CRS-2676: Start of 'ora.mdnsd' on 'SERVER1' succeeded

CRS-2672: Attempting to start 'ora.gpnpd' on 'SERVER1'

CRS-2676: Start of 'ora.gpnpd' on 'SERVER1' succeeded

CRS-2672: Attempting to start 'ora.cssdmonitor' on 'SERVER1'

CRS-2672: Attempting to start 'ora.gipcd' on 'SERVER1'

CRS-2676: Start of 'ora.cssdmonitor' on 'SERVER1' succeeded

CRS-2676: Start of 'ora.gipcd' on 'SERVER1' succeeded

CRS-2672: Attempting to start 'ora.cssd' on 'SERVER1'

CRS-2672: Attempting to start 'ora.diskmon' on 'SERVER1'

CRS-2676: Start of 'ora.diskmon' on 'SERVER1' succeeded

CRS-2676: Start of 'ora.cssd' on 'SERVER1' succeeded

CRS-2672: Attempting to start 'ora.cluster_interconnect.haip' on 'SERVER1'

CRS-2672: Attempting to start 'ora.ctssd' on 'SERVER1'

CRS-2676: Start of 'ora.ctssd' on 'SERVER1' succeeded

CRS-2676: Start of 'ora.cluster_interconnect.haip' on 'SERVER1' succeeded

CRS-2672: Attempting to start 'ora.asm' on 'SERVER1'

CRS-2676: Start of 'ora.asm' on 'SERVER1' succeeded

CRS-2672: Attempting to start 'ora.storage' on 'SERVER1'

CRS-2676: Start of 'ora.storage' on 'SERVER1' succeeded

CRS-2672: Attempting to start 'ora.crf' on 'SERVER1'

CRS-2676: Start of 'ora.crf' on 'SERVER1' succeeded

CRS-2672: Attempting to start 'ora.crsd' on 'SERVER1'

CRS-2676: Start of 'ora.crsd' on 'SERVER1' succeeded

CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'SERVER1'

CRS-2673: Attempting to stop 'ora.crsd' on 'SERVER1'

CRS-2677: Stop of 'ora.crsd' on 'SERVER1' succeeded

CRS-2673: Attempting to stop 'ora.mdnsd' on 'SERVER1'

CRS-2673: Attempting to stop 'ora.storage' on 'SERVER1'

CRS-2673: Attempting to stop 'ora.gpnpd' on 'SERVER1'

CRS-2677: Stop of 'ora.storage' on 'SERVER1' succeeded

CRS-2673: Attempting to stop 'ora.crf' on 'SERVER1'

CRS-2673: Attempting to stop 'ora.ctssd' on 'SERVER1'

CRS-2673: Attempting to stop 'ora.evmd' on 'SERVER1'

CRS-2673: Attempting to stop 'ora.asm' on 'SERVER1'

CRS-2677: Stop of 'ora.mdnsd' on 'SERVER1' succeeded

CRS-2677: Stop of 'ora.gpnpd' on 'SERVER1' succeeded

CRS-2677: Stop of 'ora.crf' on 'SERVER1' succeeded

CRS-2677: Stop of 'ora.ctssd' on 'SERVER1' succeeded

CRS-2677: Stop of 'ora.evmd' on 'SERVER1' succeeded

CRS-2677: Stop of 'ora.asm' on 'SERVER1' succeeded

CRS-2673: Attempting to stop 'ora.cluster_interconnect.haip' on 'SERVER1'

CRS-2677: Stop of 'ora.cluster_interconnect.haip' on 'SERVER1' succeeded

CRS-2673: Attempting to stop 'ora.cssd' on 'SERVER1'

CRS-2677: Stop of 'ora.cssd' on 'SERVER1' succeeded

CRS-2673: Attempting to stop 'ora.gipcd' on 'SERVER1'

CRS-2677: Stop of 'ora.gipcd' on 'SERVER1' succeeded

CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'SERVER1' has completed

CRS-4133: Oracle High Availability Services has been stopped.

CRS-4123: Starting Oracle High Availability Services-managed resources

CRS-2672: Attempting to start 'ora.mdnsd' on 'SERVER1'

CRS-2672: Attempting to start 'ora.evmd' on 'SERVER1'

CRS-2676: Start of 'ora.mdnsd' on 'SERVER1' succeeded

CRS-2676: Start of 'ora.evmd' on 'SERVER1' succeeded

CRS-2672: Attempting to start 'ora.gpnpd' on 'SERVER1'

CRS-2676: Start of 'ora.gpnpd' on 'SERVER1' succeeded

CRS-2672: Attempting to start 'ora.gipcd' on 'SERVER1'

CRS-2676: Start of 'ora.gipcd' on 'SERVER1' succeeded

CRS-2672: Attempting to start 'ora.cssdmonitor' on 'SERVER1'

CRS-2676: Start of 'ora.cssdmonitor' on 'SERVER1' succeeded

CRS-2672: Attempting to start 'ora.cssd' on 'SERVER1'

CRS-2672: Attempting to start 'ora.diskmon' on 'SERVER1'

CRS-2676: Start of 'ora.diskmon' on 'SERVER1' succeeded

CRS-2676: Start of 'ora.cssd' on 'SERVER1' succeeded

CRS-2672: Attempting to start 'ora.cluster_interconnect.haip' on 'SERVER1'

CRS-2672: Attempting to start 'ora.ctssd' on 'SERVER1'

CRS-2676: Start of 'ora.ctssd' on 'SERVER1' succeeded

CRS-2676: Start of 'ora.cluster_interconnect.haip' on 'SERVER1' succeeded

CRS-2672: Attempting to start 'ora.asm' on 'SERVER1'

CRS-2676: Start of 'ora.asm' on 'SERVER1' succeeded

CRS-2672: Attempting to start 'ora.storage' on 'SERVER1'

CRS-2676: Start of 'ora.storage' on 'SERVER1' succeeded

CRS-2672: Attempting to start 'ora.crf' on 'SERVER1'

CRS-2676: Start of 'ora.crf' on 'SERVER1' succeeded

CRS-2672: Attempting to start 'ora.crsd' on 'SERVER1'

CRS-2676: Start of 'ora.crsd' on 'SERVER1' succeeded

CRS-6023: Starting Oracle Cluster Ready Services-managed resources

CRS-6017: Processing resource auto-start for servers: SERVER1

CRS-2672: Attempting to start 'ora.net1.network' on 'SERVER1'

CRS-2672: Attempting to start 'ora.oc4j' on 'SERVER1'

CRS-2676: Start of 'ora.net1.network' on 'SERVER1' succeeded

CRS-2672: Attempting to start 'ora.cvu' on 'SERVER1'

CRS-2672: Attempting to start 'ora.SERVER1.vip' on 'SERVER1'

CRS-2672: Attempting to start 'ora.inp44xpdb1284.vip' on 'SERVER1'

CRS-2672: Attempting to start 'ora.scan1.vip' on 'SERVER1'

CRS-2672: Attempting to start 'ora.scan3.vip' on 'SERVER1'

CRS-2672: Attempting to start 'ora.scan2.vip' on 'SERVER1'

CRS-2676: Start of 'ora.cvu' on 'SERVER1' succeeded

CRS-2672: Attempting to start 'ora.ons' on 'SERVER1'

CRS-2676: Start of 'ora.SERVER1.vip' on 'SERVER1' succeeded

CRS-2676: Start of 'ora.ons' on 'SERVER1' succeeded

CRS-2676: Start of 'ora.inp44xpdb1284.vip' on 'SERVER1' succeeded

CRS-2676: Start of 'ora.scan1.vip' on 'SERVER1' succeeded

CRS-2672: Attempting to start 'ora.LISTENER_SCAN1.lsnr' on 'SERVER1'

CRS-2676: Start of 'ora.scan3.vip' on 'SERVER1' succeeded

CRS-2672: Attempting to start 'ora.LISTENER_SCAN3.lsnr' on 'SERVER1'

CRS-2676: Start of 'ora.scan2.vip' on 'SERVER1' succeeded

CRS-2672: Attempting to start 'ora.LISTENER_SCAN2.lsnr' on 'SERVER1'

CRS-2676: Start of 'ora.LISTENER_SCAN1.lsnr' on 'SERVER1' succeeded

CRS-2672: Attempting to start 'ora.LISTENER.lsnr' on 'SERVER1'

CRS-2676: Start of 'ora.LISTENER_SCAN3.lsnr' on 'SERVER1' succeeded

CRS-2676: Start of 'ora.LISTENER_SCAN2.lsnr' on 'SERVER1' succeeded

CRS-2676: Start of 'ora.LISTENER.lsnr' on 'SERVER1' succeeded

CRS-2672: Attempting to start 'ora.oemdbmn.db' on 'SERVER1'

CRS-2676: Start of 'ora.oc4j' on 'SERVER1' succeeded

CRS-2676: Start of 'ora.oemdbmn.db' on 'SERVER1' succeeded

CRS-6016: Resource auto-start has completed for server SERVER1

CRS-6024: Completed start of Oracle Cluster Ready Services-managed resources

CRS-4123: Oracle High Availability Services has been started.

2017/08/10 13:07:53 CLSRSC-343: Successfully started Oracle Clusterware stack

PRKO-2190 : VIP exists for node SERVER1, VIP name SERVER1-vip.vsnl.co.in

PRKZ-1072 : SCAN name "XPDBoemdbmn" is already registered on network 1

PRCS-1028 : Single Client Access Name (SCAN) listener resources already exist on network 1

PRCN-3004 : Listener MGMTLSNR already exists

PRCR-1086 : resource ora.cvu is already registered

PRCC-1014 : scan1 was already running

PRCR-1004 : Resource ora.scan1.vip is already running

PRCR-1079 : Failed to start resource ora.scan1.vip

CRS-5702: Resource 'ora.scan1.vip' is already running on 'SERVER1'

PRCC-1014 : scan2 was already running

PRCR-1004 : Resource ora.scan2.vip is already running

PRCR-1079 : Failed to start resource ora.scan2.vip

CRS-5702: Resource 'ora.scan2.vip' is already running on 'SERVER1'

PRCC-1014 : scan3 was already running

PRCR-1004 : Resource ora.scan3.vip is already running

PRCR-1079 : Failed to start resource ora.scan3.vip

CRS-5702: Resource 'ora.scan3.vip' is already running on 'SERVER1'

PRCC-1014 : LISTENER_SCAN1 was already running

PRCR-1004 : Resource ora.LISTENER_SCAN1.lsnr is already running

PRCR-1079 : Failed to start resource ora.LISTENER_SCAN1.lsnr

CRS-5702: Resource 'ora.LISTENER_SCAN1.lsnr' is already running on 'SERVER1'

PRCC-1014 : LISTENER_SCAN2 was already running

PRCR-1004 : Resource ora.LISTENER_SCAN2.lsnr is already running

PRCR-1079 : Failed to start resource ora.LISTENER_SCAN2.lsnr

CRS-5702: Resource 'ora.LISTENER_SCAN2.lsnr' is already running on 'SERVER1'

PRCC-1014 : LISTENER_SCAN3 was already running

PRCR-1004 : Resource ora.LISTENER_SCAN3.lsnr is already running

PRCR-1079 : Failed to start resource ora.LISTENER_SCAN3.lsnr

CRS-5702: Resource 'ora.LISTENER_SCAN3.lsnr' is already running on 'SERVER1'

PRCC-1014 : cvu was already running

PRCR-1004 : Resource ora.cvu is already running

PRCR-1079 : Failed to start resource ora.cvu

CRS-5702: Resource 'ora.cvu' is already running on 'SERVER1'

Preparing packages...

cvuqdisk-1.0.9-1.x86_64

PRCN-3004 : Listener LISTENER already exists

PRCC-1015 : LISTENER was already running on SERVER1

PRCR-1004 : Resource ora.LISTENER.lsnr is already running

2017/08/10 13:08:36 CLSRSC-325: Configure Oracle Grid Infrastructure for a Cluster ... succeeded

CRS-6706: Oracle Clusterware Release patch level ('mmmmm') does not match Software patch level ('nnnnn'). Oracle Clusterware cannot be started

Error:-

[ora12c@Server1~]$ sudo /rdbms/app/12.1.0.2/grid/bin/crsctl start crs
CRS-6706: Oracle Clusterware Release patch level ('1009910258') does not match Software patch level ('93699375'). Oracle Clusterware cannot be started.
CRS-4000: Command Start failed, or completed with errors.
[ora12c@Server1 ~]$ sudo /rdbms/app/12.1.0.2/grid/bin/crsctl stop  crs -f
CRS-4639: Could not contact Oracle High Availability Services
CRS-4000: Command Stop failed, or completed with errors.


Solution :- 

This can be encountered in cases where the post patch application measures to run 'rootcrs.pl -postpatch' is not followed.

In 12c, GI home must have identical patches for the clusterware to start cluster

Login as root

Run "<GI_HOME>/crs/install/rootcrs.sh -patch"  on the problematic node and the patch level should be get corrected.

Above command will correct all issue related to patch level.

Once it is completed you can apply patch on second node.

Thats it :)

"Recovery Manager Failed to Restore Datafiles" - Using DBCA To Create Database 12.1.0.2.0 on Grid Infrastructure / ASM 12.2.0.1.0

Error:-
while creating a database using Database Configuration Assistant (DBCA) the following error is encountered:  "Recovery Manager Failed to Restore Datafiles"

"Recovery Manager Failed to Restore Datafiles" - Using DBCA To Create Database 12.1.0.2.0 on Grid Infrastructure / ASM 12.2.0.1.0 

Solution 1 :-

Check the compatibility of ASM & Database .

SQL> select NAME,STATE,COMPATIBILITY,DATABASE_COMPATIBILITY from v$asm_diskgroup;

NAME          STATE       COMPATIBILITY   DATABASE_COMPATIBILITY

------------- ----------- --------------- -------------------------

FRA           MOUNTED     12.1.0.0.0      10.1.0.0.0

OCR_VOTING    MOUNTED     12.1.0.0.0      10.1.0.0.0

DATA          MOUNTED     12.1.0.0.0      10.1.0.0.0

Change compatibility of database as like ASM

Here ASM having 12.1.0.0.0 & DB having 10.1.0.0.0

SQL> alter diskgroup FRA set ATTRIBUTE 'compatible.rdbms' ='12.1.0.0.0';

Diskgroup altered.

SQL> select NAME,STATE,COMPATIBILITY,DATABASE_COMPATIBILITY from v$asm_diskgroup;

NAME          STATE       COMPATIBILITY   DATABASE_COMPATIBILITY

------------- ----------- --------------- -------------------------

FRA           MOUNTED     12.1.0.0.0      12.1.0.0.0

OCR_VOTING    MOUNTED     12.1.0.0.0      10.1.0.0.0

DATA          MOUNTED     12.1.0.0.0      10.1.0.0.0

SQL> alter diskgroup DATA set ATTRIBUTE 'compatible.rdbms' ='12.1.0.0.0';

Diskgroup altered.

SQL> select NAME,STATE,COMPATIBILITY,DATABASE_COMPATIBILITY from v$asm_diskgroup;

NAME          STATE       COMPATIBILITY   DATABASE_COMPATIBILITY

------------- ----------- --------------- -------------------------

FRA           MOUNTED     12.1.0.0.0      12.1.0.0.0

OCR_VOTING    MOUNTED     12.1.0.0.0      10.1.0.0.0

DATA          MOUNTED     12.1.0.0.0      12.1.0.0.0

Solution 2 :- 

Change permission of oracle file to 6751 for both grid & databse

[ora12c@RAC1~]$ cd /rdbms/app/oracle/12.1.0.2/db/bin

[ora12c@RAC1 bin]$ ls -ltr oracle*

-rwxr-x---. 1 ora12c oinstall         0 Jul  7  2014 oracleO

-rwsr-s--x. 1 ora12c oinstall 323649840 Aug 11 09:56 oracle

[ora12c@RAC1~]$ chmod 6751 /rdbms/app/oracle/12.1.0.2/db/bin/oracle

For detail steps go to below link

How to change permission of oracle file to 6751

Thats it:)

ORA-17503: ksfdopn:2 Failed to open file +DATA/dbname/spfiledb.ora

Issue:

PRCR-1013 : Failed to start resource ora.DBname.db
PRCR-1064 : Failed to start resource ora.DBname.db on node servername
CRS-5017: The resource action "ora.DBname.db start" encountered the following error:
ORA-01078: failure in processing system parameters
ORA-01565: error in identifying file '+DATA/DBname/spfileDBname.ora'
ORA-17503: ksfdopn:2 Failed to open file +DATA/DBname/spfileDBname.ora
ORA-27140: attach to post/wait facility failed
ORA-27300: OS system dependent operation:invalid_egid failed with status: 1
ORA-27301: OS failure message: Operation not permitted
ORA-27302: failure occurred at: skgpwinit6
ORA-27303: additional information: startup egid = 501 (oinstall), current egid = 502 (dba)
. For details refer to "(:CLSN00107:)" in "/rdbms/app/oracle/diag/crs/servername/crs/trace/crsd_oraagent_ora12c.trc".

CRS-2674: Start of 'ora.DBname.db' on 'servername' failed

Solution :

Login to server by oracle user

Set DB profile (.bash_profile) and check the permission of oracle 

Ora12c:

RAC Server 1  :  -rwsr-s--x. 1 oracle oinstall 234M dec 12 02:32 oracle

RAC Server 2  :  -rwsr-s--x. 1 oracle oinstall 234M dec 12 03:45 oracle

Set DB profile (.bash_profile) and check the permission of oracle 

Grid :

Set ASM profile (.bash_profile_asm ) and check the permission of oracle 

RAC Server 1   : -rwxrwxr-x. 1 grid oinstall 239734521 dec 12 02:32 /rdbms/app/12.1.0/grid/bin/oracle

RAC Server 2   : -rwsr-s--x. 1 grid oinstall 239734521 12 03:45 /rdbms/app/12.1.0/grid/bin/oracle

Login using root user and change permission to 6751 (node which causing issue i.e node 1 in my case)

chmod 6751 /rdbms/app/12.1.0/grid/bin/oracle

-rwsr-s--x. 1 grid oinstall 239734521 12 03:45 /rdbms/app/12.1.0/grid/bin/oracle

then start DB using 

[ora12c@server1 bin]$ srvctl start instance -i instancename -d DBname

For reference:

PFB.

Server  RAC 1

Oracle

[ora12c@server1 ~]$ echo $ORACLE_HOME

/rdbms/app/12.1.0/dbhome

[ora12c@server1 ~]$ cd /rdbms/app/12.1.0/dbhome

[ora12c@server1 dbhome]$ cd bin

[ora12c@server1 bin]$ ls -ltr oracle*

-rwsr-s--x. 1 oracle oinstall 234M dec 12 02:32 oracle

Grid

[ora12c@server1 ~]$ . ./.bash_profile_asm

[ora12c@server1 ~]$ cd $ORACLE_HOME/bin

[ora12c@server1 bin]$ ls -ltr oracle*

-rwxrwxr-x. 1 grid oinstall 239734521 dec 12 02:32 oracle

[ora12c@server1 bin]$

Server  RAC 2

Oracle

[ora12c@server2 ~]$ echo $ORACLE_HOME

/rdbms/app/12.1.0/dbhome

[ora12c@server2 ~]$ cd /rdbms/app/12.1.0/dbhome

[ora12c@server2 dbhome]$ cd bin

[ora12c@server2 bin]$ ls -ltr oracle*

-rwsr-s--x. 1 oracle oinstall 234M dec 12 02:32 oracle

Grid

[ora12c@server2 ~]$ . ./.bash_profile_asm

[ora12c@server2~]$ cd $ORACLE_HOME/bin

[ora12c@server2 bin]$ ls -ltr oracle*

-rwsr-s--x. 1 grid oinstall 239734521 12 03:45 oracle

[ora12c@server1 bin]$

Login by root

[ora12c@server1 bin]$ sudo su -

Last login: Wed Aug  2 11:42:51 IST 2017 on tty1

[root@server1 ~]# chmod 6751 /rdbms/app/12.1.0.2/grid/bin/oracle

[root@server1 ~]# ls -ltr /rdbms/app/12.1.0.2/grid/bin/oracle

-rwsr-s--x. 1 ora12c oinstall 291255117 Jun 30 21:55 /rdbms/app/12.1.0.2/grid/bin/oracle

[root@server1 ~]# logout

[ora12c@server1 bin]$ srvctl start instance -i DBname1 -d DBname

[ora12c@server1 bin]$ ps -ef | grep pmon

ora12c   19636     1  0 11:29 ?        00:00:00 ora_pmon_DBname1

ora12c   19911 18507  0 11:29 pts/0    00:00:00 grep --color=auto pmon

ora12c   47857     1  0 Aug02 ?        00:00:04 asm_pmon_+ASM1

[ora12c@server1 bin]$

thats it :)

TNS-01192: Missing SID_LIST_ value left of equation for SID description in LISTENER.ORA

While starting listener getting error 

ora12c@Server1 admin]$ lsnrctl start LISTENER

LSNRCTL for Linux: Version 12.1.0.2.0 - Production on 03-AUG-2017 11:11:45

Copyright (c) 1991, 2014, Oracle.  All rights reserved.

Starting /rdbms/app/12.1.0/grid/bin/tnslsnr: please wait...

TNSLSNR for Linux: Version 12.1.0.2.0 - Production
System parameter file is /rdbms/app/12.1.0/grid/network/admin/listener.ora
Log messages written to /rdbms/app/12.1.0/grid_base/diag/tnslsnr/Server1 /listener/alert/log.xml
Listening on: (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=Server1 .vsnl.co.in)(PORT=1527)))
TNS-01192: Missing SID_LIST_ value left of equation for SID description in LISTENER.ORA

Listener failed to start. See the error message(s) above...

Solution :

If you look into log.xml file , you get to know below message

<msg time='2017-08-03T11:11:45.885+05:30' org_id='oracle' comp_id='tnslsnr'
 type='UNKNOWN' level='16' host_id='Server1 .vsnl.co.in'
 host_addr='x.x.x.x'>
 <txt>TNS-01192: Missing SID_LIST_ value left of equation for SID description in LISTENER.ORA
 </txt>
</msg>
<msg time='2017-08-03T11:11:45.885+05:30' org_id='oracle' comp_id='tnslsnr'
 type='UNKNOWN' level='16' host_id='Server1 .vsnl.co.in'
 host_addr='x.x.x.x'>
 <txt>No longer listening on: (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=Server1 .vsnl.co.in)(PORT=1527)))
 </txt>
</msg>

set asm profile and edit in listener.ora file.

VALID_NODE_CHECKING_REGISTRATION_LISTENER = SUBNET

LISTENER =
  (DESCRIPTION_LIST =
    (DESCRIPTION =
      (ADDRESS = (PROTOCOL = TCP)(HOST = Server..co.in)(PORT = 1527))
    )
  )

ENABLE_GLOBAL_DYNAMIC_ENDPOINT_LISTENER = ON

ENABLE_GLOBAL_DYNAMIC_ENDPOINT_LISTENER = ON

VALID_NODE_CHECKING_REGISTRATION_LISTENER = SUBNET

Thats it :)

Mirrored databases are disconnected after you restart the database mirror in SQL Server

Issue:- 
Mirrored databases are disconnected after restart of SQL instance due to 

• Server Hang , 
• You end the process of the SQL Server secondary instance by using Task Manager
• Network Issue 

in SQL Server 2005 and above.

Resolution 1:

In above image you can see DBA database is in Disconnected mode.
How to over come on this ???
T-sql commands.

First need to find endpoint name of mirrored server.

SELECT name,type_desc,state_desc,role_desc FROM sys.database_mirroring_endpoints;

On the principal database, execute the following SQL script to stop the endpoint:
         ALTER ENDPOINT <Endpoint Name> STATE=STOPPED
     eg: ALTER ENDPOINT Mirroring STATE=STOPPED 

Execute the following SQL script to restart the endpoint:
         ALTER ENDPOINT <Endpoint Name> STATE=STARTED
     eg: ALTER ENDPOINT Mirroring STATE=STARTED



Resolution 2 : 

Drop and re-create the database mirroring endpoints on both servers.

First need to find endpoint name of mirrored server.

SELECT name,type_desc,state_desc,role_desc FROM sys.database_mirroring_endpoints;



On the principal database, execute the following SQL script to drop the endpoint

drop endpoint <END POINTNAME>
go

On the principal database, execute the following SQL script to Create Endpoint :

CREATE ENDPOINT Mirroring
STATE = STARTED
AS TCP ( LISTENER_PORT = 5022 )
FOR DATABASE_MIRRORING 
(ENCRYPTION = DISABLED,ROLE=ALL)

GO

Use correct Encryption , else DB would be in disconnected mode 

Thats it :)

More :

➤ Steps for Configuration of Mirroring setup