Presentation: Oracle GoldenGate Microservices Overview (with DEMO)

Webinar: Oracle GoldenGate Microservices Overview (with DEMO)

PRVF-6402 : Core file name pattern is not same on all the nodes

Problem:

Oracle 18c GI configuration prerequisite checks failed with the following error:

PRVF-6402 : Core file name pattern is not same on all the nodes. Found core filename pattern "core" on nodes "primrac1". Found core filename pattern "core.%p" on nodes "primrac2".  
- Cause:  The core file name pattern is not same on all the nodes.  
- Action:  Ensure that the mechanism for core file naming works consistently on all the nodes. Typically for Linux, the elements to look into are the contents of two files /proc/sys/kernel/core_pattern or /proc/sys/kernel/core_uses_pid. Refer OS vendor documentation for platforms AIX, HP-UX, and Solaris.

Comparing parameter values on both nodes:

[root@primrac1 ~]# cat /proc/sys/kernel/core_uses_pid
0
[root@primrac2 ~]# cat /proc/sys/kernel/core_uses_pid
1 

[root@primrac1 ~]# sysctl -a|grep core_uses_pid
kernel.core_uses_pid = 0

[root@primrac2 ~]# sysctl -a|grep core_uses_pid
kernel.core_uses_pid = 1

Strange fact was that this parameter was not defined explicitly in sysctl.conf file, but still had different default values:

[root@primrac1 ~]# cat /etc/sysctl.conf |grep core_uses_pid
[root@primrac2 ~]# cat /etc/sysctl.conf |grep core_uses_pid 

Solution:

I’ve set parameter to 1 explicitly in sysctl.conf on both nodes:

[root@primrac1 ~]# cat /etc/sysctl.conf |grep core_uses_pid
kernel.core_uses_pid=1 

[root@primrac2 ~]# cat /etc/sysctl.conf |grep core_uses_pid
kernel.core_uses_pid=1

[root@primrac1 ~]# sysctl -p 
[root@primrac2 ~]# sysctl -p

[root@primrac1 ~]# sysctl -a|grep core_uses_pid 
kernel.core_uses_pid = 1

[root@primrac2 ~]# sysctl -a|grep core_uses_pid 
kernel.core_uses_pid = 1

Pressed Check Again button and GI configuration succeeded.

XAG Integration mode global setting of true does not match command line setting of false

Problem:

Not able to start Service Manager in Oracle Golden Gate Microservices architecture:

$ export OGG_CONF_HOME=/GG_HOME/sm/etc/conf 
$ /GG_HOME/ma/bin/ServiceManager

Errors in ServiceManager.log:

2019-03-23T21:59:50.237+0000 INFO | Read SharedContext from store for length 1372 of file '/GG_HOME/deploy/var/lib/conf/ServiceManager-config.dat'. (main)
2019-03-23T21:59:50.242+0000 INFO | Read SharedContext from store for length 245 of file '/GG_HOME/deploy/var/lib/conf/global.dat'. (main)
2019-03-23T21:59:50.245+0000 INFO | Read SharedContext from store for length 19 of file '/GG_HOME/deploy/var/lib/conf/ServiceManager-resources.dat'. (main)
2019-03-23T21:59:50.247+0000 ERROR| XAG Integration mode global setting of true does not match command line setting of false. (main)
2019-03-23T21:59:50.249+0000 ERROR| XAG Configuration failed (main)

Solution:

Pass `--xagEnabled` option when starting ServiceManager:

$ export OGG_CONF_HOME=/GG_HOME/sm/etc/conf
$ /GG_HOME/ma/bin/ServiceManager --xagEnabled

Oracle GoldenGate Service Manager for Oracle
Version 18.1.0.0.0 OGGCORE_18.1.0.0.0_PLATFORMS_180928.0432

Copyright (C) 1995, 2018, Oracle and/or its affiliates. All rights reserved.

Linux, x64, 64bit (optimized) on Sep 28 2018 17:31:51
Operating system character set identified as UTF-8.

Oracle Database 18c: why Oracle skipped 6 releases?

“If you’re confused as to why Oracle skipped 6 releases, it may be simpler to regard Oracle Database 18c  as  Oracle Database 12c Release 2 12.2.0.2, where Oracle simply changed the naming to reflect the year in which the product is released.

Oracle Database 18c is the first version of the product to follow a yearly release pattern. “

https://blogs.oracle.com/database/oracle-database-18c-:-now-available-on-the-oracle-cloud-and-oracle-engineered-systems

Oracle DB Support Roadmap 2019

GGSCI: error while loading shared libraries: libclntshcore.so.18.1: cannot open shared object file: No such file or directory

Problem:

Oracle GoldenGate Software Command Interface (GGSCI) 18c raises error:

$ /GG_HOME/home_1/ggsci
/GG_HOME/home_1/ggsci: error while loading shared libraries: libclntshcore.so.18.1: cannot open shared object file: No such file or directory

Cause: 

GoldenGate put all necessary shared objects (.so files) under its own home directory, but several shared objects may not be found there. They should be borrowed from RDBMS or GI home.

Solution:

# find / -name libclntshcore.so.18.1

/u01/app/18.3.0/grid/.patch_storage/28090523_Jul_14_2018_00_03_50/files/lib/libclntshcore.so.18.1
/u01/app/18.3.0/grid/.patch_storage/28655784_Oct_8_2018_21_27_28/files/lib/libclntshcore.so.18.1
/u01/app/18.3.0/grid/lib/libclntshcore.so.18.1
/u01/app/18.3.0/grid/inventory/Scripts/ext/lib/libclntshcore.so.18.1

Create symbolic link:

$ cd /GG_HOME/home_1
$ ln -s /u01/app/18.3.0/grid/lib/libclntshcore.so.18.1 libclntshcore.so.18.1

Rerun ggsci.

 

What is a Flex ASM and how to check if it is enabled?

In versions prior to 12c, the ASM instance needed to be run on each of the nodes of the cluster. In case ASM was not able to start, the database instance located on the same node was not able to come up also. There were a hard dependency between database and ASM instances.

With Oracle Flex ASM, databases are able to connect remote ASM using network connection(ASM network). In case of ASM instance fails, the database instance will reconnect to another ASM instance on another node. This feature is called Oracle Flex ASM.

Check if you are using such a great feature using the following command:

[grid@rac1 ~]$ asmcmd
ASMCMD> showclustermode
ASM cluster : Flex mode enabled

 

GGSCI: error while loading shared libraries: libclntsh.so.18.1: cannot open shared object file: No such file or directory

Problem:

Oracle GoldenGate Software Command Interface (GGSCI) 18c raises error:

$ /GG_HOME/home_1/ggsci
/GG_HOME/home_1/ggsci: error while loading shared libraries: libclntsh.so.18.1: cannot open shared object file: No such file or directory

Cause: 

GoldenGate put all necessary shared objects (.so files) under its own home directory, but several shared objects may not be found there. They should be borrowed from RDBMS or GI home.

Solution:

# find / -name libclntsh.so.18.1

/u01/app/18.3.0/grid/.patch_storage/28090523_Jul_14_2018_00_03_50/files/lib/libclntsh.so.18.1
/u01/app/18.3.0/grid/.patch_storage/28655784_Oct_8_2018_21_27_28/files/lib/libclntsh.so.18.1
/u01/app/18.3.0/grid/lib/libclntsh.so.18.1
/u01/app/18.3.0/grid/inventory/Scripts/ext/lib/libclntsh.so.18.1

Create symbolic link:

$ cd /GG_HOME/home_1
$ ln -s /u01/app/18.3.0/grid/lib/libclntsh.so.18.1 libclntsh.so.18.1

Rerun ggsci.

 

ORA-17635: failure in obtaining physical sector size for ‘+DATA’

Action:

I was trying to create a spfile on asm diskgroup from standby database.

SYS @ shcat > create spfile='+DATA' from pfile='/tmp/initshcat_stby.ora';
create spfile='+DATA' from pfile='/tmp/initshcat_stby.ora'
*
ERROR at line 1:
ORA-17635: failure in obtaining physical sector size for '+DATA'
ORA-12547: TNS:lost contact
ORA-12547: TNS:lost contact

Troubleshooting:

I’ve checked the sector size and it was ok:

SQL> select name,sector_size from v$asm_diskgroup;

NAME       SECTOR_SIZE
---------- ------------
DATA       512

I’ve checked if oracle account was able to see ASM diskgroups in DBCA and it was not. The diskgroup list was empty.

Causes:

There are several causes:

1) File permissions in <Grid_home>/bin/oracle executable not set properly.

2) Oracle user is not a part of asmdba group

Solution: 

1)  Change permissions:

[root@stbycat ~]# chmod 6751 /u01/app/18.3.0/grid/bin/oracle

2)  Add oracle to asmdba group

[root@stbycat ~]# usermod -g oinstall -G oper,dba,asmdba oracle

In my case it was 1st.

My permissions:

$ ll /u01/app/18.3.0/grid/bin/oracle
-rwxr-x--x 1 grid oinstall 413844056 Nov 4 09:14 /u01/app/18.3.0/grid/bin/oracle

Must be:

$ ll /u01/app/18.3.0/grid/bin/oracle
-rwsr-s--x 1 grid oinstall 413844056 Nov 4 08:45 /u01/app/18.3.0/grid/bin/oracle

 

ORA-01103: when creating a Standby Database on the same Host as the Primary Database

Typically the standby and the primary databases are located on the different hosts to ensure the full DR capabilities. However, there are some situations when you want to have the primary and standby database on the same Host.

Problem #1: You are not able to start two databases with the same SID on the same server.

Problem #2: You cannot change db_name, because it is used in the controlfile and if you try to duplicate the standby database from the primary using different db_name, you will get the following error:

ORA-01103: database name 'orcldgst' in control file is not 'orcldg'

Assume db_name=orcldg and ORACLE_SID for the primary is orcldg1. To solve problem #1 and problem #2, you need to the following steps:

db_name must be the same for both databases. But during startup nomount of the standby database, you need to set ORACLE_SID to the different value:

$ export ORACLE_SID=orcldgst1
$ sqlplus / as sysdba
SQL> startup nomount pfile='/tmp/mypfile.ora'

After that you will be able to run RMAN duplicate command to create the standby database on the same host as the primary.