agctl start goldengate hangs and fails
March 5, 2020 2 Comments
In this post I will discuss one of the reasons why golden gate startup may hang and fail on both nodes:
Problem:
[root@primrac1 tmp]# /u01/app/grid/xag/bin/agctl start goldengate gg_replicate ^[[ACRS-2672: Attempting to start 'xag.gg_replicate.goldengate' on 'primrac2' CRS-2674: Start of 'xag.gg_replicate.goldengate' on 'primrac2' failed CRS-2679: Attempting to clean 'xag.gg_replicate.goldengate' on 'primrac2' CRS-2681: Clean of 'xag.gg_replicate.goldengate' on 'primrac2' succeeded CRS-2563: Attempt to start resource 'xag.gg_replicate.goldengate' on 'primrac2' has failed. Will re-retry on 'primrac1' now. CRS-2672: Attempting to start 'xag.gg_replicate.goldengate' on 'primrac1' CRS-2674: Start of 'xag.gg_replicate.goldengate' on 'primrac1' failed CRS-2679: Attempting to clean 'xag.gg_replicate.goldengate' on 'primrac1' CRS-2681: Clean of 'xag.gg_replicate.goldengate' on 'primrac1' succeeded CRS-2632: There are no more servers to try to place resource 'xag.gg_replicate.goldengate' on that would satisfy its placement policy CRS-4000: Command Start failed, or completed with errors.
Troubleshooting:
Check crsd_scriptagent_oracle.trc
trace file on each database node. For my environment the full path of this file is /u01/app/grid/diag/crs/primrac2/crs/trace/crsd_scriptagent_oracle.trc
I found that the following error was repeating in the trace file:
2020-03-05 23:36:30.467 : CLSDADR:3560113920: ERRORdynamic component [R2530613] mapped to [CLSDYNAM]
On the metalink found only this PRCR-1079 : Failed to start resource ora.oc4j, the problem is not the same, but glanced a sentence containing config file.. so here I realized it could be related to the config file. And I was correct.
Solution:
Delete existing GG resource:
# /u01/app/grid/xag/bin/agctl remove goldengate gg_replicate
Readd using correct config file location:--config_home /GG_HOME/sm/etc/conf --var_home /GG_HOME/sm/var
# /u01/app/grid/xag/bin/agctl add goldengate gg_replicate --gg_home /GG_HOME/ma --service_manager --config_home /GG_HOME/sm/etc/conf --var_home /GG_HOME/sm/var --port 9001 --adminuser oggadmin --user oracle --group oinstall --filesystems ora.ggdg.acfsgg.acfs --db_services ora.primorcl.orclservice.svc --use_local_services
Start Golden Gate, relocate and check:
[root@primrac1 ~]# /u01/app/grid/xag/bin/agctl start goldengate gg_replicate [root@primrac1 ~]# /u01/app/grid/xag/bin/agctl status goldengate gg_replicate Goldengate instance 'gg_replicate' is running on primrac2 [root@primrac1 ~]# /u01/app/grid/xag/bin/agctl relocate goldengate gg_replicate [root@primrac1 ~]# /u01/app/grid/xag/bin/agctl status goldengate gg_replicate Goldengate instance 'gg_replicate' is running on primrac1