Golden Gate Time Since Chkpt Unknown!!

Golden Gate in Time Since Chkpt display unknown solution

An abnormal phenomenon

[oracle@localhost ~]$ ggsci

Oracle GoldenGate Command Interpreter for Oracle
Version 11.2.1.0.3 14400833 OGGCORE_11.2.1.0.3_PLATFORMS_120823.1258_FBO
Linux, x64, 64bit (optimized), Oracle 11g on Aug 23 2012 20:20:21

Copyright (C) 1995, 2012, Oracle and/or its affiliates. All rights reserved.



GGSCI (localhost.localdomain) 1> info all

Program     Status      Group       Lag           Time Since Chkpt

MANAGER     RUNNING                                          
EXTRACT     RUNNING     EXT12345     00:00:00      unknown    
EXTRACT     RUNNING     EXT67889     00:00:00      unknown    
EXTRACT     RUNNING     PUMP1234     00:00:00      unknown    
EXTRACT     RUNNING     PUMP5678     00:00:00      unknown    
REPLICAT    RUNNING     REP12345     00:00:00      unknown    


2) Tried to close the abnormal process restart

GGSCI (localhost.localdomain) 2> stop *

Sending STOP request to EXTRACT EXT12345 ...

ERROR: sending message to EXTRACT EXT12345 (Timeout waiting for message).

Sending STOP request to EXTRACT EXT67889 ...

ERROR: sending message to EXTRACT EXT67889 (Timeout waiting for message).

Sending STOP request to EXTRACT PUMP1234 ...

ERROR: sending message to EXTRACT PUMP1234 (Timeout waiting for message).

Sending STOP request to EXTRACT PUMP5678 ...

ERROR: sending message to EXTRACT PUMP5678 (Timeout waiting for message).

Sending STOP request to REPLICAT REP12345 ...

ERROR: sending message to REPLICAT REP12345 (Timeout waiting for message).

GGSCI (localhost.localdomain) 3> stop mgr!

Sending STOP request to MANAGER ...
Request processed.
Manager stopped.

GGSCI (localhost.localdomain) 4> info all

Program     Status      Group       Lag           Time Since Chkpt

MANAGER     STOPPED                                          
EXTRACT     RUNNING     EXT12345     00:00:00      unknown    
EXTRACT     RUNNING     EXT67889     00:00:00      unknown    
EXTRACT     RUNNING     PUMP1234     00:00:00      unknown    
EXTRACT     RUNNING     PUMP5678     00:00:00      unknown    
REPLICAT    RUNNING     REP12345     00:00:00      unknown    

GGSCI (localhost.localdomain) 5> kill EXT12345

ERROR: Manager not currently running.

GGSCI (localhost.localdomain) 6> kill EXT67889

ERROR: Manager not currently running.


GGSCI (localhost.localdomain) 7> info all

Program     Status      Group       Lag           Time Since Chkpt

MANAGER     STOPPED                                          
EXTRACT     RUNNING     EXT12345     00:00:00      unknown    
EXTRACT     RUNNING     EXT67889     00:00:00      unknown    
EXTRACT     RUNNING     PUMP1234     00:00:00      unknown    
EXTRACT     RUNNING     PUMP5678     00:00:00      unknown    
REPLICAT    RUNNING     REP12345     00:00:00      unknown  

GGSCI (localhost.localdomain) 8> exit
- Use stop the process, stop MGR, kill  processes are not shut down these processes

3) The system processes the system level kill related ogg
[oracle@localhost OGG]$ ps -ef|grep /opt/OGG
oracle    7479     1  0 Nov10 ?        00:03:31 /opt/OGG/extract PARAMFILE /opt/OGG/dirprm/EXT12345.prm REPORTFILE /opt/OGG/dirrpt/EXT12345.rpt PROCESSID EXT12345 USESUBDIRS
oracle    7480     1  0 Nov10 ?        00:02:30 /opt/OGG/extract PARAMFILE /opt/OGG/dirprm/EXT67889.prm REPORTFILE /opt/OGG/dirrpt/EXT67889.rpt PROCESSID EXT67889 USESUBDIRS
oracle    7483     1  0 Nov10 ?        00:00:01 /opt/OGG/extract PARAMFILE /opt/OGG/dirprm/PUMP1234.prm REPORTFILE /opt/OGG/dirrpt/PUMP1234.rpt PROCESSID PUMP1234 USESUBDIRS
oracle    7485     1  0 Nov10 ?        00:00:03 /opt/OGG/replicat PARAMFILE /opt/OGG/dirprm/REP12345.prm REPORTFILE /opt/OGG/dirrpt/REP12345.rpt PROCESSID REP12345 USESUBDIRS
oracle    7518     1  0 Nov10 ?        00:00:01 ./server -p 7847 -k -l /opt/OGG/ggserr.log
oracle    7677     1  0 Nov10 ?        00:00:15 /opt/OGG/extract PARAMFILE /opt/OGG/dirprm/PUMP5678.prm REPORTFILE /opt/OGG/dirrpt/PUMP5678.rpt PROCESSID PUMP5678 USESUBDIRS
oracle 25261 25112 0 24:48 pts / 1     0:00:00 grip  / opt / OGG
[oracle@localhost OGG]$ kill -9 7479 7480 7482 7483 7485  7518 7677
[oracle@localhost OGG]$ ps -ef|grep /opt/OGG
oracle 25264 25112 0 24:48 pts / 1     0:00:00 grip  / opt / OGG

4) Restart the process all ogg
[oracle@localhost OGG]$ ggsci

Command Interpreter Oracle GoldenGate for  Oracle
Version 11.1.1.0.0 Build 078
Linux, x64, 64bit (optimized), Oracle 10 on Jul 28 2010 13:21:11

Copyright (C) 1995, 2010, Oracle and/or its affiliates. All rights reserved.



GGSCI (localhost.localdomain) 1> info all

Program     Status      Group       Lag           Time Since Chkpt

MANAGER     STOPPED                                          
EXTRACT     ABENDED     EXT12345     00:00:00      unknown    
EXTRACT     ABENDED     EXT67889     00:00:00      unknown    
EXTRACT     ABENDED     PUMP1234     00:00:00      unknown    
EXTRACT     ABENDED     PUMP5678     00:00:00      unknown    
REPLICAT    ABENDED     REP12345     00:00:00      unknown    

- Process status or abnormal

GGSCI (localhost.localdomain) 2> start mgr

Manager started.


GGSCI (localhost.localdomain) 3> info all

Program     Status      Group       Lag           Time Since Chkpt

MANAGER     RUNNING                                          
EXTRACT     RUNNING     EXT12345     00:00:00      unknown    
EXTRACT     RUNNING     EXT67889     00:00:00      unknown    
EXTRACT     RUNNING     PUMP1234     00:00:00      unknown    
EXTRACT     RUNNING     PUMP5678     00:00:00      unknown    
REPLICAT    RUNNING     REP12345     00:00:00      unknown
   
- The process up, but Time Since Chkpt or incorrect

GGSCI (localhost.localdomain) 4> stop EXT12345

Sending STOP request to EXTRACT EXT12345 ...
Request processed.


GGSCI (localhost.localdomain) 5> info all

Program     Status      Group       Lag           Time Since Chkpt

MANAGER     RUNNING                                          
EXTRACT     STOPPED     EXT12345     unknown       00:00:02  
EXTRACT     RUNNING     EXT67889     00:00:00      unknown    
EXTRACT     RUNNING     PUMP1234     00:00:00      unknown    
EXTRACT     RUNNING     PUMP5678     00:00:00      unknown    
REPLICAT    RUNNING     REP12345     00:00:00      unknown    

- Close the EXT12345 test, the normal state

GGSCI (localhost.localdomain) 6> start EXT12345

Sending START request to MANAGER ...
EXTRACT EXT12345 starting


GGSCI (localhost.localdomain) 7> info all

Program     Status      Group       Lag           Time Since Chkpt

MANAGER     RUNNING                                          
EXTRACT     RUNNING     EXT12345     unknown       00:00:14  
EXTRACT     RUNNING     EXT67889     00:00:00      unknown    
EXTRACT     RUNNING     PUMP1234     00:00:00      unknown    
EXTRACT     RUNNING     PUMP5678     00:00:00      unknown    
REPLICAT    RUNNING     REP12345     00:00:00      unknown    

--Lag Abnormal, wait for recovery

GGSCI (localhost.localdomain) 8> stop EXT67889

Sending STOP request to EXTRACT EXT67889 ...

Recovery is not complete.  This normal stop will wait and checkpoint recovery's
work when recovery has finished. To force Extract to stop now,
use the SEND EXTRACT EXT67889, FORCESTOP command.
- Because there is no complete recovery led to the prompt, you can ignore, wait

GGSCI (localhost.localdomain) 9> info all

Program     Status      Group       Lag           Time Since Chkpt

MANAGER     RUNNING                                          
EXTRACT     RUNNING     EXT12345     unknown       00:00:02  
EXTRACT     STOPPED     EXT67889     01:51:12      00:00:01  
EXTRACT     RUNNING     PUMP1234     00:00:00      unknown    
EXTRACT     RUNNING     PUMP5678     00:00:00      unknown    
REPLICAT    RUNNING     REP12345     00:00:00      unknown    


GGSCI (localhost.localdomain) 10> start EXT67889

Sending START request to MANAGER ...
EXTRACT EXT67889 starting


GGSCI (localhost.localdomain) 11> info all

Program     Status      Group       Lag           Time Since Chkpt

MANAGER     RUNNING                                          
EXTRACT     RUNNING     EXT12345     99:53:02      00:00:01  
EXTRACT     RUNNING     EXT67889     01:51:12      00:00:10  
EXTRACT     RUNNING     PUMP1234     00:00:00      unknown    
EXTRACT     RUNNING     PUMP5678     00:00:00      unknown    
REPLICAT    RUNNING     REP12345     00:00:00      00:00:00  



GGSCI (localhost.localdomain) 15> stop PUMP1234

Sending STOP request to EXTRACT PUMP1234 ...
Request processed.


GGSCI (localhost.localdomain) 16> start PUMP1234

Sending START request to MANAGER ...
EXTRACT PUMP1234 starting


GGSCI (localhost.localdomain) 17> stop PUMP5678

Sending STOP request to EXTRACT PUMP5678 ...
Request processed.


GGSCI (localhost.localdomain) 18> start PUMP5678

Sending START request to MANAGER ...
EXTRACT PUMP5678 starting


GGSCI (localhost.localdomain) 19> info all

Program     Status      Group       Lag           Time Since Chkpt

MANAGER     RUNNING                                          
EXTRACT     RUNNING     EXT12345     00:00:00      00:00:01  
EXTRACT     RUNNING     EXT67889     00:00:00      00:00:10  
EXTRACT     RUNNING     PUMP1234     00:00:00      00:00:04  
EXTRACT     RUNNING     PUMP5678     00:00:00      00:00:05  
REPLICAT    RUNNING     REP12345     00:00:00      00:00:05  

- Restart all exceptions process, ogg work

GGSCI (localhost.localdomain) 20>

5) Summarizes the processing steps
forced off mgr, system-level kill processes related ogg, ogg open primary process, restart the relevant processes