Wednesday, August 10, 2016

ORA-01092 & ORA-39701 during Startup upgrade


SQL> startup upgrade
ORACLE instance started.

Total System Global Area 4.0486E+10 bytes
Fixed Size                  2261968 bytes
Variable Size            6845107248 bytes
Database Buffers         3.3554E+10 bytes
Redo Buffers               84606976 bytes
Database mounted.
ORA-01092: ORACLE instance terminated. Disconnection forced
ORA-39701: database must be mounted EXCLUSIVE for UPGRADE or DOWNGRADE
Process ID: 122696
Session ID: 267 Serial number: 3

Issue : The parameter cluster_database was set to true. We need to change it to false to start the RAC database in upgrade database

SQL> startup
ORACLE instance started.

Total System Global Area 4.0486E+10 bytes
Fixed Size                  2261968 bytes
Variable Size            6845107248 bytes
Database Buffers         3.3554E+10 bytes
Redo Buffers               84606976 bytes
Database mounted.
Database opened.
SQL> alter system set cluster_database=false scope=spfile sid='*';

System altered.

SQL> show parameter cluster

NAME                                 TYPE        VALUE
------------------------------------ ----------- ------------------------------
cluster_database                     boolean     TRUE
cluster_database_instances           integer     2
cluster_interconnects                string

SQL> shutdown immediate
Database closed.
Database dismounted.
ORACLE instance shut down.

SQL> startup upgrade 
ORACLE instance started.

Total System Global Area 4.0486E+10 bytes
Fixed Size                  2261968 bytes
Variable Size            6845107248 bytes
Database Buffers         3.3554E+10 bytes
Redo Buffers               84606976 bytes
Database mounted.
Database opened.

Tuesday, May 17, 2016

Delete two hours old files in Sun OS


In Sun OS mmin doesnt work to delete the older files..

To delete older files on SunOS use "newer" option

oracle@sev2(1125) db2 /opt/app/db2/oracle/admin/db2/udump
$ date
Tue May 17 18:33:22 EDT 2016
oracle@sev2(1126) db2 /opt/app/db2/oracle/admin/db2/udump
$ touch 05171633 /tmp/TIMESTAMP 
[05=month 17 = date 1633 = 16:33 pm ( 2 hours older than current time)]
oracle@sev2(1128) db2 /opt/app/db2/oracle/admin/db2/udump
$ nohup find . ! -newer /tmp/TIMESTAMP -exec rm {} \;

Tuesday, May 10, 2016

Linux : how to mail all files under a directory

the below command can help sending all the files under a directory

cd /location/where/files/are/present/
for file in .* *; do mailx -s "${file}" -a "${file}" <email@id> < ${file}; done

FYI : each file will be sent as separate email.

Monday, March 28, 2016

Oracle : Recreate AWR for 11gR2 databases

Steps to recreate awr

For complete steps please refer to : How to Recreate The AWR ( AUTOMATIC WORKLOAD ) Repository ? (Doc ID 782974.1)

=================================================
save a copy of spfile
=================================================
db1@SEV3(!) db1 /opt/app/db1
$ cd $OH/dbs
db1@SEV3(!) db1 /opt/app/db1/oracle/product/11.2.0.4/dbs
$ sql
SQL*Plus: Release 11.2.0.4.0 Production on Mon Mar 28 12:11:57 2016
Copyright (c) 1982, 2013, Oracle.  All rights reserved.
Connected to:
Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production
With the Partitioning, OLAP, Data Mining and Real Application Testing options
SQL> create pfile='spfile_beforeawr.txt' from spfile;
File created.

=================================================
In 10g and 11g , if sga_target is not 0, then in pfile or spfile set the following parameters:
=================================================
The example below refers to spfile:
alter system set shared_pool_size = 5000m scope = spfile;
// Oracle recommended 200M, but my database didnt start and failed with error ORA-00093 .. so changed the size to 5000m
alter system set db_cache_size = 300m scope = spfile;
alter system set java_pool_size = 100 scope = spfile;
alter system set large_pool_size = 50 scope = spfile;
alter system set sga_target=0 scope= spfile;
alter system set memory_target=0 scope= spfile;
alter system reset memory_max_target scope=spfile;
####alter system set statistics_level=basic scope=spfile; // Do not run this, I got below error if we change the statistics_level=basic
##SQL> startup restrict
##ORA-00824: cannot set SGA_TARGET or MEMORY_TARGET due to existing internal settings
##ORA-00848: STATISTICS_LEVEL cannot be set to BASIC with SGA_TARGET or MEMORY_TARGET

=================================================
Setting the parameter cluster_database only applicable in RAC environment Check actual shared pool and buffer cache usage in AWR to make sure the settings are correct
=================================================
alter system set cluster_database = false scope = spfile;

=================================================
Shutdown database and startup in restrict mode so that no transactions will occur while dropping the AWR repository:
=================================================
sqlplus /nolog
connect / as sysdba
shutdown immediate
startup restrict

=================================================
Drop and recreate the AWR objects The following scripts drop AWR tables and then recreates them. 
After recreating ,utlrp is run in order to validate all views and objects dependent on the AWR tables. On both 10g, 11g, and 12c drop AWR
=================================================
start ?/rdbms/admin/catnoawr.sql
alter system flush shared_pool;

================================================
Check to see if all the objects are dropped :
================================================
SQL> select table_name from dba_tables where table_name like 'WRM$_%' or table_name like 'WRH$_%';

>>>> If there are objects after running catnoawr.sql, drop them manually:
drop type AWR_OBJECT_INFO_TABLE_TYPE;
drop type AWR_OBJECT_INFO_TYPE;
drop table WRH$_PLAN_OPERATION_NA ME;
drop table WRH$_PLAN_OPTION_NAME;
drop table WRH$_MV_PARAMETER;
drop table WRH$_MV_PARAMETER_BL;
drop table WRH$_DYN_REMASTER_STATS;
drop table WRH$_PERSISTENT_QMN_CACHE;
drop table WRH$_DISPATCHER;
drop table WRH$_SHARED_SERVE R_SUMMARY;
drop table WRM$_WR_USAGE;
drop table WRM$_SNAPSHOT_DETAILS;

===============================================
Now create AWR:
===============================================
start ?/rdbms/admin/catawrtb.sql
start ?/rdbms/admin/utlrp.sql
start ?/rdbms/admin/execsvrm.sql

=================================================
Note: If you receive the following errors when executing "?/rdbms/admin/execsvrm.sql", as follows:
start ?/rdbms/admin/execsvrm.sql
Fails with the following errors :
ERROR at line 1:
ORA04068:
existing state of packages has been discarded
ORA04061:
existing state of package body "SYS.DBMS_SWRF_ INTERNAL" has been
invalidated
then recompile the object(s).
alter package dbms_swrf_internal compile;
alter package dbms_swrf_internal compile body;
It is important to do this even if the object(s) (dbms_swrf_internal in this case) appear valid. You will
then need to reexecute
the "?/rdbms/admin/execsvrm.sql" script.
=================================================

=================================================
Create spfile from pfile we created before
=================================================
SQL> create spfile from pfile='spfile_beforeawr.txt';

File created.

=================================================
Restart database in normal mode
=================================================
shutdown immediate
startup

=================================================
Check invalid objects exists are not , if exists then please compile it manually. As we have run utlrp.sql, any
invalid objects should already have been reported there:
=================================================
spool objects.lst
set pagesize500
set linesize 100
select substr(comp_name,1,40) comp_name, status, substr(version,1,10) version
from dba_registry
order by comp_name ;
select substr(object_name,1,40) object_name,substr(owner,1,15) owner,object_type
from dba_objects
where status='INVALID' order by owner,object_type;
select owner,object_type,count(*)
from dba_objects
where status='INVALID'
group by owner,object_type order by owner,object_type ;
spool off
alter package <schema name>.<package_name> compile;
alter package <schema name>.<package_name> compile b ody;
alter view <schema name>.<view_name> compile;
alter trigger <schema).<trigger_name> compile ;

=================================================
To take the AWR snapshots:
=================================================
exec dbms_workload_repository.create_snapshot;
wait for 5 min
exec dbms_workload_repository.create_snapshot;

=================================================
To create AWR report run the script:
=================================================
start $ORACLE_HOME/rdbms/admin/awrrpt.sql

Thursday, February 25, 2016

Oracle : Prerequisite check "CheckActiveFilesAndExecutables" failed ; OPatch failed with error code 73

While patching the oracle database, the patch apply failed with below error

Following executables are active :
/opt/app/oracle_db/oracle/product/oracle_db/lib/libclntsh.so.11.1
UtilSession failed: Prerequisite check "CheckActiveFilesAndExecutables" failed.
Log file location: /opt/app/oracle_db/oracle/product/oracle_db/cfgtoollogs/opatch/opatch2016-02-25_11-13-41AM_1.log

OPatch failed with error code 73

Cause :
Something is using the library , we need to find what is using it and kill it.

find the culprit using fuser

Solution :

oracle_db@sev2(174) oracle_db /opt/app/patches
$ fuser /opt/app/oracle_db/oracle/product/oracle_db/lib/libclntsh.so.11.1
/opt/app/oracle_db/oracle/product/oracle_db/lib/libclntsh.so.11.1: 118942m
oracle_db@sev2(175) oracle_db /opt/app/patches
$ ps -ef | grep 118942
oracle_db  95662  57918  0 11:15 pts/0    00:00:00 grep 118942
oracle_db 118942 118378  0 Jan21 ?        00:00:00 sqlplus -s              @/opt/app/oracle_db/backup/rman/common/info.sql  oracle_db
oracle_db@sev2(176) oracle_db /opt/app/patches
$ kill -9 118942

And then start the patch apply. Hope this resolves your problem



Monday, February 1, 2016

Oracle : How to add comments to Alert Log

To add comments to Alert log of Oracle Database, you can run the below command

SQL> execute sys.dbms_system.ksdwrt(2,'Hi, Hello World');

PL/SQL procedure successfully completed.

Note that the first parameter MUST be a 2.


Monday, November 16, 2015

ERROR OGG-01028 Formatting error on: table name + Error converting timestamp with timezone from Oracle to ASCII format for column

2015-11-16 17:51:18  INFO    OGG-00993  Oracle GoldenGate Capture for Oracle, EXTRACT1.prm:  EXTRACT EXTRACT1 started.
2015-11-16 17:51:18  INFO    OGG-01055  Oracle GoldenGate Capture for Oracle, EXTRACT1.prm:  Recovery initialization completed for target file /usr/app/datab1/ggs/trails/a1000007, at RBA 1144.
2015-11-16 17:51:18  INFO    OGG-01478  Oracle GoldenGate Capture for Oracle, EXTRACT1.prm:  Output file /usr/app/datab1/ggs/trails/a1 is using format RELEASE 11.2.
2015-11-16 17:51:18  INFO    OGG-01026  Oracle GoldenGate Capture for Oracle, EXTRACT1.prm:  Rolling over remote file /usr/app/datab1/ggs/trails/a1000007.
2015-11-16 17:51:18  INFO    OGG-01053  Oracle GoldenGate Capture for Oracle, EXTRACT1.prm:  Recovery completed for target file /usr/app/datab1/ggs/trails/a1000008, at RBA 1144.
2015-11-16 17:51:18  INFO    OGG-01057  Oracle GoldenGate Capture for Oracle, EXTRACT1.prm:  Recovery completed for all targets.
2015-11-16 17:51:18  INFO    OGG-01517  Oracle GoldenGate Capture for Oracle, EXTRACT1.prm:  Position of first record processed Sequence 17507, RBA 21008, SCN 3332.2582756465, Nov 16, 2015 5:32:16 PM.
2015-11-16 17:51:19  INFO    OGG-01020  Oracle GoldenGate Delivery for Oracle, rccp004a.prm:  Processed extract process RESTART_ABEND record at seq 8, rba 1081 (aborted 0 records).
2015-11-16 17:51:21  ERROR   OGG-01028  Oracle GoldenGate Capture for Oracle, EXTRACT1.prm:  Formatting error on: table name CSI_CAET.ERROR_HIT, rowid AAAYnTAAGAAACM/AAA, XID 1.1.2642, position (Seqno 17507, RBA 21616). Error converting timestamp with timezone from Oracle to ASCII format for column LAST_UPDATE_DATE.
2015-11-16 17:51:21  ERROR   OGG-01668  Oracle GoldenGate Capture for Oracle, EXTRACT1.prm:  PROCESS ABENDING.


GGSCI (Sever2) 1> info all

Program     Status      Group       Lag at Chkpt  Time Since Chkpt

MANAGER     RUNNING                                           
EXTRACT     ABENDED     EXTRACT1    00:14:10      00:58:53    


GGSCI (Sever2) 2> edit params EXTRACT1

Solution :
Oracle reported bug 14053498 for the above error {GoldenGate Extract Abends: OGG-01028 Formatting Error on: table name xxxx.xxxx rowid , XID nnnn.nn.nnnn, Position (Seqno n, RBA n) (Doc ID 1558791.1)}

Another Solution

Check the database characterset

SQL> select * from nls_database_parameters;

PARAMETER                      VALUE
------------------------------ ----------------------------------------
NLS_NCHAR_CHARACTERSET         AL16UTF16
NLS_LANGUAGE                   AMERICAN
NLS_TERRITORY                  AMERICA
NLS_CURRENCY                   $
NLS_ISO_CURRENCY               AMERICA
NLS_NUMERIC_CHARACTERS         .,
NLS_CHARACTERSET               WE8ISO8859P1
NLS_CALENDAR                   GREGORIAN
NLS_DATE_FORMAT                DD-MON-RR
NLS_DATE_LANGUAGE              AMERICAN
NLS_SORT                       BINARY


Add below two parameters to the extract prm file

SETENV (NLS_LANG="AMERICAN_AMERICA.WE8ISO8859P1") -- Please make sure you have the correct characterset from the above output
TRANLOGOPTIONS INCLUDEREGIONID

GGSCI (Sever2) 4> start EXTRACT1

Sending START request to MANAGER ...
EXTRACT EXTRACT1 starting


GGSCI (Sever2) 5> info EXTRACT1

EXTRACT    EXTRACT1  Last Started 2015-11-16 18:50   Status RUNNING
Checkpoint Lag       01:13:36 (updated 00:00:01 ago)
Log Read Checkpoint  Oracle Redo Logs
                     2015-11-16 17:37:08  Seqno 17507, RBA 176876032
                     SCN 3332.2582759675 (14313413789947)