Transaction recovery: lock conflict caught and ignored

ALERT.LOG:

..... Transaction recovery: lock conflict caught and ignored
.....

And also some incident files are being created in $ORACLE_BASE/diag/rdbms/dbname/instancename/incident folder.

In my case the error started after SUPPLEMENTAL LOGGING enabled in a RAC environment. After disabling it the messages have not disappeared, but incident files are no longer being created.

1. Dead Trasaction

SQL> select b.name useg, b.inst# instid, b.status$ status, a.ktuxeusn

xid_usn, a.ktuxeslt xid_slot, a.ktuxesqn xid_seq, a.ktuxesiz undoblocks,

a.ktuxesta txstatus

from x$ktuxe a, undo$ b

where a.ktuxecfl like ‘%DEAD%’

and a.ktuxeusn = b.us#;

USEG INSTID STATUS XID_USN XID_SLOT XID_SEQ UNDOBLOCKS TXSTATUS
_SYSSMU7_881277423$ 1 3 7 13 1829999 1 ACTIVE
_SYSSMU8_4204495590$ 1 3 8 32 3045564 1 ACTIVE
_SYSSMU10_1314081219$ 1 3 10 3 11844457 1 ACTIVE

Transaction id is  XID_USN.XID_SLOT.XID_SEQ

So in our case, for the first row it will be 7.13.1829999

2.  Read transaction table from undo header.

ALTER SYSTEM DUMP UNDO HEADER ‘_SYSSMU7_881277423$’;

….

  TRN TBL::

 
  index  state cflags  wrap#    uel         scn            dba            parent-xid    nub     stmt_num    cmt

  ————————————————————————————————

   0x00    9    0x03  0x1bf45c  0x000b  0x0000.789de808  0x00c242eb  0x0000.000.00000000  0x00000001   0x00c242eb  1367258143

   0x01    9    0x00  0x1c031b  0x0014  0x0000.789e6018  0x00c242fa  0x0000.000.00000000  0x00000001   0x00000000  1367258225

   0x02    9    0x00  0x1c147a  0x000e  0x0000.789e694b  0x00c242fa  0x0000.000.00000000  0x00000001   0x00000000  1367258230

   0x03    9    0x00  0x1c06f9  0x0016  0x0000.789e601c  0x00c242fa  0x0000.000.00000000  0x00000001   0x00000000  1367258225

   0x04    9    0x00  0x1c06c8  0x0009  0x0000.789e3566  0x00c242f9  0x0000.000.00000000  0x00000001   0x00000000  1367258192

   0x05    9    0x00  0x1c1167  0x0015  0x0000.789e357f  0x00c242ec  0x0000.000.00000000  0x00000001   0x00000000  1367258192

   0x06    9    0x00  0x1c2716  0x0017  0x0000.789e69e1  0x00c242fa  0x0000.000.00000000  0x00000001   0x00000000  1367258230

   0x07    9    0x00  0x1c1045  0x000c  0x0000.789e1bdb  0x00c242eb  0x0000.000.00000000  0x00000001   0x00000000  1367258170

   0x08    9    0x00  0x1c2614  0x0005  0x0000.789e357e  0x00c242ec  0x0000.000.00000000  0x00000001   0x00000000  1367258192

   0x09    9    0x00  0x1bfa03  0x0021  0x0000.789e3574  0x00c242f9  0x0000.000.00000000  0x00000001   0x00000000  1367258192

   0x0a    9    0x00  0x1bf712  0x001e  0x0000.789e3246  0x00c242f1  0x0000.000.00000000  0x00000001   0x00000000  1367258190

   0x0b    9    0x00  0x1c1e01  0x0007  0x0000.789e1bd9  0x00c242eb  0x0000.000.00000000  0x00000001   0x00000000  1367258170

   0x0c    9    0x00  0x1c08e0  0x000a  0x0000.789e3244  0x00c242f1  0x0000.000.00000000  0x00000006   0x00000000  1367258190

   0x0d   10    0x90  0x1bec6f  0x0038  0x0000.789e783e  0x00c242fb  0x0000.000.00000000  0x00000001   0x00c242fb  0

   0x0e    9    0x00  0x1c068e  0x0010  0x0000.789e694d  0x00c242fa  0x0000.000.00000000  0x00000001   0x00000000  1367258230

   0x0f    9    0x00  0x1c151d  0x0012  0x0000.789e3578  0x00c242ec  0x0000.000.00000000  0x00000001   0x00000000  1367258192

   0x10    9    0x00  0x1c26bc  0x0006  0x0000.789e69df  0x00c242fa  0x0000.000.00000000  0x00000001   0x00000000  1367258230

   0x11    9    0x00  0x1c16eb  0x0000  0x0000.789cbd77  0x00c242eb  0x0000.000.00000000  0x00000001   0x00000000  1367257923

   0x12    9    0x00  0x1c082a  0x001d  0x0000.789e357c  0x00c242ec  0x0000.000.00000000  0x00000001   0x00000000  1367258192

   0x13    9    0x00  0x1c1459  0x001f  0x0000.789e7891  0x00c242fc  0x0000.000.00000000  0x00000001   0x00000000  1367258238

   0x14    9    0x00  0x1c14b8  0x0003  0x0000.789e601a  0x00c242fa  0x0000.000.00000000  0x00000001   0x00000000  1367258225

   0x15    9    0x00  0x1c0457  0x0020  0x0000.789e39d3  0x00c242ec  0x0000.000.00000000  0x00000001   0x00000000  1367258195

   0x16    9    0x00  0x1c1326  0x0002  0x0000.789e601d  0x00c242fa  0x0000.000.00000000  0x00000001   0x00000000  1367258225

   0x17    9    0x00  0x1c0db5  0x001c  0x0000.789e788a  0x00c242fc  0x0000.000.00000000  0x00000001   0x00000000  1367258238

   0x18    9    0x00  0x1bffe4  0x001b  0x0000.789e400d  0x00c242fa  0x0000.000.00000000  0x00000001   0x00000000  1367258200

   0x19    9    0x00  0x1c16e3  0x0001  0x0000.789e5fd2  0x00c242fa  0x0000.000.00000000  0x00000001   0x00000000  1367258225

   0x1a    9    0x00  0x1bdbb2  0x0018  0x0000.789e400b  0x00c242fa  0x0000.000.00000000  0x00000001   0x00000000  1367258200

   0x1b    9    0x00  0x1c1141  0x0019  0x0000.789e453a  0x00c242fa  0x0000.000.00000000  0x00000001   0x00000000  1367258204

   0x1c    9    0x00  0x1bc9a0  0x0013  0x0000.789e788e  0x00c242fc  0x0000.000.00000000  0x00000001   0x00000000  1367258238

   0x1d    9    0x00  0x1c02ef  0x0008  0x0000.789e357d  0x00c242ec  0x0000.000.00000000  0x00000001   0x00000000  1367258192

   0x1e    9    0x00  0x1c0b6e  0x0004  0x0000.789e3250  0x00c242f9  0x0000.000.00000000  0x00000009   0x00000000  1367258190

   0x1f    9    0x00  0x1c00ad  0xffff  0x0000.789e78a1  0x00c242fc  0x0000.000.00000000  0x00000001   0x00000000  1367258238

   0x20    9    0x00  0x1c166c  0x001a  0x0000.789e39dd  0x00c242fa  0x0000.000.00000000  0x00000002   0x00000000  1367258195

   0x21    9    0x00  0x1c160b  0x000f  0x0000.789e3576  0x00c242ec  0x0000.000.00000000  0x00000001   0x00000000  1367258192

  EXT TRN CTL::

  usn: 7

 
State# 10 means active transaction.

dba points to starting UNDO block address.

usn: Undo segment number

usn.index.wrap# gives transaction id.

 
An active transaction 0x0007.00d.001bec6f is available in slot 0x0d which has a dba of 0x00c242fb (12731131 in decimal)

3. Reading UNDO Block:

Identify fileID and blockID:

fileID:

select DBMS_UTILITY.DATA_BLOCK_ADDRESS_FILE(12731131) from x$dual;

3

blockID:

select DBMS_UTILITY.DATA_BLOCK_ADDRESS_BLOCK(12731131) from x$dual;

148219

Dumping block

alter system dump datafile 3 block 148219;


UNDO BLK: 
xid: 0x0007.00d.001bec6f  seq: 0x41f9 cnt: 0x6   irb: 0x5   icl: 0x0   flg: 0x0000

 
Rec Offset      Rec Offset      Rec Offset      Rec Offset      Rec Offset

—————————————————————————

0x01 0x1f98     0x02 0x1f2c     0x03 0x1d7c     0x04 0x1d10     0x05 0x1ca0    
0x06 0x1bfc    
 
*—————————–

* Rec #0x1  slt: 0x0d  objn: 0(0x00000000)  objd: 0  tblspc: 0(0x00000000)

*       Layer:   5 (Transaction Undo)   opc: 7  rci 0x00  
Undo type:  Regular undo    Begin trans    Last buffer split:  No

Temp Object:  No

Tablespace Undo:  No

rdba: 0x00000000Ext idx: 0

flg2: 0

*—————————–

uba: 0x00c242fa.41f9.37 ctl max scn: 0x0000.789b7668 prv tx scn: 0x0000.789bb8d7

txn start scn: scn: 0x0000.789e783e logon user: 88

prev brb: 12731116 prev bcl: 0

 
*—————————–

* Rec #0x2  slt: 0x0d  objn: 110769(0x0001b0b1)  objd: 110769  tblspc: 6(0x00000006)

*       Layer:  11 (Row)   opc: 1  rci 0x00  
Undo type:  Regular undo    User Undo Applied  Last buffer split:  No

Temp Object:  No

Tablespace Undo:  No

rdba: 0x00000000

*—————————–

KDO undo record:

KTB Redo

op: 0x04  ver: 0x01 
compat bit: 4 (post-11) padding: 1

op: L  itl: xid:  0x0012.01c.00322281 uba: 0x0102c5f0.3fa9.0a

                      flg: C—    lkc:  0     scn: 0x0000.789ca3f4

KDO Op code: LKR row dependencies Disabled

  xtype: XA flags: 0x00000000  bdba: 0x038180fc  hdba: 0x018d64e2

itli: 1  ispac: 0  maxfr: 4858

tabn: 0 slot: 14 to: 0

 
*—————————–

* Rec #0x3  slt: 0x0d  objn: 110769(0x0001b0b1)  objd: 110769  tblspc: 6(0x00000006)

*       Layer:  11 (Row)   opc: 1   rci 0x02  
Undo type:  Regular undo    User Undo Applied  Last buffer split:  No

Temp Object:  No

Tablespace Undo:  No

rdba: 0x00000000

*—————————–

KDO undo record:

KTB Redo

op: 0x02  ver: 0x01 
compat bit: 4 (post-11) padding: 1

op: C  uba: 0x00c242fb.41f9.02

KDO Op code: URP row dependencies Disabled

  xtype: XA flags: 0x00000000  bdba: 0x038180fc  hdba: 0x018d64e2

itli: 1  ispac: 0  maxfr: 4858

tabn: 0 slot: 14(0xe) flag: 0x2c lock: 1 ckix: 0

ncol: 9 nnew: 6 size: 0

col  1: [ 7]  78 71 04 1d 13 01 01

col  2: [ 2]  c1 13

col  3: [ 1]  80

col  4: [16]  10 e5 00 2e 10 d1 10 d0 10 d7 10 e3 10 db 10 d8

col  5: [174]

10 d0 10 ed 10 d0 10 e0 10 d8 10 e1 00 20 10 d0 00 2e 10 e0 00 2e 00 20 10

de 10 e0 10 dd 10 d9 10 e3 10 e0 10 d0 10 e2 10 e3 10 e0 10 d8 10 e1 00 20

10 e1 10 d0 10 d2 10 d0 10 db 10 dd 10 eb 10 d8 10 d4 10 d1 10 dd 00 20 10

dc 10 d0 10 ec 10 d8 10 da 10 d8 10 e1 00 20 10 e3 10 e4 10 e0 10 dd 10 e1

00 20 10 d2 10 d0 10 db 10 dd 10 db 10 eb 10 d8 10 d4 10 d1 10 d4 10 da 10

e1 00 20 10 d1 10 d0 10 e2 10 dd 10 dc 00 20 10 d2 10 d8 10 dd 10 e0 10 d2

10 d8 00 20 10 de 10 d4 10 e0 10 d0 10 dc 10 d8 10 eb 10 d4 10 e1 00 2e

col  6: [36]

00 54 00 01 04 0c 00 00 00 02 00 00 00 01 00 00 09 07 b0 63 00 10 09 00 00

00 00 00 00 00 00 00 00 00 00 00

 
*—————————–

* Rec #0x4  slt: 0x0d  objn: 89834(0x00015eea)  objd: 93214  tblspc: 6(0x00000006)

*       Layer:  11 (Row)   opc: 1   rci 0x03  
Undo type:  Regular undo    User Undo Applied  Last buffer split:  No

Temp Object:  No

Tablespace Undo:  No

rdba: 0x00000000

*—————————–

KDO undo record:

KTB Redo

op: 0x04  ver: 0x01 
compat bit: 4 (post-11) padding: 1

op: L  itl: xid:  0x000c.017.000d65d6 uba: 0x0103df2c.22a5.20

                      flg: C—    lkc:  0     scn: 0x0000.789c4694

KDO Op code: LKR row dependencies Disabled

  xtype: XA flags: 0x00000000  bdba: 0x03833994  hdba: 0x0181f832

itli: 1  ispac: 0  maxfr: 4858

tabn: 0 slot: 7 to: 0

 
*—————————–

* Rec #0x5  slt: 0x0d  objn: 89834(0x00015eea)  objd: 93214  tblspc: 6(0x00000006)

*       Layer:  11 (Row)   opc: 1  rci 0x04  
Undo type:  Regular undo   Last buffer split:  No

Temp Object:  No

Tablespace Undo:  No

rdba: 0x00000000

*—————————–

KDO undo record:

KTB Redo

op: 0x02  ver: 0x01 
compat bit: 4 (post-11) padding: 1

op: C  uba: 0x00c242fb.41f9.04

KDO Op code: LMN row dependencies Disabled

  xtype: XA flags: 0x00000000  bdba: 0x03833994  hdba: 0x0181f832

itli: 1  ispac: 0  maxfr: 4858

 
*—————————–

* Rec #0x6  slt: 0x0d  objn: 89703(0x00015e67)  objd: 92020  tblspc: 6(0x00000006)

*       Layer:  11 (Row)   opc: 1  rci 0x05  
Undo type:  Regular undo    User Undo Applied  Last buffer split:  No

Temp Object:  No

Tablespace Undo:  No

rdba: 0x00000000

*—————————–

KDO undo record:

irb points to last UNDO RECORD in UNDO block.

rci points to previous UNDO RECORD. if rci=0, it’s the first UNDO RECORD.

Recovery operation starts from irb and chain is followed by rci until rci is zero.

The transaction starts recovery from UNDO RECORD of 0x5.

4. Reading UNDO Records:


* Rec #0x5  slt: 0x0d  objn: 89834(0x00015eea)  objd: 93214  tblspc: 6(0x00000006)

*       Layer:  11 (Row)   opc: 1   rci 0x04  
….

* Rec #0x4  slt: 0x0d  objn: 89834(0x00015eea)  objd: 93214  tblspc: 6(0x00000006)

*       Layer:  11 (Row)   opc: 1   rci 0x03  
….

* Rec #0x3  slt: 0x0d objn: 110769(0x0001b0b1)  objd: 110769  tblspc: 6(0x00000006)

*       Layer:  11 (Row)   opc: 1   rci 0x02  

* Rec #0x2  slt: 0x0d  objn: 110769(0x0001b0b1)  objd: 110769  tblspc: 6(0x00000006)

*       Layer:  11 (Row)   opc: 1   rci 0x00  

objn means object id.

5. Find these objects

The following objects need recovery:

select * from dba_objects

where object_id in (89834,110769);

………………………………………………………..

This problem is Oracle Bug:9857702:

.....
Affects:
Product (Component) Oracle Server (Rdbms)  
Range of versions believed to be affected Versions >= 11.1 but BELOW 12.1  
Versions confirmed as being affected
•11.2.0.1 
•11.1.0.7 
 
Platforms affected Generic (all / most platforms affected)  

Fixed:
This issue is fixed in
•12.1 (Future Release) 
•11.2.0.2 (Server Patch Set) 
•11.1.0.7.8 Patch Set Update 
•11.1.0.7 Patch 40 on Windows Platforms  
.....

6. Workaround:

  • Recreate objects that need recovery.
  • Or drop them Smile

Automatic Tablespace Point-in-Time Recovery(TSPIR) in Oracle 11gR2

Automatic tablespace point-in-time recovery is very useful in the following situations:

1.  You want to recover just one tablespace to the point in time other than rest of the tablespaces.

2. You want to recover dropped tablespace.

3. You want to recover data in table that was lost after changing structure of the table. Note that you are not able to use Flashback Table feature to recover data that was lost after changing table structure, it is not permitted.

TSPITR has a number of restrictions, a quick summary of which follows:

1.  You can’t restore tablespaces with objects owned by SYS. Any tablespace with replicated master tables cannot be recovered with TSPITR.

2. Tablespaces with snapshot logs are not supported.

3. You can’t restore tablespaces that contain rollback segments.

4. If an object within the tablespace to be recovered has one of the following types, then TSPITR is not supported:

    1. VARRAY
    2. Nested tables 
    3. External files

5. The current physical structure of the undo and rollback segments in the target database
must be unchanged between the point you wish to recover to and the current point.

6. Once you have completed TSPITR on a given tablespace, all previous backups of that tablespace are no longer usable for future TSPITR recoveries of that tablespace. So you should backup your tablespace after TSPITR in case you need to run another TSPITR.

So, let’s start automatic TSPITR…Note that we are not using recovery catalog.

1. Identify the time to which you want to recover your tablespace. Don’t misidentify the time or you will not be able to retry the recovery.(As we mentioned in the section 6 of TSPITR restrictions, all previous backups will no longer be usable for future TSPITR)

2. Make sure that objects in that tablespace are self-contained.

Let’s say we have the tablesapace called “TEST”.  It contains table “TEST_TABLE” but indexes on that table resides on another tablespace called “TEST_IDX”. In this case, if you want to recover “TEST” tablespace to some point-in-time , then you must also recover “TEST_IDX” tablespace to the same point-in-time(or you will loose indexes).

To identify dependent tablesapaces, run the following:

SQL> select obj1_owner
       ,obj1_name
       ,obj1_type
       ,ts2_name
       ,reason
from sys.ts_pitr_check
where ( ts1_name in (‘TEST’)
and ts2_name not in (‘TEST’) )
or ( ts1_name not in (‘TEST’)
and ts2_name in (‘TEST’) );

OBJ1_OWNER OBJ1_NAME  OBJ1_TYPE TS2_NAME REASON
TEST_USR       TEST_TABLE TABLE           TEST_IDX Tables and associated indexes not fully contained in the recovery set

This would return no rows if there were no dependencies. If there were any, you would see a
row describing each of them, as shown above.

In our case, there is an index associated with TEST_TABLE table and it resides in TEST_IDX tablespace. We should restore it also.

3. Identify what objects will be dropped after recovering tablespace.

Run the following,

SQL> select *
from ts_pitr_objects_to_be_dropped
where tablespace_name=’TEST’

OWNER NAME CREATION_TIME TABLESPACE_NAME
TEST_USR TEST_TABLE 4/19/2013 12:19:18 PM TEST

For example, if you are going to recover tablespace to 4/19/2013 12:00:00PM  then TEST_TABLE will be dropped.

4. Start test case.

Now we create 2 tablespaces TEST and TEST_IDX . Create TEST_USR user. Create TEST_TABLE table in TEST tablespace and TEST_INDEX in TEST_IDX. Fill table with data. Backup database. Before dropping table we save current SCN and then  drop table. We will do TBSPIR until the saved SCN.

Step 0 :  Clean the previous failed TSPIR

SQL> exec dbms_backup_restore.manageauxinstance (‘instance_name’,1) ;

instance_name will be generated automatically, and you should see it in RMAN output.

Step 1. Create TEST and TEST_IDX tablespaces.

SQL> CREATE TABLESPACE TEST DATAFILE
  ‘D:\APP\ORACLE\ORADATA\ORCL\TEST.DBF’ SIZE 104857600
  AUTOEXTEND ON NEXT 1048576 MAXSIZE 32767M;
 
  SQL> CREATE TABLESPACE TEST_IDX DATAFILE
  ‘D:\APP\ORACLE\ORADATA\ORCL\TEST_IDX.DBF’ SIZE 104857600
  AUTOEXTEND ON NEXT 1048576 MAXSIZE 32767M; 

Step 2. Create TEST_USR user, grant privileges.

SQL> CREATE USER TEST_USR IDENTIFIED BY TEST
DEFAULT TABLESPACE TEST;

SQL> GRANT CONNECT,RESOURCE TO TEST_USR;

Step 3. Create TEST_TABLE table.

SQL> CREATE TABLE TEST_USR.TEST_TABLE(A DATE);

Step 4. Create TEST_INDEX index.

SQL> CREATE INDEX TEST_USR.TEST_INDEX
ON  TEST_USR.TEST_TABLE(A)
TABLESPACE TEST_IDX;

Step 5. Fill data.

SQL> INSERT INTO TEST_USR.TEST_TABLE
VALUES(SYSDATE);

SQL> INSERT INTO TEST_USR.TEST_TABLE
VALUES(SYSDATE);

SQL> COMMIT;

Step 6.

SQL> SELECT *
FROM TEST_USR.TEST_TABLE

A
4/19/2013 12:20:16 PM
4/19/2013 12:20:16 PM

Step 7. Backup database.

RMAN> backup database plus archivelog;

Step 8.  Note the current SCN.

SQL> SELECT CURRENT_SCN
FROM V$DATABASE

CURRENT_SCN
2091595

Step 9. Drop table.

DROP TABLE TEST_USR.TEST_TABLE

Step 10. Create tables in SYSTEM and TEST_IDX tablespaces.

SQL> CREATE TABLE TEST_USR.TEST_USR_SYSTEM(A DATE)
TABLESPACE SYSTEM;

SQL> INSERT INTO TEST_USR.TEST_USR_SYSTEM VALUES(SYSDATE);

SQL> COMMIT;

SQL> CREATE TABLE TEST_USR.TEST_USR_TEST_IDX(A DATE)
TABLESPACE TEST_IDX;

SQL> INSERT INTO TEST_USR.TEST_USR_TEST_IDX VALUES(SYSDATE);

SQL> COMMIT;

This is just for to see that just TEST and TEST_IDX tablespaces are affected. And not SYSTEM or any other tablespaces. Smile

Step 11. Do TSPIR.

Note: Auxiliary destination must already exist and oracle software owner must be able to write into it.

!!! You should recover dependent tablespaces also , or data in these tablespaces will be lost.

For example: If you just recover TEST tablespace, then index on TEST_TABLE will be lost(resides in TEST_IDX tablespasce)… Note also that if you try to recover just TEST_IDX tablespace it will cause the following error:

The transportable set is not self-contained, violation list is Index TEST_USR.TEST_INDEX in tablespace TEST_IDX points to table TEST_USR.TEST_TABLE in tablespace TEST.

!!! Recoverable tablespaces must not contain SYS objects, or it will fail.

!!! Identify the exact point of time or after recovering tablespace, you will not be able to re-run TSPIR by existing backups.

After recovering tablespaces, if you run list backup command:

RMAN> list backup;

File LV Type Ckp SCN    Ckp Time            Name
—- — —- ———- ——————- —-
1       Full 2084320    19-04-2013 16:52:21 D:\APP\ORACLE\ORADATA\ORCL\SYSTEM01.DBF
2       Full 2084320    19-04-2013 16:52:21 D:\APP\ORACLE\ORADATA\ORCL\SYSAUX01.DBF
3       Full 2084320    19-04-2013 16:52:21 D:\APP\ORACLE\ORADATA\ORCL\UNDOTBS01.DBF
4       Full 2084320    19-04-2013 16:52:21 D:\APP\ORACLE\ORADATA\ORCL\USERS01.DBF
5       Full 2084320    19-04-2013 16:52:21
6       Full 2084320    19-04-2013 16:52:21

There are empty strings across recovered tablespaces.

So after TSPIR , you must re-take backup of the recovered tablespaces.

Starting TSPIR…

RMAN> recover tablespace "TEST","TEST_IDX" until scn 2091595 auxiliary destination ‘D:\oracle’;

Starting recover at 19-04-2013 17:57:28
using target database control file instead of recovery catalog
allocated channel: ORA_DISK_1
channel ORA_DISK_1: SID=17 device type=DISK
RMAN-05026: WARNING: presuming following set of tablespaces applies to specified point-in-time

List of tablespaces expected to have UNDO segments
Tablespace SYSTEM
Tablespace UNDOTBS1

Creating automatic instance, with SID=’uecp’

initialization parameters used for automatic instance:
db_name=ORCL
db_unique_name=uecp_tspitr_ORCL
compatible=11.2.0.0.0
db_block_size=8192
db_files=200
sga_target=280M
processes=50
db_create_file_dest=D:\oracle
log_archive_dest_1=’location=D:\oracle’
#No auxiliary parameter file used

starting up automatic instance ORCL

Oracle instance started

Total System Global Area     292933632 bytes

Fixed Size                     1374164 bytes
Variable Size                100665388 bytes
Database Buffers             184549376 bytes
Redo Buffers                   6344704 bytes
Automatic instance created
Running TRANSPORT_SET_CHECK on recovery set tablespaces
TRANSPORT_SET_CHECK completed successfully

contents of Memory Script:
{
# set requested point in time
set until  scn 2091595;
# restore the controlfile
restore clone controlfile;
# mount the controlfile
sql clone ‘alter database mount clone database’;
# archive current online log
sql ‘alter system archive log current’;
# avoid unnecessary autobackups for structural changes during TSPITR
sql ‘begin dbms_backup_restore.AutoBackupFlag(FALSE); end;’;
}
executing Memory Script

executing command: SET until clause

Starting restore at 19-04-2013 17:58:39
allocated channel: ORA_AUX_DISK_1
channel ORA_AUX_DISK_1: SID=59 device type=DISK

channel ORA_AUX_DISK_1: starting datafile backup set restore
channel ORA_AUX_DISK_1: restoring control file
channel ORA_AUX_DISK_1: reading from backup piece D:\APP\ORACLE\FLASH_RECOVERY_AREA\ORCL\BACKUPSET\2013_04_19\O1_MF_NCSNF_TAG20130419T175122_8Q2M0XGP_.BKP
channel ORA_AUX_DISK_1: piece handle=D:\APP\ORACLE\FLASH_RECOVERY_AREA\ORCL\BACKUPSET\2013_04_19\O1_MF_NCSNF_TAG20130419T175122_8Q2M0XGP_.BKP tag=TAG20130419T175122
channel ORA_AUX_DISK_1: restored backup piece 1
channel ORA_AUX_DISK_1: restore complete, elapsed time: 00:00:01
output file name=D:\ORACLE\ORCL\CONTROLFILE\O1_MF_8Q2MDJ5K_.CTL
Finished restore at 19-04-2013 17:58:41

sql statement: alter database mount clone database

sql statement: alter system archive log current

sql statement: begin dbms_backup_restore.AutoBackupFlag(FALSE); end;

contents of Memory Script:
{
# set requested point in time
set until  scn 2091595;
plsql <<<– tspitr_2
declare
  sqlstatement       varchar2(512);
  offline_not_needed exception;
  pragma exception_init(offline_not_needed, -01539);
begin
  sqlstatement := ‘alter tablespace ‘||  ‘"TEST"’ ||’ offline immediate’;
  krmicd.writeMsg(6162, sqlstatement);
  krmicd.execSql(sqlstatement);
exception
  when offline_not_needed then
    null;
end; >>>;
plsql <<<– tspitr_2
declare
  sqlstatement       varchar2(512);
  offline_not_needed exception;
  pragma exception_init(offline_not_needed, -01539);
begin
  sqlstatement := ‘alter tablespace ‘||  ‘"TEST_IDX"’ ||’ offline immediate’;
  krmicd.writeMsg(6162, sqlstatement);
  krmicd.execSql(sqlstatement);
exception
  when offline_not_needed then
    null;
end; >>>;
# set destinations for recovery set and auxiliary set datafiles
set newname for clone datafile  1 to new;
set newname for clone datafile  3 to new;
set newname for clone datafile  2 to new;
set newname for clone tempfile  1 to new;
set newname for datafile  5 to
"D:\APP\ORACLE\ORADATA\ORCL\TEST.DBF";
set newname for datafile  6 to
"D:\APP\ORACLE\ORADATA\ORCL\TEST_IDX.DBF";
# switch all tempfiles
switch clone tempfile all;
# restore the tablespaces in the recovery set and the auxiliary set
restore clone datafile  1, 3, 2, 5, 6;
switch clone datafile all;
}
executing Memory Script

executing command: SET until clause

sql statement: alter tablespace "TEST" offline immediate

sql statement: alter tablespace "TEST_IDX" offline immediate

executing command: SET NEWNAME

executing command: SET NEWNAME

executing command: SET NEWNAME

executing command: SET NEWNAME

executing command: SET NEWNAME

executing command: SET NEWNAME

renamed tempfile 1 to D:\ORACLE\ORCL\DATAFILE\O1_MF_TEMP_%U_.TMP in control file

Starting restore at 19-04-2013 17:58:58
using channel ORA_AUX_DISK_1

channel ORA_AUX_DISK_1: starting datafile backup set restore
channel ORA_AUX_DISK_1: specifying datafile(s) to restore from backup set
channel ORA_AUX_DISK_1: restoring datafile 00001 to D:\ORACLE\ORCL\DATAFILE\O1_MF_SYSTEM_%U_.DBF
channel ORA_AUX_DISK_1: restoring datafile 00003 to D:\ORACLE\ORCL\DATAFILE\O1_MF_UNDOTBS1_%U_.DBF
channel ORA_AUX_DISK_1: restoring datafile 00002 to D:\ORACLE\ORCL\DATAFILE\O1_MF_SYSAUX_%U_.DBF
channel ORA_AUX_DISK_1: restoring datafile 00005 to D:\APP\ORACLE\ORADATA\ORCL\TEST.DBF
channel ORA_AUX_DISK_1: restoring datafile 00006 to D:\APP\ORACLE\ORADATA\ORCL\TEST_IDX.DBF
channel ORA_AUX_DISK_1: reading from backup piece D:\APP\ORACLE\FLASH_RECOVERY_AREA\ORCL\BACKUPSET\2013_04_19\O1_MF_NNNDF_TAG20130419T175122_8Q2LYVQ3_.BKP
channel ORA_AUX_DISK_1: piece handle=D:\APP\ORACLE\FLASH_RECOVERY_AREA\ORCL\BACKUPSET\2013_04_19\O1_MF_NNNDF_TAG20130419T175122_8Q2LYVQ3_.BKP tag=TAG20130419T175122
channel ORA_AUX_DISK_1: restored backup piece 1
channel ORA_AUX_DISK_1: restore complete, elapsed time: 00:01:05
Finished restore at 19-04-2013 18:00:04

datafile 1 switched to datafile copy
input datafile copy RECID=4 STAMP=813175204 file name=D:\ORACLE\ORCL\DATAFILE\O1_MF_SYSTEM_8Q2MF36L_.DBF
datafile 3 switched to datafile copy
input datafile copy RECID=5 STAMP=813175204 file name=D:\ORACLE\ORCL\DATAFILE\O1_MF_UNDOTBS1_8Q2MF3K3_.DBF
datafile 2 switched to datafile copy
input datafile copy RECID=6 STAMP=813175204 file name=D:\ORACLE\ORCL\DATAFILE\O1_MF_SYSAUX_8Q2MF3BV_.DBF

contents of Memory Script:
{
# set requested point in time
set until  scn 2091595;
# online the datafiles restored or switched
sql clone "alter database datafile  1 online";
sql clone "alter database datafile  3 online";
sql clone "alter database datafile  2 online";
sql clone "alter database datafile  5 online";
sql clone "alter database datafile  6 online";
# recover and open resetlogs
recover clone database tablespace  "TEST", "TEST_IDX", "SYSTEM", "UNDOTBS1", "SYSAUX" delete archivelog;
alter clone database open resetlogs;
}
executing Memory Script

executing command: SET until clause

sql statement: alter database datafile  1 online

sql statement: alter database datafile  3 online

sql statement: alter database datafile  2 online

sql statement: alter database datafile  5 online

sql statement: alter database datafile  6 online

Starting recover at 19-04-2013 18:00:06
using channel ORA_AUX_DISK_1

starting media recovery

archived log for thread 1 with sequence 15 is already on disk as file D:\APP\ORACLE\FLASH_RECOVERY_AREA\ORCL\ARCHIVELOG\2013_04_19\O1_MF_1_15_8Q2M11L9_.ARC
archived log for thread 1 with sequence 16 is already on disk as file D:\APP\ORACLE\FLASH_RECOVERY_AREA\ORCL\ARCHIVELOG\2013_04_19\O1_MF_1_16_8Q2MDSHL_.ARC
archived log file name=D:\APP\ORACLE\FLASH_RECOVERY_AREA\ORCL\ARCHIVELOG\2013_04_19\O1_MF_1_15_8Q2M11L9_.ARC thread=1 sequence=15
archived log file name=D:\APP\ORACLE\FLASH_RECOVERY_AREA\ORCL\ARCHIVELOG\2013_04_19\O1_MF_1_16_8Q2MDSHL_.ARC thread=1 sequence=16
media recovery complete, elapsed time: 00:00:02
Finished recover at 19-04-2013 18:00:10

database opened

contents of Memory Script:
{
# make read only the tablespace that will be exported
sql clone ‘alter tablespace  "TEST" read only’;
sql clone ‘alter tablespace  "TEST_IDX" read only’;
# create directory for datapump import
sql "create or replace directory TSPITR_DIROBJ_DPDIR as ”
D:\oracle”";
# create directory for datapump export
sql clone "create or replace directory TSPITR_DIROBJ_DPDIR as ”
D:\oracle”";
}
executing Memory Script

sql statement: alter tablespace  "TEST" read only

sql statement: alter tablespace  "TEST_IDX" read only

sql statement: create or replace directory TSPITR_DIROBJ_DPDIR as ”D:\oracle”

sql statement: create or replace directory TSPITR_DIROBJ_DPDIR as ”D:\oracle”

Performing export of metadata…
   EXPDP> Starting "SYS"."TSPITR_EXP_uecp":
   EXPDP> Processing object type TRANSPORTABLE_EXPORT/PLUGTS_BLK
   EXPDP> Processing object type TRANSPORTABLE_EXPORT/TABLE
   EXPDP> Processing object type TRANSPORTABLE_EXPORT/INDEX
   EXPDP> Processing object type TRANSPORTABLE_EXPORT/INDEX_STATISTICS
   EXPDP> Processing object type TRANSPORTABLE_EXPORT/POST_INSTANCE/PLUGTS_BLK
   EXPDP> Master table "SYS"."TSPITR_EXP_uecp" successfully loaded/unloaded
   EXPDP> ******************************************************************************
   EXPDP> Dump file set for SYS.TSPITR_EXP_uecp is:
   EXPDP>   D:\ORACLE\TSPITR_UECP_36510.DMP
   EXPDP> ******************************************************************************
   EXPDP> Datafiles required for transportable tablespace TEST:
   EXPDP>   D:\APP\ORACLE\ORADATA\ORCL\TEST.DBF
   EXPDP> Datafiles required for transportable tablespace TEST_IDX:
   EXPDP>   D:\APP\ORACLE\ORADATA\ORCL\TEST_IDX.DBF
   EXPDP> Job "SYS"."TSPITR_EXP_uecp" successfully completed at 18:01:30
Export completed

contents of Memory Script:
{
# shutdown clone before import
shutdown clone immediate
# drop target tablespaces before importing them back
sql ‘drop tablespace  "TEST" including contents keep datafiles’;
sql ‘drop tablespace  "TEST_IDX" including contents keep datafiles’;
}
executing Memory Script

database closed
database dismounted
Oracle instance shut down

sql statement: drop tablespace  "TEST" including contents keep datafiles

sql statement: drop tablespace  "TEST_IDX" including contents keep datafiles

Performing import of metadata…
   IMPDP> Master table "SYS"."TSPITR_IMP_uecp" successfully loaded/unloaded
   IMPDP> Starting "SYS"."TSPITR_IMP_uecp":
   IMPDP> Processing object type TRANSPORTABLE_EXPORT/PLUGTS_BLK
   IMPDP> Processing object type TRANSPORTABLE_EXPORT/TABLE
   IMPDP> Processing object type TRANSPORTABLE_EXPORT/INDEX
   IMPDP> Processing object type TRANSPORTABLE_EXPORT/INDEX_STATISTICS
   IMPDP> Processing object type TRANSPORTABLE_EXPORT/POST_INSTANCE/PLUGTS_BLK
   IMPDP> Job "SYS"."TSPITR_IMP_uecp" successfully completed at 18:01:57
Import completed

contents of Memory Script:
{
# make read write and offline the imported tablespaces
sql ‘alter tablespace  "TEST" read write’;
sql ‘alter tablespace  "TEST" offline’;
sql ‘alter tablespace  "TEST_IDX" read write’;
sql ‘alter tablespace  "TEST_IDX" offline’;
# enable autobackups after TSPITR is finished
sql ‘begin dbms_backup_restore.AutoBackupFlag(TRUE); end;’;
}
executing Memory Script

sql statement: alter tablespace  "TEST" read write

sql statement: alter tablespace  "TEST" offline

sql statement: alter tablespace  "TEST_IDX" read write

sql statement: alter tablespace  "TEST_IDX" offline

sql statement: begin dbms_backup_restore.AutoBackupFlag(TRUE); end;

Removing automatic instance
Automatic instance removed
auxiliary instance file D:\ORACLE\ORCL\DATAFILE\O1_MF_TEMP_8Q2MHOX7_.TMP deleted
auxiliary instance file D:\ORACLE\ORCL\ONLINELOG\O1_MF_3_8Q2MHJJJ_.LOG deleted
auxiliary instance file D:\ORACLE\ORCL\ONLINELOG\O1_MF_2_8Q2MHGMY_.LOG deleted
auxiliary instance file D:\ORACLE\ORCL\ONLINELOG\O1_MF_1_8Q2MHC0J_.LOG deleted
auxiliary instance file D:\ORACLE\ORCL\DATAFILE\O1_MF_SYSAUX_8Q2MF3BV_.DBF deleted
auxiliary instance file D:\ORACLE\ORCL\DATAFILE\O1_MF_UNDOTBS1_8Q2MF3K3_.DBF deleted
auxiliary instance file D:\ORACLE\ORCL\DATAFILE\O1_MF_SYSTEM_8Q2MF36L_.DBF deleted
auxiliary instance file D:\ORACLE\ORCL\CONTROLFILE\O1_MF_8Q2MDJ5K_.CTL deleted
Finished recover at 19-04-2013 18:02:05

RMAN>

Step 12. Make TEST and TEST_IDX tablespaces online.

ALTER TABLESPACE TEST ONLINE;

ALTER TABLESPACE TEST_IDX ONLINE;

Step 13.  Check what was and wasn’t recovered.

SQL> select *
from TEST_USR.TEST_USR_SYSTEM;

A
4/19/2013 5:53:01 PM

SQL> select *
from TEST_USR.TEST_USR_TEST_IDX;

ORA-00942: table or view does not exist

SQL> select *
from TEST_USR.TEST_TABLE;

A
4/19/2013 5:50:44 PM
4/19/2013 5:50:44 PM

 

So just “TEST” and “TEST_IDX” tablespaces are affected, nice Smile

Good Luck!

Multipath configuration on RHEL6

1. Check if you have already installed device-mapper-multipath rpm, if not then install it.

rpm -qa device-mapper-multipath

2. If /etc/multipath.conf file doesn’t exist, then copy it from /usr/share/doc/device-mapper-multipath-*

cp /usr/share/doc/device-mapper-multipath-0.4.9/multipath.conf /etc/multipath.conf

3. Find WWIDs that should be added to multipath configuration.

# scsi_id -g -u /dev/sdb
36001438009b044d90000900000780000

4. Edit the /etc/multipath.conf configuration file

defaults {
        user_friendly_names yes
        path_grouping_policy    failover
}

blacklist {
        wwid "*"
}

blacklist_exceptions {
        wwid "36001438009b044d90000900000780000"
}

multipaths {
        multipath {
                wwid                    "36001438009b044d90000900000780000"
                alias                   asm1
        }
}

5.  Add module to the Linux kernel:

modprobe dm-multipath

6. Start multipath service:

service multipathd start

7. If you have any syntax errors or any parameters that do not work in your Linux version, the following command will show:

multipath -d

8. Commit the configuration:

multipath -v2

9. The following command must find the paths , or you have a bad configuration in multipath.conf file:

multipath -ll

10. Make devices configured after a reboot:

chkconfig multipathd on

If you have made any mistakes in multipath.conf file then correct them and do  the following steps to make changes take affect :

1. edit the /etc/multipath.conf

2.  Reload the multipath service:

service multipathd reload

3.  Remove all unused multipath devices

multipath -F

4.  Check again that syntax is correct:

multipath –d

5.  Commit the changes:

multipath –v2

Note that, this configuration is very simple, but it is working also perfectly.

For more multipath options and more sophisticated configuration, see the following documentation.

Installation problem of OEL6 on HP ProLiant DL360e Gen8 with HP Dynamic Smart Array B320i Controller

This post is dedicated to the Oracle Enterprise Linux 6.x installation on HP server with Dynamic Smart Array B320i Controller.

Brief description of the problem:

During OEL6 installation on HP server installer was not able to see local disks, on which system should be installed. Local disk specification was the following: Two 300GB disks involved into the RAID 1 and controller for RAID was Dynamic Smart Array B320i.

Installer was able to see multipath devices and any other external devices, like flash drive, but not local disks.

On HP site, there is written that the minimum supported Oracle Linux updates for HP  DL380e series are the following:

DL380e Gen8

  • Oracle Linux/UEK 6.2
  • Oracle Linux/UEK 5.8

Minimum support includes all future updates of the indicated release unless a maximum is listed. (Example: Oracle 6 implies support for Oracle 6.x, unless it’s specifically called out in the notes as "not supported with 6.x")”

And HP also declaims that it gives its drivers to the supported OSs manufacturers.

To tell the truth, I’ve tried Oracle Enterprise Linux 5.5, 6.0, 6.2 and 6.4 versions, but none of them was able to see the local storage.

After a lot of troubleshooting, I found the solution.

Solution:

The solution implies the HP Dynamic Smart Array SATA RAID Controller Driver for Red Hat Enterprise Linux 6 installation before OS installation.

1. Download drivers from the following link, click here.

Note: hpvsa-1.2.4-4.rhel6u1.x86_64.dd.gz is for RHEL6.1
hpvsa-1.2.4-4.rhel6u2.x86_64.dd.gz is for RHEL6.2
hpvsa-1.2.4-4.rhel6u3.x86_64.dd.gz is for RHEL6.3

Because of we are installing Oracle Enterprise Linux 6.2 which is based on RHEL6.2, we need hpvsa-1.2.4-4.rhel6u2.x86_64.dd.gz.

2. Extract gz file, you should have file with the extension of dd. Write this file to the flash drive and connect it to the server.

3. Put Oracle Enterprise Linux 6.2 installation disk and when the following window appears, press the TAB.

image

At the end of the command write install dd(separated from the previous command with space) and press Enter.

4. Choose the driver file, that we mentioned earlier and continue the installation.

Oracle EL should now see the local disk without any problem. Good Luck.

Configure SSL Connections on Report Server(https)

You must have a server certificate installed on the computer. Ask for the certificate to your sysadmins(windows admins), or request a certificate from a trusted certificate authority.

The following steps do not include instructions for requesting, generating, downloading, or installing a certificate. You must have a certificate installed and available to use.

So let’s start.

1. Run Reporting Services Configuration Manager and connect to the reporting server.

2. In Web Service URL section –> SSL Certificate –> choose your certificate –> click Apply.

3. Report Manager URL section –> Advanced button –> Multiple SSL identities for Report Manager section –>

click Add button –> in Certificate choose your certificate and click OK –> OK.

4. Stop and Start your Reporting service, from the first section of the configuration manager.

5. Check that the URL works by indicating https instead of http.