History | Log In     View a printable version of the current page.  
   HOME       BROWSE PROJECT       FIND ISSUES   
    QUICK SEARCH:  
Issue Details (XML | Word | Printable)

Key: QA-47
Type: Oracle - Operating System Oracle - Operating System
Status: Closed Closed
Resolution: Answered
Priority: Major Major
Assignee: ubTools Support
Reporter: ubTools Support
Votes: 0
Watchers: 0
Operations

If you were logged in you would be able to see more operations.
Questions & Answers

ORA-00354 ORA-00353 ORA-00312: Redolog Block Corruption

Created: 10/Mar/09 02:33 AM   Updated: 10/Apr/09 01:13 PM
Fix Version/s: None

Product Version: Oracle 10.2.0.4 SE,RAC
Operating System: Solaris
Operating System Version: 10


 Description  « Hide
Problem:

Import causes instance to be hang. During import only one instance is open.

imp system/manager file=../yedek/gedik_full.dmp log=../yedek/gedik_full_imp3.log full=y FEEDBACK=1000000
buffer=10000000 RESUMABLE=y RESUMABLE_TIMEOUT=72000

Diagnostic Data for Oracle:

Alert Log:

Mon Mar  9 19:38:45 2009
ARC0: Log corruption near block 50941 change 9160702125 time ?
Mon Mar  9 19:38:45 2009
Errors in file /u01/app/oracle/admin/ORCL/bdump/orcl1_arc0_26085.trc:
ORA-00354: corrupt redo log block header
ORA-00353: log corruption near block 50941 change 9160702125 time 03/09/2009 1
9:38:35
ORA-00312: online log 1 thread 1: '+DATA/orcl/onlinelog/group_1.516.680795507'
ARC0: All Archive destinations made inactive due to error 354
Mon Mar  9 19:38:45 2009
ARC0: Closing local archive destination LOG_ARCHIVE_DEST_1: '/u01/app/oracle/p
roduct/10.2.0/dbs/arch/1_27_681074311.dbf' (error 354)
 (ORCL1)
Committing creation of archivelog '/u01/app/oracle/product/10.2.0/dbs/arch/1_2
7_681074311.dbf' (error 354)
ARCH: Archival stopped, error occurred. Will continue retrying

Archive Log Trace:

Corrupt redo block 50941 detected: bad block number
Flag: 0x30 Format: 0x38 Block: 0x20302030 Seq: 0x5c305c79 Beg: 0x3030 Cks:0x5
c31
----- Dump of Corrupt Redo Buffer -----
5c463830203020305c305c795c3130305c3230305c305c305c305c3020665c30
3030433c5c305c345c305c305c305c305c305c3035320a303a35323920202009
5c305c5030305c303033203120305c32303920383022203520305c315c353034
5c305c3020305c3020725c305c3820373035317231305c315c305c330a305c30
3239353220093a35203843203030433f20372034203530395c3230225c393830
30313030203020315c3630795c3431305c3430305c463830203020305c305c79
5c3130305c32303035320a303a3532395c2020095c305c305c305c30433c2066
5c3430305c305c305c305c305c305c3020305c305c305c5030305c3030292031
20305c32303920383022203520305c310a3320363239353220093a355c305c20
5c305c30203620303038203331725c455c625c365c3331305c305c3020665c30
3030433c20372034203530395c3230225c3938305c3530302030203035320a79
3a3532393020200931305c365c305c3038305c6230305c4641305c3431305c35
5c305c3030305c3020305c3542305c3f5c305c3031305c3030305c3131305c31
425420440a4920353239353220093a35314345205c305c395c3130305c323062
20382030203530395c313022203530305c305c345c305c302035303020372034
303530365c3831315c3230304646463035320a463a3532394320200938412032
Rereading log member '+DATA/orcl/onlinelog/group_1.516.680795507' (corruption
)
...
Corrupt redo block 50941 detected: bad block number
Flag: 0x0 Format: 0x0 Block: 0x00000000 Seq: 0x00000000 Beg: 0x0 Cks:0x0
----- Dump of Corrupt Redo Buffer -----
0000000000000000000000000000000000000000000000000000000000000000
0000000000000000000000000000000000000000000000000000000000000000
0000000000000000000000000000000000000000000000000000000000000000
0000000000000000000000000000000000000000000000000000000000000000
0000000000000000000000000000000000000000000000000000000000000000
0000000000000000000000000000000000000000000000000000000000000000
0000000000000000000000000000000000000000000000000000000000000000
0000000000000000000000000000000000000000000000000000000000000000
0000000000000000000000000000000000000000000000000000000000000000
0000000000000000000000000000000000000000000000000000000000000000
0000000000000000000000000000000000000000000000000000000000000000
0000000000000000000000000000000000000000000000000000000000000000
0000000000000000000000000000000000000000000000000000000000000000
0000000000000000000000000000000000000000000000000000000000000000
0000000000000000000000000000000000000000000000000000000000000000
0000000000000000000000000000000000000000000000000000000000000000
Rereading log member '+DATA/orcl/onlinelog/group_1.516.680795507' (corruption
)
...
Corrupt redo block 50941 detected: bad block number
Flag: 0x30 Format: 0x38 Block: 0x20302030 Seq: 0x5c305c79 Beg: 0x3030 Cks:0x5
c31
----- Dump of Corrupt Redo Buffer -----
5c463830203020305c305c795c3130305c3230305c305c305c305c3020665c30
3030433c5c305c345c305c305c305c305c305c3035320a303a35323920202009
5c305c5030305c303033203120305c32303920383022203520305c315c353034
5c305c3020305c3020725c305c3820373035317231305c315c305c330a305c30
3239353220093a35203843203030433f20372034203530395c3230225c393830
30313030203020315c3630795c3431305c3430305c463830203020305c305c79
5c3130305c32303035320a303a3532395c2020095c305c305c305c30433c2066
5c3430305c305c305c305c305c305c3020305c305c305c5030305c3030292031
20305c32303920383022203520305c310a3320363239353220093a355c305c20
5c305c30203620303038203331725c455c625c365c3331305c305c3020665c30
3030433c20372034203530395c3230225c3938305c3530302030203035320a79
3a3532393020200931305c365c305c3038305c6230305c4641305c3431305c35
5c305c3030305c3020305c3542305c3f5c305c3031305c3030305c3131305c31
425420440a4920353239353220093a35314345205c305c395c3130305c323062
20382030203530395c313022203530305c305c345c305c302035303020372034
303530365c3831315c3230304646463035320a463a3532394320200938412032
*** 2009-03-10 03:55:10.757 62692 kcrr.c

As seen above, even if the database hangs, the contents of redo buffer dump change.

Diagnostic Data for Solaris:

Soft Link Mapping to Raw Devices:

oravol1: disk@g600a0b80005a81660000074949959b42:b,raw
oravol2: disk@g600a0b80005a816600000742499595ea:b,raw
oravol3: disk@g600a0b80005a8166000007444995971e:b,raw
oravol4: disk@g600a0b80005a8c9f000004f049959717:b,raw
oravol5: disk@g600a0b80005a8c9f000004f249959991:b,raw

Open File Descriptors of ARCH process:

bash-3.00$ ps -ef|grep arc0
  oracle 19941 14227   0 04:10:31 pts/12      0:00 grep arc0
  oracle 26085     1   0 19:25:05 ?           0:29 ora_arc0_ORCL1

bash-3.00$ ls -ltr /proc/26085/path
...
lrwxrwxrwx   1 oracle   oinstall       0 Mar  9 19:25 261
 -> /devices/scsi_vhci/disk@g600a0b80005a816600000742499595ea:b,raw
lrwxrwxrwx   1 oracle   oinstall       0 Mar  9 19:25 260
 -> /devices/scsi_vhci/disk@g600a0b80005a8c9f000004f049959717:b,raw
lrwxrwxrwx   1 oracle   oinstall       0 Mar  9 19:25 259
 -> /devices/scsi_vhci/disk@g600a0b80005a81660000074949959b42:b,raw
lrwxrwxrwx   1 oracle   oinstall       0 Mar  9 19:25 257
 -> /devices/scsi_vhci/disk@g600a0b80005a8166000007444995971e:b,raw
lrwxrwxrwx   1 oracle   oinstall       0 Mar  9 19:25 256
 -> /devices/scsi_vhci/disk@g600a0b80005a8c9f000004f249959991:b,raw
...
bash-3.00$

Gathering truss output for ARCH:

truss -fae -w 261,260,259,257,256 -r 261,260,259,257,256 -o arc0.truss.log -p 26085

The command above will trace system calls with pread()/pwrite() IO buffer dumping for fd of 261,260,259,257,256.

Open File Descriptors of LGWR process:

bash-3.00$ ps -ef|grep lgwr
  oracle 28447     1   0   Mar 04 ?           0:17 asm_lgwr_+ASM1
  oracle 25925     1   0 19:24:49 ?           0:38 ora_lgwr_ORCL1
  oracle 26468 14227   0 04:21:02 pts/12      0:00 grep lgwr

bash-3.00$ ls -ltr /proc/25925/path
...
lrwxrwxrwx   1 oracle   oinstall       0 Mar  9 19:24 260
 -> /devices/scsi_vhci/disk@g600a0b80005a816600000742499595ea:b,raw
lrwxrwxrwx   1 oracle   oinstall       0 Mar  9 19:24 259
 -> /devices/scsi_vhci/disk@g600a0b80005a81660000074949959b42:b,raw
lrwxrwxrwx   1 oracle   oinstall       0 Mar  9 19:24 258
 -> /devices/scsi_vhci/disk@g600a0b80005a8c9f000004f049959717:b,raw
lrwxrwxrwx   1 oracle   oinstall       0 Mar  9 19:24 257
 -> /devices/scsi_vhci/disk@g600a0b80005a8166000007444995971e:b,raw
lrwxrwxrwx   1 oracle   oinstall       0 Mar  9 19:24 256
 -> /devices/scsi_vhci/disk@g600a0b80005a8c9f000004f249959991:b,raw
...

Gathering truss output for ARCH:

bash-3.00$ truss -fae -w 260,259,258,257,256 -r 260,259,258,257,256 -o lgwr.truss.log -p 25925 &

The command above will trace system calls with pread()/pwrite() IO buffer dumping for fd of 260,259,258,257,256.

 All   Comments   Change History      Sort Order: Ascending order - Click to sort in descending order
Change by ubTools Support - 10/Apr/09 01:13 PM
Field Original Value New Value
Status Open [ 1 ] Closed [ 6 ]
Resolution Answered [ 10 ]


Powered by Atlassian JIRA the Professional Issue Tracker. (Standard Edition, Version: 3.12.3-#302) - Bug/feature request - Atlassian news - Contact Administrators