<< Back to previous view

[QA-53] Starting Listener Hangs with "TNS-12531: TNS:cannot allocate memory" in Listener Log Created: 02/Jul/12  Updated: 12/Jul/12

Status: Closed
Project: Questions & Answers
Fix Version/s: None

Type: Oracle - SQL*Net Priority: Major
Reporter: ubTools Support Assignee: ubTools Support
Resolution: Answered Votes: 0

Product Version: 11.2.0.3
Operating System: Linux
Host Name: .
Database Name: .

 Description   
Starting the LISTENER hangs. The following errors appear as an infinite loop in the listener.log: 
02-JUL-2012 15:54:06 * 12531
TNS-12531: TNS:cannot allocate memory
02-JUL-2012 15:54:06 * 12531
TNS-12531: TNS:cannot allocate memory
02-JUL-2012 15:54:06 * 12531
TNS-12531: TNS:cannot allocate memory
02-JUL-2012 15:54:06 * 12531
TNS-12531: TNS:cannot allocate memory
02-JUL-2012 15:54:06 * 12531
TNS-12531: TNS:cannot allocate memory


 Comments   
Comment by ubTools Support [ 02/Jul/12 02:19 PM ]
LISTENER trace enabled in listener.ora as below: 
TRACE_LEVEL_LISTENER     = 16
TRACE_FILE_LISTENER      = listener.trc
TRACE_UNIQUE_LISTENER    = TRUE
TRACE_TIMESTAMP_LISTENER = TRUE

listener.trc was generated in $ORACLE_BASE/diag/tnslsnr/linux1/listener/trace/ as below:

2012-07-02 15:55:03.847203 : snlinGetAddrInfo:entry
2012-07-02 15:55:03.847276 : snlinGetAddrInfo:getaddrinfo() failed with error -3
2012-07-02 15:55:03.847295 : snlinGetAddrInfo:exit
2012-07-02 15:55:03.847307 : nserror:entry
2012-07-02 15:55:03.847319 : nserror:nsres: id=0, op=65, ns=12531, ns2=0; nt[0]=0, nt[1]=0, nt[2]=0; ora[0]=0, ora[1]=0, ora[2]=0
2012-07-02 15:55:03.847331 : nsmfr:entry
2012-07-02 15:55:03.847342 : nsmfr:1528 bytes at 0xa193a0
2012-07-02 15:55:03.847352 : nsmfr:normal exit
2012-07-02 15:55:03.847363 : nsopenmplx:error exit
2012-07-02 15:55:03.847373 : nsopen:unable to allocate context area
2012-07-02 15:55:03.847384 : nsopen:error exit
2012-07-02 15:55:03.847395 : nsanswer:error exit
2012-07-02 15:55:03.847411 : nsglhc:nsanswer error 12531

The problem appeared in getaddrinfo() system call.

Comment by ubTools Support [ 02/Jul/12 02:22 PM ]
The IPv4 for hostname was defined in the /etc/hosts; but there was no IPv6 definition.

Even though, only IPv4 address was used in the listener.ora, the problem occured again.

The problem has been disappeared after adding the same hostname as IPv6 to the /etc/hosts.




[QA-46] ORA-12545: Connect failed in RAC environment because of an implicit redirect to another node. Created: 27/Feb/09  Updated: 27/Feb/09

Status: Closed
Project: Questions & Answers
Fix Version/s: None

Type: Oracle - SQL*Net Priority: Major
Reporter: ubTools Support Assignee: ubTools Support
Resolution: Not a Problem Votes: 0

Product Version: Oracle 10.2.0.4 Standard Edition, RAC
Operating System: Solaris
Operating System Version: 10

 Description   
Description:

The clients can not connect to the database with ORA-12545 error even if They can ping the database server.

Diagnostic Data for Oracle:

Remote and Local Listeners for Both Nodes:

SQL> show parameter listener

NAME                                 TYPE        VALUE
------------------------------------ ----------- ------------------------------
local_listener                       string
remote_listener                      string      LISTENERS_ORCL
SQL>

Remote Listener Configuration for Both Nodes:

LISTENERS_ORCL =
  (ADDRESS_LIST =
    (ADDRESS = (PROTOCOL = TCP)(HOST = gdksun1-vip)(PORT = 1521))
    (ADDRESS = (PROTOCOL = TCP)(HOST = gdksun2-vip)(PORT = 1521))
    (ADDRESS = (PROTOCOL = TCP)(HOST = gdksun1-pubext-vip)(PORT = 1521))
    (ADDRESS = (PROTOCOL = TCP)(HOST = gdksun2-pubext-vip)(PORT = 1521))
  )

tns alias

SUNGDK =
 (DESCRIPTION =
   (ADDRESS = (PROTOCOL = TCP)(HOST = <IP0>)(PORT = 1521))
   (ADDRESS = (PROTOCOL = TCP)(HOST = <IP1>)(PORT = 1521))
   (LOAD_BALANCE = yes)
   (CONNECT_DATA =
     (SERVER = DEDICATED)
     (SERVICE_NAME = ORCL)
     (FAILOVER_MODE =
       (TYPE = SELECT)
       (METHOD = BASIC)
       (RETRIES = 180)
       (DELAY = 5)
     )
   )
 )

sqlnet trace parameters

TRACE_LEVEL_CLIENT      = 16
TRACE_FILE_CLIENT       = sqlnet.trc
TRACE_DIRECTORY_CLIENT  = <dizinAdı>
TRACE_UNIQUE_CLIENT     = ON
TRACE_TIMESTAMP_CLIENT  = ON

sqlnet trace

(5996) [27-ŞUB-2009 20:57:50:875] nttgetport: port resolved to 1521
(5996) [27-ŞUB-2009 20:57:50:875] nttgetport: exit
(5996) [27-ŞUB-2009 20:57:50:875] nttbnd2addr: using host IP address: <IP1>
(5996) [27-ŞUB-2009 20:57:50:875] nttbnd2addr: exit
(5996) [27-ŞUB-2009 20:57:50:875] nsc2addr: normal exit

The host IP and port are resolved to <IP1> and 1521, respectively.

(5996) [27-ŞUB-2009 20:57:50:937] nscon: sending NSPTCN packet
(5996) [27-ŞUB-2009 20:57:50:937] nspsend: entry
(5996) [27-ŞUB-2009 20:57:50:937] nspsend: plen=58, type=1
(5996) [27-ŞUB-2009 20:57:50:937] nttwr: entry
(5996) [27-ŞUB-2009 20:57:50:937] nttwr: socket 420 had bytes written=58
(5996) [27-ŞUB-2009 20:57:50:937] nttwr: exit
(5996) [27-ŞUB-2009 20:57:50:937] nspsend: 58 bytes to transport
(5996) [27-ŞUB-2009 20:57:50:937] nspsend: packet dump
(5996) [27-ŞUB-2009 20:57:50:937] nspsend: 00 3A 00 00 01 00 00 00  |.:......|
(5996) [27-ŞUB-2009 20:57:50:937] nspsend: 01 38 01 2C 00 00 08 00  |.8.,....|
(5996) [27-ŞUB-2009 20:57:50:937] nspsend: 7F FF 86 0E 00 00 01 00  |........|
(5996) [27-ŞUB-2009 20:57:50:937] nspsend: 01 3E 00 3A 00 00 02 00  |.>.:....|
(5996) [27-ŞUB-2009 20:57:50:937] nspsend: 21 21 00 00 00 00 00 00  |!!......|
(5996) [27-ŞUB-2009 20:57:50:937] nspsend: 00 00 00 00 0A C0 00 00  |........|
(5996) [27-ŞUB-2009 20:57:50:937] nspsend: 00 0A 00 00 00 00 00 00  |........|
(5996) [27-ŞUB-2009 20:57:50:937] nspsend: 00 00                    |..      |
(5996) [27-ŞUB-2009 20:57:50:937] nspsend: normal exit

A connect packet (NSPTCN) sent to <IP1>.

(5996) [27-ŞUB-2009 20:57:50:937] nsdofls: sending NSPTDA packet
(5996) [27-ŞUB-2009 20:57:50:937] nspsend: entry
(5996) [27-ŞUB-2009 20:57:50:937] nspsend: plen=328, type=6
(5996) [27-ŞUB-2009 20:57:50:937] nttwr: entry
(5996) [27-ŞUB-2009 20:57:50:937] nttwr: socket 420 had bytes written=328
(5996) [27-ŞUB-2009 20:57:50:937] nttwr: exit
(5996) [27-ŞUB-2009 20:57:50:937] nspsend: 328 bytes to transport
(5996) [27-ŞUB-2009 20:57:50:937] nspsend: packet dump
(5996) [27-ŞUB-2009 20:57:50:937] nspsend: 01 48 00 00 06 00 00 00  |.H......|
(5996) [27-ŞUB-2009 20:57:50:937] nspsend: 00 00 28 44 45 53 43 52  |..(DESCR|
(5996) [27-ŞUB-2009 20:57:50:937] nspsend: 49 50 54 49 4F 4E 3D 28  |IPTION=(|
(5996) [27-ŞUB-2009 20:57:50:937] nspsend: 41 44 44 52 45 53 53 3D  |ADDRESS=|
(5996) [27-ŞUB-2009 20:57:50:937] nspsend: 28 50 52 4F 54 4F 43 4F  |(PROTOCO|
(5996) [27-ŞUB-2009 20:57:50:937] nspsend: 4C 3D 54 43 50 29 28 48  |L=TCP)(H|
...
(5996) [27-ŞUB-2009 20:57:50:937] nspsend: 75 72 61 64 3F 54 75 6C  |urad?Tul|
(5996) [27-ŞUB-2009 20:57:50:937] nspsend: 75 6E 61 79 29 29 29 29  |unay))))|
(5996) [27-ŞUB-2009 20:57:50:937] nspsend: normal exit

A data packet (NSPTDA) sent to <IP1>.

(5996) [27-ŞUB-2009 20:57:50:937] nscon: recving a packet
(5996) [27-ŞUB-2009 20:57:50:937] nsprecv: entry
(5996) [27-ŞUB-2009 20:57:50:937] nsbal: entry
(5996) [27-ŞUB-2009 20:57:50:937] nsbgetfl: entry
(5996) [27-ŞUB-2009 20:57:50:937] nsbgetfl: normal exit
(5996) [27-ŞUB-2009 20:57:50:937] nsmal: entry
(5996) [27-ŞUB-2009 20:57:50:937] nsmal: 48 bytes at 0x15bcf60
(5996) [27-ŞUB-2009 20:57:50:937] nsmal: normal exit
(5996) [27-ŞUB-2009 20:57:50:937] nsbal: normal exit
(5996) [27-ŞUB-2009 20:57:50:937] nsprecv: reading from transport...
(5996) [27-ŞUB-2009 20:57:50:937] nttrd: entry
(5996) [27-ŞUB-2009 20:57:50:968] nttrd: socket 420 had bytes read=10
(5996) [27-ŞUB-2009 20:57:50:968] nttrd: exit
(5996) [27-ŞUB-2009 20:57:50:968] nsprecv: 10 bytes from transport
(5996) [27-ŞUB-2009 20:57:50:968] nsprecv: tlen=10, plen=10, type=5
(5996) [27-ŞUB-2009 20:57:50:968] nsprecv: packet dump
(5996) [27-ŞUB-2009 20:57:50:968] nsprecv: 00 0A 00 00 05 02 00 00  |........|
(5996) [27-ŞUB-2009 20:57:50:968] nsprecv: 01 85                    |..      |
(5996) [27-ŞUB-2009 20:57:50:968] nsprecv: normal exit
(5996) [27-ŞUB-2009 20:57:50:968] nscon: got NSPTRD packet

Got a redirect packet (NSPTRD) from <IP1>.

(5996) [27-ŞUB-2009 20:57:50:968] nsrdr: recving a packet
(5996) [27-ŞUB-2009 20:57:50:968] nsprecv: entry
(5996) [27-ŞUB-2009 20:57:50:968] nsprecv: reading from transport...
(5996) [27-ŞUB-2009 20:57:50:968] nttrd: entry
(5996) [27-ŞUB-2009 20:57:50:968] nttrd: socket 420 had bytes read=399
(5996) [27-ŞUB-2009 20:57:50:968] nttrd: exit
(5996) [27-ŞUB-2009 20:57:50:968] nsprecv: 399 bytes from transport
(5996) [27-ŞUB-2009 20:57:50:968] nsprecv: tlen=399, plen=399, type=6
(5996) [27-ŞUB-2009 20:57:50:968] nsprecv: packet dump
(5996) [27-ŞUB-2009 20:57:50:968] nsprecv: 01 8F 00 00 06 00 00 00  |........|
(5996) [27-ŞUB-2009 20:57:50:968] nsprecv: 00 40 28 41 44 44 52 45  |.@(ADDRE|
(5996) [27-ŞUB-2009 20:57:50:968] nsprecv: 53 53 3D 28 50 52 4F 54  |SS=(PROT|
(5996) [27-ŞUB-2009 20:57:50:968] nsprecv: 4F 43 4F 4C 3D 54 43 50  |OCOL=TCP|
(5996) [27-ŞUB-2009 20:57:50:968] nsprecv: 29 28 48 4F 53 54 3D 67  |)(HOST=g|
(5996) [27-ŞUB-2009 20:57:50:968] nsprecv: 64 6B 73 75 6E 32 29 28  |dksun2)(|
(5996) [27-ŞUB-2009 20:57:50:968] nsprecv: 50 4F 52 54 3D 31 35 32  |PORT=152|
(5996) [27-ŞUB-2009 20:57:50:968] nsprecv: 31 29 29 00 28 44 45 53  |1)).(DES|
(5996) [27-ŞUB-2009 20:57:50:968] nsprecv: 43 52 49 50 54 49 4F 4E  |CRIPTION|
(5996) [27-ŞUB-2009 20:57:50:968] nsprecv: 3D 28 41 44 44 52 45 53  |=(ADDRES|
(5996) [27-ŞUB-2009 20:57:50:968] nsprecv: 53 3D 28 50 52 4F 54 4F  |S=(PROTO|
(5996) [27-ŞUB-2009 20:57:50:968] nsprecv: 43 4F 4C 3D 54 43 50 29  |COL=TCP)|
...
(5996) [27-ŞUB-2009 20:57:50:968] nsprecv: 3D 4D 75 72 61 64 3F 54  |=Murad?T|
(5996) [27-ŞUB-2009 20:57:50:968] nsprecv: 75 6C 75 6E 61 79 29 29  |ulunay))|
(5996) [27-ŞUB-2009 20:57:50:968] nsprecv: 28 49 4E 53 54 41 4E 43  |(INSTANC|
(5996) [27-ŞUB-2009 20:57:50:968] nsprecv: 45 5F 4E 41 4D 45 3D 6F  |E_NAME=o|
(5996) [27-ŞUB-2009 20:57:50:968] nsprecv: 72 63 6C 32 29 29 29     |rcl2))) |
(5996) [27-ŞUB-2009 20:57:50:968] nsprecv: normal exit
(5996) [27-ŞUB-2009 20:57:50:968] nsrdr: got NSPTDA packet

Got a data packet (NSPTDA) from <IP1>.

(5996) [27-ŞUB-2009 20:57:50:984] nttgetport: port resolved to 1521
(5996) [27-ŞUB-2009 20:57:50:984] nttgetport: exit
(5996) [27-ŞUB-2009 20:57:50:984] nttbnd2addr: looking up IP addr for host: gdksun2
(5996) [27-ŞUB-2009 20:57:53:640] nttbnd2addr:  *** hostname lookup failure! ***
(5996) [27-ŞUB-2009 20:57:53:640] nttbnd2addr: exit


As seen above, even if the initial request was sent to <IP1>, now it's redirected to an host named gdksun2.

(5996) [27-ŞUB-2009 20:57:53:640] nserror: entry
(5996) [27-ŞUB-2009 20:57:53:640] nserror: nsres: id=0, op=77, ns=12545, ns2=12560; nt[0]=515,
 nt[1]=1001, nt[2]=0; ora[0]=0, ora[1]=0, ora[2]=0
(5996) [27-ŞUB-2009 20:57:53:640] snsbitts_ts: entry
(5996) [27-ŞUB-2009 20:57:53:640] snsbitts_ts: acquired the bit
(5996) [27-ŞUB-2009 20:57:53:640] snsbitts_ts: normal exit
(5996) [27-ŞUB-2009 20:57:53:640] snsbitcl_ts: entry
(5996) [27-ŞUB-2009 20:57:53:640] snsbitcl_ts: normal exit
(5996) [27-ŞUB-2009 20:57:53:640] nsc2addr: error exit
(5996) [27-ŞUB-2009 20:57:53:640] nsmfr: entry
(5996) [27-ŞUB-2009 20:57:53:640] nsmfr: 318 bytes at 0x15bce20
(5996) [27-ŞUB-2009 20:57:53:640] nsmfr: normal exit
(5996) [27-ŞUB-2009 20:57:53:640] nsmfr: entry
(5996) [27-ŞUB-2009 20:57:53:640] nsmfr: 164 bytes at 0x15b9920
(5996) [27-ŞUB-2009 20:57:53:640] nsmfr: normal exit
(5996) [27-ŞUB-2009 20:57:53:640] nladtrm: entry
(5996) [27-ŞUB-2009 20:57:53:640] nladtrm: exit
(5996) [27-ŞUB-2009 20:57:53:640] nscall: error exit
(5996) [27-ŞUB-2009 20:57:53:640] nioqper:  error from nscall
(5996) [27-ŞUB-2009 20:57:53:640] nioqper:    nr err code: 0
(5996) [27-ŞUB-2009 20:57:53:640] nioqper:    ns main err code: 12545
(5996) [27-ŞUB-2009 20:57:53:640] nioqper:    ns (2)  err code: 12560
(5996) [27-ŞUB-2009 20:57:53:640] nioqper:    nt main err code: 515
(5996) [27-ŞUB-2009 20:57:53:640] nioqper:    nt (2)  err code: 1001
(5996) [27-ŞUB-2009 20:57:53:640] nioqper:    nt OS   err code: 0
(5996) [27-ŞUB-2009 20:57:53:640] niomapnserror: entry
(5996) [27-ŞUB-2009 20:57:53:640] niqme: entry
(5996) [27-ŞUB-2009 20:57:53:640] niqme: reporting NS-12545 error as ORA-12545
(5996) [27-ŞUB-2009 20:57:53:640] niqme: exit
(5996) [27-ŞUB-2009 20:57:53:640] niomapnserror: returning error 12545
(5996) [27-ŞUB-2009 20:57:53:640] niomapnserror: exit
(5996) [27-ŞUB-2009 20:57:53:640] niotns: Couldn't connect, returning 12545

Then, the client got ORA-12545 error.

 Comments   
Comment by ubTools Support [ 27/Feb/09 10:16 PM ]
In this issue, the client was redirected to the less loaded other node, which is not reachable by the remote client.

This is expected behavior as below:

  • According to listener.ora configuration, listener sends IP or hostname back to client.
  • When Load Balancing is in use in RAC environment, request sent to listener may be redirected to other node if other node is less loaded.

For both cases, If listener sends an unreachable IP or hostname, client encounters an error.

Solutions:

  • Change hostname to IP address in listener.ora or add hostname to DNS,"/etc/hosts"-like configuration file in client side.
  • If there are multiple IP addresses in database server, and they are reachable by some group of clients only, then define multiple listeners for each group and allow only 1 listener in load balancing.



Generated at Fri Apr 26 12:07:18 UTC 2024 using JIRA Standard Edition, Version: 3.12.3-#302.