System is in manitance mode. Please try again later OBIEE

When trying to update any reports in OBIEE the below errors appears :-

to solve this issue just follow the screen :-

1- Access to OBIEE Link http://ip-hostname:9704/analytics
2- Choose Administration from the upper right corner.

3- Turn Maintenance mode to off by press the link.

Good Luck.

Cheers
Osama

OID is Down For Ambiguous Reasons

Today one of the customer called complaining about accessing to the system and how its not working ? Since everything is configured as single sign on (SSO) first thing to check was OID or oracle internet Directory.

The logs indicate to the below error :-

[2015-05-01T07:00:18+03:00] [OID] [NOTIFICATION:16] [] [OIDMON] [host: MOAPRDOID1] [pid: 5244] [tid: 0] Guardian: [oidmon]: Unable to connect to database,
will retry again after 10 sec

[2015-05-01T07:00:28+03:00] [OID] [NOTIFICATION:16] [] [OIDMON] [host: MOAPRDOID1] [pid: 5244] [tid: 0] Guardian: Connecting to database, connect string is oiddb

[2015-05-01T07:00:28+03:00] [OID] [NOTIFICATION:16] [] [OIDMON] [host: MOAPRDOID1] [pid: 5244] [tid: 0] Guardian: [gsdsiConnect] ORA-12514, ORA-12514: TNS:listener does not currently know of service requested in connect descriptor.

from the above error the OID is unable to connect to database server i checked database and listener everything is up without any issue and all the other application working without any issues.

after investigation I checked the sqlnet.ora and add the following :-

SQLNET.AUTHENTICATION_SERVICES= (NONE)
NAMES.DIRECTORY_PATH= (TNSNAMES)

Restart the services and everything is working.

maybe this solution will not lead or help in so many cases i am working to know why this will effect on OID since it’s was working without any issues for 1 year

Thanks
Osama 

Pool EDNLocalTxDataSource is Suspended, cannot allocate resources to applications..

Connection test failed with the following exception: weblogic.common.resourcepool.ResourceDisabledException: Pool EDNLocalTxDataSource is Suspended, cannot allocate resources to applications..

This Error Appeared on Weblogic Depend on Managed Server, Tuning Connection Pool By Increasing Maximum Capacity parameter in Datasource.

My Case Datasource EDNLOCALTXDatasource as the above error.

Solution :-

Increase depend on your application.
Thank you 
Osama Mustafa


Oracle SOA Tablespace Is full

When Creating Domain for Oracle SOA it’s Mandatory to Run Oracle RCU before to create SOA Repository, This will create tables configuration and information about Oracle SOA.

But What happened the this tablespace was getting full, increase it 20GB is not enough which mean it’s strange case and not acceptable.

after investigation Table Called “REFERENCE_INSTANCE” include information about Running Composite in SOA ans instances. the size for this table 18GB.

Select * from Reference_instance; 

Check the data in the above query, just in case you can use export data-bump utility for this table, however after this Truncate the table and from full tablespace toonly 2% usage.

Thank you
Osama Mustafa

NP_SOAINFRA.WL_LLR_ADMINSERVER Access Failed

 java.sql.SQLException: JDBC LLR, table verify failed for table ‘NP_SOAINFRA.WL_LLR_ADMINSERVER’, row ‘JDBC LLR Domain//Server’ record had unexpected value

The above error appeared when trying to start Adminserver nothing changed on the Domain

Solution :-

  • Goto  domain location –> config folder.
  •  remove this line from the config.xml file, Take backup before you do this.


    wlsbjmsrpDataSource
    AdminServer,osb_server1
    jdbc/wlsbjmsrpDataSource-jdbc.xml
 

Thank you
Osama Mustafa 

Resource temporarily unavailable in tsStartJavaThread

Resource temporarily unavailable in tsStartJavaThread (lifecycle.c:1096).
Java heap 3G reserved, 3G committed
Paged memory=3145728/36425712K.
Your Java heap size might be set too high.
Try to reduce the Java heap size using -Xmx: (e.g. “-
java.lang.OutOfMemoryError: Resource temporarily unavailable in tsStartJavaThread (lifecycle.c:1096).
Java heap 6G reserved, 6G committed
Paged memory=3145728/36425712K.
Your Java heap size might be set too high.
Try to reduce the Java heap size using -Xmx: 

The Above error appeared on OEDQ when trying to run the process this error related to Java Heap Size the OS control the Heap Size to solve this issue add the below parameters depend on your enviroment  :-

  • Add the following to /etc/sysctl.conf:

kernel.shmmni = 4096
kernel.sem = 256 32000 100 142
net.ipv4.ip_local_port_range = 9000 65500
net.core.rmem_default = 4194304
net.core.rmem_max = 4194304
net.core.wmem_default = 4194304
net.core.wmem_max = 4194304
fs.file-max = 134283264
fs.aio-max-nr = 1048576
vm.min_free_kbytes = 512000 

  • Add the following to /etc/security/limits.conf:

oracle soft nproc 16384
oracle hard nproc 63536
oracle soft nofile 16384
oracle hard nofile 63536

  •  Add the following to /etc/pam.d/login:

session required /lib/security/pam_limits.so

Thank you
Osama Mustafa

java.lang.securityexception access denied. required roles admin

Weblogic Adminserver logs indicated to the below error :-

java.lang.securityexception access denied. required roles admin

To solve the above error, Follow the below steps :-

From the Navigation Tree –> Security Realms :-

New Page Will be Opened click on my Realms :-

From the configuration –> General 

Thank you 
Osama Mustafa

weblogic.socket.MaxMessageSizeExceededException

I recently encountered below error,Adminserver.log Indicate to :-

weblogic.socket.MaxMessageSizeExceededException: Incoming message of size: ‘10000080’ bytes exceeds the configured maximum of: ‘10000000’ bytes for protocol: ‘t3’.
weblogic.socket.MaxMessageSizeExceededException: Incoming message of size: ‘10000080’ bytes exceeds the configured maximum of: ‘10000000’ bytes for protocol: ‘t3’
        at weblogic.socket.BaseAbstractMuxableSocket.incrementBufferOffset(BaseAbstractMuxableSocket.java:230)
        at weblogic.rjvm.t3.MuxableSocketT3.incrementBufferOffset(MuxableSocketT3.java:351)
        at weblogic.socket.SocketMuxer.readFromSocket(SocketMuxer.java:989)
        at weblogic.socket.SocketMuxer.readReadySocketOnce(SocketMuxer.java:920)
        at weblogic.socket.SocketMuxer.readReadySocket(SocketMuxer.java:897)
        at weblogic.socket.EPollSocketMuxer.dataReceived(EPollSocketMuxer.java:215)
        at weblogic.socket.EPollSocketMuxer.processSockets(EPollSocketMuxer.java:177)
        at weblogic.socket.SocketReaderRequest.run(SocketReaderRequest.java:29)
        at weblogic.socket.SocketReaderRequest.execute(SocketReaderRequest.java:43)
        at weblogic.kernel.ExecuteThread.execute(ExecuteThread.java:145)
        at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:117)

to fix this error follow the following steps :-

From Environment, Press on Adminserver  :-

Chose the tab like the below picture :-

Add the following parameter in arguments box, Save usually there is no need to restart :- 
Or if you would like change MaxMessageSize Directly from the console follow the below screen :-

Option #2 :- 
From the navigation tree :- 

Click on Module-FMWDFW

After choose the above tab, 3 options will be appear, press on each of them and Uncheck on Enable Watch  :-

Increase Transaction timeout in SOA Server

I received the below error in soa_server1 :-

Transaction Rolledback.: weblogic.transaction.internal.TimedOutException: Transaction timed out after 603 seconds BEA1-3FB6FB464035C13E3227 at 

From the above error, transaction timeout should be increased regarding to heavy transaction on soa server.

to do this follow the following steps :-

access weblogic console –> Deployment –> and search for Soa_infra. ( expand it)

Check the below EJBS services and click on it configuration tab :-
BPELActivityManagerBean
BPELEngineBean
BPELInstanceManagerBean
BPELProcessManagerBean
BPELServerManagerBean

If this not working on you have to increase transaction timeout for JTA, Access Weblogic Console select Services > JTA
Thank you 
Osama Mustafa