Oracle SOA DB Poller Error Caused by BINDING.JCA-11608

Hi All,

Just wanted to highlight a common mistake most developer and most testing teams do. When a need arises to change any setting including connection factory or connection pool setting in Oracle Weblogic Server

The Mistake:

1. Make changes to the weblogic server data source / Connection factory.
2. Which leads to a DB adaptor redeploy.
3. Once the DB adaptor is redeployed all the pollers using that DB adaptor incidences will fail with the following error:

Query name: [xxxxxxxxxxxxxx], Descriptor name: [xxxxxxxxxxxxx.xxxxxxx]. Polling the database for events failed on this iteration.

Caused by BINDING.JCA-11608

Connection Already Closed Exception.

This [javax.resource.cci.Connection] is already closed.

This should not happen when running within a packaged application like BPEL or ESB but may occur when the adapter is used standalone.

..

.

  This exception is considered not retriable, likely due to a modelling mistake.  This polling process will shut down, unless the fault is related to processing a particular row, in which case polling will continue but the row will be rejected (faulted).

       at oracle.tip.adapter.db.exceptions.DBResourceException.createNonRetriableException(DBResourceException.java:690)

       at oracle.tip.adapter.db.exceptions.DBResourceException.createEISException(DBResourceException.java:656)

       at oracle.tip.adapter.db.exceptions.DBResourceException.inboundReadException(DBResourceException.java:491)

       at oracle.tip.adapter.db.InboundWork.handleException(InboundWork.java:1001)

       at oracle.tip.adapter.db.InboundWork.runOnce(InboundWork.java:896)

       at oracle.tip.adapter.db.InboundWork.run(InboundWork.java:627)

       at oracle.tip.adapter.db.inbound.InboundWorkWrapper.run(InboundWorkWrapper.java:43)

       at oracle.integration.platform.blocks.executor.WorkManagerExecutor$1.run(WorkManagerExecutor.java:184)

       at weblogic.work.j2ee.J2EEWorkManager$WorkWithListener.run(J2EEWorkManager.java:184)

       at weblogic.work.DaemonWorkThread.run(DaemonWorkThread.java:30)

Caused by: BINDING.JCA-11608

Connection Already Closed Exception.

This [javax.resource.cci.Connection] is already closed.

Note: When the above error occurs, the Poller state is still active, which deadly in a production environment. 

The Corrective Measure:

1. Retire all DB Pollers which have encountered this error.
2. Activate all DB  Pollers.
3. The issue will be fixed.

The Best Practice:

1. Retire all DB Pollers. (No Polling will happen during this time).
2. Make changes to the weblogic server data source / Connection factory.
3. Which leads to a DB adaptor redeploy / Restart.
4. Once the DB adaptor is redeployed / Restarted.
5. Activate all DB Pollers.

Recomendation: Which JDK best suites SUN Java or JRockit

I have always come across this dialama, which JDK between SUN Java (Hotspot) and JRockit  to use when we are installing Oracle Integration Products, i.e, SOA BPEL, BPM, BAM etc.

Here are some of my findings and recomendations:

First we will need to understand What is the difference between the Sun JVM and JRockit JVM?
 
The most well know JVM is the implementation from Sun. The Sun JVM is called HotSpot. The Sun JVM is shipped in the Java Developer's Kit (JDK) and Java Runtime Environment (JRE) from Sun.

The JRockit JVM from is optimized for reliability and performance for server side applications. To achieve this, JRockit JVM uses technologies such as code generation, hot spot detection, code optimization, advanced garbage.

To know more about the differences or futures of JRockit refer Oracle JRockit Frequently Asked Questions
 
Also as JRockit has been the propitiatoryJDK for BEA, I would use JRockit in Production or similar environment.

For developer environment I believe we should be OK to use either or. If you plan to debug or examine your code with Java Mission Control or want extended futures from Flight recorder than undoubtedly JRockit is your JDK.

If you have more to add please comment below.

Chaw.

How to achieve dynamic binding by manipulating endpoint references at runtime - BPEL

Web services and service-oriented architecture (SOA) allow business processes to be easily extended through interaction with other business processes and applications. BPEL processes define this interaction through partner links, which define the interface (messages and operations), transport protocol, and most important, the location of each service to be used.
In most basic process designs, partner links are static; they refer to a single external process selected by the developer at design time. This approach is appropriate for highly targeted or constrained systems. However, in larger systems business processes are more complex. They interact with multiple external services and define multiple partner links, and some of these partner links might not be known at design time. As a result, all potential callouts and logic for deciding which partner links to use must be built inside the business process itself—unnecessarily complicating that process. Furthermore, as additional partner links are added, the resulting process grows more and more unwieldy, as any changes to the partner links require modification of the entire business process.
Fortunately, the BPEL language supports the concept of dynamic binding of partner links

To read more on this topic and to know how: please read further in  SOA Best Practices: The BPEL Cookbook Making BPEL Processes Dynamic

How to configure SQL IDentity Provider for ORACLE BPEL or Worklist application in 11g SOA (Troubleshooting)

If you are looking to setup SQL IDentity Provider  for ORACLE BPEL or Worklist application in 11g SOA then you are in the right place.

The solution is basically provided by an Oracle Sample.

You can download the Oracle Sample here: workflow-120-SQLIdentityProvider.zip

Please follow the steps in the sample provided. Also you can double check the steps with this blog: SQLAuthenticator and Human Worklist Application

Credits to: Marc Kelderman.

By now you have setup the SQL Identiy provider for BPEL and worklist app.

Troubleshooting: (WorklistApp only)

When you have setup the SQL Identiy Provider, there is a very much likely chance that you might face an issue with the user name case sensitivity.

For example if you had a user in mixed case such as  UserName. Then when you try to log in to worklist app, it will complain that the user does not exist, despite that you witness the UserName exist in the database. This will not work for the following reasons.

Human workflow system (Worklist app) appears to be case insensitive for user names login. By default  all user names are stored in lowercase (Not using SQL identity, as this is a different case). As is the case we need to change a property to notify worklist app. 

Solution:

User name case insensitivity property should be changed in Oracle Enterprise Manager Fusion Middleware Control.

Please follow the steps provided in the

Document:  Oracle® Fusion Middleware Developer's Guide for Oracle SOA Suite 11g Release 1 (11.1.1)

section: 29.5.5 Case Sensitivity

Following the steps mentioned in the document and set the value to "true".

This will resolve the issue.

 

How To Generate Encrypted Password For admin.encrypted.password Property In ant-orabpel.properties File

This blog post will provide steps to generate an encrypted password for admin.encrypted.password property in <Oracle Home>/bpel/utilities/ant-orabpel.properties file.

In order to encrypt the password for admin.encrypted.password property in
ant-orabpel.properties file please do the following:

For example your password is "welcome1"

Make sure you are in <Oracle Home>/bpel/bin

Issue the following commands:


[SOA:~/product/10.1.3.1/OracleAS_1/bpel/bin]$ . ./devprompt.sh
[SOA:~/product/10.1.3.1/OracleAS_1/bpel/bin]$ java com.collaxa.cube.util.EncryptPassword welcome1


This will generate the password encrypted like:
AK6qvYcrlNMqnYt1uPZFjw==

JTA Transactions - Local and Global Transaction. And example with Oracle BPEL Application.

This Blog is about understanding the basics of transaction in J2EE level and how this can be translated to Oracle, BPEL conceptually. This does not limit the interpolation of this concept in simple Java projects as well.

Now lets dig in...


Transactions

J2EE supports two kinds of transactions:

• Local Transactions - A local transaction is internal to a single resource.
• Global Transactions - A global transaction is created by an external transaction manager (JTA) and is used to scope work on multiple resources.
  

Local Transactions

When a managed data source is configured for local transactions it returns connections that can participate in local transactions but cannot participate in global transactions. This means that the connections will not be enlisted in global transactions. The data source will set the auto commit to true for retrieved connections. However, it is up to the client to determine how the connections will be used in local transactions. That is, the client can change the auto-commit mode by using setAutoCommit() on a connection.


Local transactions are transactions associated with a particalar data source (means they are resource-specific). the most common example would be a transaction associated with a JDBC connection.

Global Transactions (XA)

When a managed data source is configured for global transactions, it returns connections that can participate in global transactions. A global transaction (also called a distributed transaction) enlists more than one resource in the transaction.

Global Transactions provide the ability to work with multiple transactional resources (typically relational databases and message queues).

Sources:
Oracle® Containers for J2EE Services Guide 10g (10.1.3.1.0)
Question: Difference between local and global transaction ? - AllInterview


Now putting this in SOA, Concept for better understanding:

Diag Source: Oracle® Containers for J2EE Services Guide 10g (10.1.3.1.0)Part Number B28958-01

So with the above image, lets make some assumptions,

hint: BPEL needs a JDBC connection for its dehydration purpose.

The client Tx is the main BPEL process and the new Tx is probably the adapter / another BPEL process invocation process with in the main BPEL process.


In a local transaction setup, client Tx will be a separate transaction and new Tx will be on its own.
In a global transaction setup, new Tx will be a part of or will participate with client Tx.

In another scenario, we can have one client Tx as local and two new Tx as XA. In this scenario the BPEL process will be on tis own and the two new  Tx will be part of one global Tx.

for example this case will apply where you have one BPEL process and you have two or more DB adapters in the BPEL process. Which means, all the DB adapter Tx will roll back if any one of the DB adapters fail with an error.


Additional Info:

Source: Local and global transaction considerations - IBM docs

Local and global transaction considerations

Applications use resources, such as Java Database Connectivity (JDBC) data sources or connection factories, that are configured through the Resources view of the administrative console. How these resources participate in a global transaction depends on the underlying transaction support of the resource provider.

For example, most JDBC providers can provide either XA or non-XA versions of a data source. A non-XA data source can support only resource manager local transactions (RMLT), but an XA data source can support two-phase commit coordination, as well as local transactions.

If an application uses two or more resource providers that support only RMLTs, atomicity cannot be assured because of the one-phase nature of these resources. To ensure atomic behavior, the application should use resources that support XA coordination and should access them within a global transaction.
If an application uses only one RMLT, atomic behavior can be guaranteed by the resource manager, which can be accessed in a local transaction containment (LTC) context.

An application can also access a single resource manager in a global transaction context, even if that resource manager does not support the XA coordination. An application can do this because the application server performs an "only resource optimization" and interacts with the resource manager in a RMLT. In a global transaction context, any attempt to use more than one resource provider that supports only RMLTs causes the global transaction to be rolled back.

At any moment, an instance of an enterprise bean can have work outstanding in either a global transaction context or an LTC context, but not both. An instance of an enterprise bean can change from running in one type of context to the other (in either direction), if all outstanding work in the original context is complete. Any violation of this principle causes an exception to be thrown when the enterprise bean tries to start the new context.

How To Configure a Read Only or Read & Deployment Only or Restricted User For Oracle SOA 11g Enterprise Manager Console

If you  would like to create or configure a user, Who only has read and deploy composites permissions (Restricted Access) in EM console please do the following:

Access to the Enterprise Manager console is determined by the role assigned to the user. See Appendix C.1 Roles and Privileges in:

Document Reference: Oracle® Fusion Middleware Administrator's Guide for Oracle SOA Suite and Oracle Business Process Management Suite 11g Release 1 (11.1.1.4.0) - Part Number E10226-06

Section: C.1.1 Overall Role Functionality Matrix

For a description of the roles defined and the access granted to each role.

Steps to configure the roles described in the document document: (In this case we assign the Monitor role to the user created)

1.  Log in to the Weblogic console.
2. Click on Security Realms -> the name of the realm (myRealm is the default) -> Users and Groups.
3. Click on New to create a new user.
4. Click on the name of the user just created, then Groups and move Monitors from the list on the left to the right. Click Save.
5. Now log in to the Enterprise Manager console with admin privileges, open the SOA folder and right-click on soa-infra and select Security -> Application Roles.
6. Leave the search text empty and click on search button. This will give you a list of all Application Roles.
7. Click on SOAMonitor role to edit it.
8. Scroll down and click on add user. Click on the search button, and a list of available users will be populated. You should see the user just created on the Weblogic console. Move the user to the right and click OK and OK again.
9. Log out, then log in with the user just created.

Additional Info: If you are keen in learning about the roles and their restrictions on weblogic server console please refer:


Document: Oracle® Fusion Middleware Securing Resources Using Roles and Policies for Oracle WebLogic Server 11g Release 1 (10.3.4) - Part Number E13747-04
Section: 6 Users, Groups, And Security Roles