Friday, February 22, 2013

From the Trenches 2 | Patching OSB and SOA Suite to PS5

In my previous blog I talked about a recent upgrade from Oracle Fusion Middleware 11g PS2 to 11g PS5. This blog continues with two post-installation steps that were required to complete the upgrade.

Patching Oracle B2B for ebMS-based services

In this particular project we implemented ebMS-based services using Oracle B2B and Oracle Service Bus 11g. Besides "plain" SOAP Web Services, ebMS is part of the Dutch government standards to exchange messages and expose services between organizations in the public sector. You can read more about implementing ebMS-based services using Oracle B2B, Oracle Service Bus and Oracle SOA Suite in this presentation.

The ebMS standard makes extensive use of SOAP headers to facilitate features such as guaranteed delivery and to avoid duplicate messages. The following snippet shows part of an ebMS message header.

One of the identifiers used in the ebMS message exchange is the manifest id. According to the ebMS specification maintained by OASIS this needs to be XML NCName type. This type has some restrictions; for example that its values cannot start with a number. In Oracle B2B 11g PS2 the manifest id value is prefixed with the text oracle. This prefix is removed in 11g PS5 resulting in the following error from the B2B trading partner at runtime:

09:06:00 Local Listener(8914) Result: "Error" "Fault sent:<SOAP:Fault><faultcode>SOAP:Client</faultcode><faultstring>org.xml.sax.SAXParseException: cvc-datatype-valid.1.2.1: '0A1A1A1A1A1A1A1A1A1A1A1A1A1A1A1A' is not a valid value for 'NCName'.</faultstring></SOAP:Fault>"

Luckily there is an easy-to-apply patch that solves this problem; see article 1497168.1 on Oracle Support. After applying the patch, the manifest id is prefixed with a text value again.

Changing namespaces in WSDLs and XSDs of JAX-WS Web Services

The environment that was patched contains several Java applications running on WebLogic Server. These applications expose Web Services using JAX-WS. A meet-in-the-middle approach was used to create them: the business logic implemented in Stateless Session Beans and JPA (EclipseLink) is integrated with the Java classes generated from the predesigned WSDLs and XSDs.

Depending on the developer that created the Web Service, deployment descriptors such as webservices.xml and weblogic-webservices.xml were added to the application. Descriptors are used for configuration, overriding default settings, and adding metadata. For Web Services this can be the endpoint, port configuration, linkage of the Web Service to EJB components, and so on. When deployed, the WSDL location of Web Services is listed in the WebLogic Console and the WSDL can be retrieved at runtime.

After the patch we noticed that these artifacts weren't identical to the original WSDLs and XSDs. More specifically, the namespaces of the XSD elements in the request and response message definitions were changed to the namespace of the service itself. At runtime however, the service accepted requests and responses as defined by the original contract. This makes it difficult to test these Web Services using clients that inspect the runtime WSDL; for example when creating a default project in soapUI.

This issue was resolved by removing the webservices.xml and weblogic-webservices.xml deployment descriptors from the Java archive and redeploying the Web Services to WebLogic Server. The WSDL that can be retrieved at runtime matches the original designed WSDL again.

Saturday, February 9, 2013

From the Trenches | Patching OSB and SOA Suite to PS5

Recently I was involved in an Oracle Fusion Middleware upgrade from 11g PS2 to 11g PS5 with Jacco Landlust, Aiman Salama, and Jens Peters. The environment that was patched consists of the following domains:

  • Java domain running Java/JEE applications;
  • IDM domain running identity management components including OID;
  • SOA domain running Oracle Service Bus and Oracle SOA Suite. 
The domains consist of several Managed Servers in a single-node cluster configuration. OSB and SOA Suite both run in their own Managed Server.

There are plenty of excellent blogs that discuss the infrastructure and middleware side of such an upgrade. This blogs contains some things we encountered from the application side of things. More specific, the SOA composites and OSB projects.

Rebuild custom Java classes and JAR files in the SOA extension library

You can add custom Java classes and JAR files to SOA Suite that are used by your SOA composites. The SOA extension library for adding extension classes and JARs is available in the $ORACLE_HOME/soa/modules/oracle.soa.ext_11.1.1 directory. For example, an extension can be used to add a custom fault handler.

After applying the patchset you need to rebuild and package the JAR files by running Ant in the $ORACLE_HOME/soa/modules/oracle.soa.ext_11.1.1 directory:

Buildfile: build.xml

     [echo] Creating oracle.soa.ext at $ORACLE_HOME/soa
/modules/oracle.soa.ext_11.1.1/oracle.soa.ext.jar :$ORACLE_HOME/soa
      [jar] Updating jar: $ORACLE_HOME/soa

Total time: 0 seconds 

Reconfiguring OWSM policies

The Oracle Service Bus is used to expose services to the outside world in this environment. These services are secured using Oracle Web Services Manager (OWSM). Services can only be invoked over SSL and the SOAP requests require a WS-Security token for authentication. Among others, these services are invoked by SOA composites that run on SOA Suite and a Silverlight/.NET Portal application. The SOA composites apply the oracle/wss_username_token_over_ssl_client_policy policy to enforce SSL and add the required WSS token.

Due to security integration issues between the Silverlight/.NET Portal and the Oracle Service Bus the service provider doesn't include a timestamp on the transport layer (SSL). The include timestamp setting of the oracle/wss_username_token_over_ssl_client_policy policy was therefore disabled.

After the upgrade we noticed errors in the log files when the SOA composites invoked the Web Services exposed by the OSB: the SSL endpoints of the Web Services could not be reached. Although it seemed the Web Service was actually invoked, and the same SSL endpoint was reachable from a browser. To verify what went wrong we enforced the security/logging assertions of the policy. This causes the outbound SOAP call to be logged to the OWSM logfile: once before applying the policy (without WS-Security tokens) and once after applying the policy (message including the WS-Security tokens). The log showed that the SOAP request contained the correct WSS headers, and that a valid SOAP response was returned by the Web Service. So the Web Service call actually took place. However, the OWSM client policy returned an error that the SSL endpoint was unreachable. It seemed the timestamp setting was activated again by the patch causing a validation error on transport level. Disabling the include timestamp setting fixed the problem.

See Securing Heterogeneous Systems Using Oracle Web Services Manager on Slideshare for more info on this subject.

Using the default SOA Suite XSDs from MDS

SOA Suite supplies out-of-the-box XSDs for interacting with Human Workflow, using Sensors, catch BPEL-specific faults, and so on. These XSDs are preseeded in MDS: both in the MDS schema created by RCU and in JDeveloper after installing the SOA extension in $JDEVELOPER/jdeveloper/integration/seed/soa/shared. Applying the patch will upgrade some of these XSDs. It is therefore important to refer to those XSDs in your SOA composites using MDS. Don't copy and package these XSDs locally inside your composites. Doing so results in errors since SOA Suite detects discrepancies between the packaged SOA Suite XSDs (old patchset), and the same XSDs in MDS (that are updated by applying the patch).

Invoking the TaskService.acquireTask operation 

The Portal application retrieves Human Tasks by invoking the out-of-the-box SOA Suite APIs. When tasks are acquired by users, authentication information needs to be passed to the TaskService.acquireTask operation. A required token is obtained by invoking the TaskQueryService.authenticate operation. The SOAP request that is generated by the Portal includes both the user and onBehalfOfUser as part of the authentication information. While this worked in PS2, in PS5 this resulted in an authentication error indicating that the user had too few privileges to authenticate on behalf of another user. The issue is resolved by removing the onBehalfOfUser element in the SOAP request. Since the SOA Suite APIs are exposed through the OSB in this environment we could alter the request pipeline of the OSB routing flow to delete this element as a temporary solution. A better solution is to upgrade the Portal so the SOAP request doesn't include the onBehalfOfUser element altogether.

Workflow Notification settings

The SOA composites use the Workflow Notification capabilities to send (actionable) emails and email alerts for Human Worfklow. After applying the patch the following error occurs:

<Feb 7, 2013 12:04:38 PM CET> <Warning> <> <BEA-000000> <<.> With the current setting, only Email notifications will be sent; Notifications via voice, SMS or IM will not be sent. If you would like to enable them, please configure corresponding sdpmessaging driver. Then modify the accounts and set NotificationMode attribute to ALL in workflow-notification-config.xml> 
<Feb 7, 2013 12:04:39 PM CET> <Error> <> <BEA-000000> <<.> Incorrect incoming Notification Configuration.
Incorrect Incoming Configuration : AccountName : Default.
The AccountName in workflow-config.xml should be one of the names used in workflow-notification-config.xml.


Incorrect incoming Notification Configuration.
Incorrect Incoming Configuration : AccountName : Default.
The AccountName in workflow-config.xml should be one of the names used in workflow-notification-config.xml.

Reconfigure the Workflow Notification settings in Enterprise Manager to solve this issue.


All in all, the Oracle Fusion Middleware upgrade of all domains was done in a few days. We had some excellent help from Oracle Support for one issue (config.xml with an incorrect Coherence configuration after the patching) that was quickly resolved. Just as any project, good preparation is half the battle. Make sure to read and follow the Oracle patching guides and use the tips & tricks from the available blogs. Be sure to backup the environment, including the OFM database schema's such as MDS and SOAINFRA, and test the upgrade on a production look-a-like test environment. Even if all consoles such as the Service Bus Console and Enterprise Manager work correctly, also inspect the log files for any error. Finally, in case you have longer running SOA composites, make sure you have a testset of open and running composite instance before patching the environment for regression testing.