Oracle BPM Suite 12c and SOA Suite 12c at eProseed World 2014

As most of you know, Oracle officially released SOA Suite 12c (12.1.3) and BPM Suite 12c (12.1.3) a couple of week ago. Exciting times for people like us, who have worked extensively with BPEL Process Manager, Oracle ESB, SOA Suite 10g, and SOA Suite 11g! The same week that 12c was released, eProseed World 2014 took place. During this event all eProseed employees meet for a fun couple of days full of social and work-related sessions. Because Vennster is merging into the eProseed Netherlands office we will soon migrate this blog to an eProseed domain. For us it was an excellent chance to meet all our new colleagues. In between the tour of Luxembourg city and an afternoon full of Highland Games (nobody got injured :-), multiple sessions where organized to present and discuss Oracle BPM and SOA 12c.

eProseed World 2014 - Update on Oracle SOA Suite 12c from Ronald van Luttikhuizen

The SOA Suite 12c presentation included an overview of 10 years of Oracle & SOA, new features and important themes in Oracle SOA Suite 12c, and information on the migration tool bundled with SOA Suite 12c to perform in-flight upgrade from 11g to 12c.

Nordic OTN Tour 2013

This year I am part of the team that is presenting at the Nordic OTN Tour 2013. It covers three countries: Sweden, Denmark and Norway and is organized by the local user groups. Tim Hall, Mike Dietrich, Sten Vesterli and me are presenting on Database and Middleware in all countries.

Sweden, October 22nd

Today we presented in Stockholm. The program can be found on their website.

It was an interesting day, both from a Middleware perspective and from a Database perspective. The user group decided to plan two parallel tracks; the sessions about Middleware in the morning and Database sessions in the afternoon. Because of this, the Middleware sessions were competing with each other and the Database sessions were competing with each other. From that perspective it would have been nicer to have a Middleware track and a Database track running in parallel. The advantage of the Orcan approach however, is that Database people will attend the Middleware sessions that they might otherwise have skipped.

Denmark, October 23rd

The programs are not exactly the same in all three countries. In Denmark there are three parallel tracks, and instead of talking about using Oracle Fusion Middleware 11g to realize a SOA, I will talk about Oracle BPM Suite. The program can be found on the website of the Danish user group. Apart from the people that were at the Swedish day, there are a number of other speakers in Copenhagen:

• Rasmus Knappe
• Jørgen Christian Olsen
• Gordon Flemming
• Lars Bo Vanting 

Norway, October 24th

The last day is in Oslo. In Norway there are two parallel tracks, like in Sweden. I will do the same presentations: Overview of Oracle SOA Suite 11g and Creating SOA with Oracle Fusion Middleware 11g.

The same team is doing the presentations. In addition there are a few Norwegian speakers as well:

• Trond Brenna
• Harald Eri and Bjørn Dag Johansen

The agenda is published here. 

All in all a very interesting tour, I look forward to meeting the different user groups and spend time with the other ACE Director and talk about Oracle stuff all day ;)

Article published | Fault Handling and Prevention (II)

Oracle Technology Network (OTN) published the article Fault Handling and Prevention - Part 2 (Fault Handling and Prevention for Services in Oracle Service Bus) by Ronald van Luttikhuizen and Guido Schmutz.


Part 1 of this article series on Fault Handling and Prevention discussed what fault handling is and why it is important. It also addressed the specific challenges in handling faults in a service-oriented landscape as compared to traditional systems. Part 1 concluded by presenting a sample scenario, an Order process implemented in a BPM and SOA environment, discussed potential pitfalls, and described generic fault prevention and recovery patterns.

Part 2 concentrates on concrete fault handling and prevention measures in the integration layer that are realized through Oracle Service Bus (OSB). The integration layer covers typical elements and integration functionality, such as Adapters for connectivity to back-end systems, Routing, Transformation, and Filtering.

More resources

• Fault Handling Slides and Q&A, Blog, Vennster

• Service Oriented Architecture: An Integration Blueprint, Book, Packt Publishing

• Patterns for Fault Tolerant Software, Book, Wiley Software Patterns Series

• Oracle Fusion Middleware Developer's Guide for Oracle SOA Suite, Oracle Technology Network
• SOA Made Simple, Book, Packt Publishing

Stay tuned for parts III and IV of the article! 

About the authors

Ronald van Luttikhuizen is Managing Partner and Architect with Vennster and an Oracle ACE Director
Guido Schmutz is Technology Manager for SOA and Emerging Trends with Trivadis and an Oracle ACE Director

SOA Made Simple

The book SOA Made Simple is published! SOA Made Simple is written by Vennster's Lonneke Dikmans and Ronald van Luttikhuizen. You can download a sample chapter and order the book from the Packt Publishing website. 

SOA Made Simple is a concise and indispensable handbook for finally understanding exactly what Service Oriented Architecture is. Split into three clear sections, in this book you’ll learn from both theory as well as step-by-step implementation examples to aid in your understanding of this often poorly- articulated industry term.

A short abstract for SOA Made Simple:

SOA is an industry term which is often preached like a religion rather than taught like a technology, and over time, grasping the concept has become unnecessarily difficult. Many companies proclaim that they don’t know where to begin with SOA, while others have begun their SOA effort but haven’t reaped the benefits they were convinced it would bring. “SOA Made Simple” explains what SOA is in simple terminology and by using real-life examples. Service-orientation is already a very natural way of thinking for business stakeholders that want to realize and sell services to potential clients, and this book helps you to realize that concept both in theory and practice. After reading “SOA Made Simple” you will have a clear understanding ofwhat SOA is so you can implement and govern SOA in your own organization. If you are an architect who wants to be completely clear in your understanding of what SOA is, then this book is essential. In fact, anyone (designer, developer, administrator or team lead) who is implementing or about to implement an architecture in an IT environment should not miss out on “SOA Made Simple”. 

"Start Small, Grow Fast" in Top 10 articles for architects (2012)

The whitepaper Start Small, Grow Fast by Demed L’Her, Edwin Biemond, and Ronald van Luttikhuizen ranks 3rd in the Top 10 most popular OTN articles for architects (2012). The full Top 10 including links to these articles is listed here.

The whitepaper explains a set of pragmatic best practices for deploying a simple and sound SOA footprint that can grow with business demand. The paper contains details about administrative considerations, infrastructure considerations, development considerations, and architectural considerations. 

You can find a companion checklist that summarizes the Oracle SOA Suite best practices in a previous blog.

Eventing Hello World

This week I presented the "Introduction in Eventing in Oracle SOA Suite 11g" session at the DOAG conference in Nürnberg. I used several demos in this session to show the eventing capabilities of SOA Suite. This blog contains the source code and accompanying explanation so you can replay the demo yourself.

Introduction
An event is the occurrence of something relevant, signals a change in state that might require an action. Examples of events are: an invoice that has been paid, a customer that moved to a new address, a new purchase order, and so on. Events are complimentary to processes and services: processes and services describe what should be done, events about when something important occurs. SOA is not only about (synchronous) services and processes (what); but also about events (when). Eventing improves decoupling in your SOA landscape.

Slides
The presentation slides can be viewed and downloaded from my SlideShare account. The presentation introduces the concept of eventing, discusses some basic eventing patterns, and talks about the implementation of eventing in SOA Suite using AQ, JMS, and EDN.

DOAG 2012 - Introduction in Eventing in SOA Suite 11g from Ronald van Luttikhuizen

Code
You can download a zipfile containing the JDeveloper workspaces for the SOA Composites (DOAG2012_Eventing_SOASuite) and Java projects (DOAG2012_Eventing_Java) used in the demo here.

Prerequisites
You'll need an Oracle SOA Suite 11g runtime to deploy and run the SOA Composites. Oracle provides a pre-built SOA and BPM virtual machine for testing purposes that is easy to install.

Download JDeveloper 11g (11.1.1.6) or later from Oracle Technology Network to inspect and modify the SOA Composite and Java projects.

Setup
In order for the demos to run some plumbing needs to be done for AQ and JMS:

• The configuration steps required for the AQ demos are listed in the aq_configuration.txt file that is part of the Queuing_Utilities JDeveloper project.
• The configuration steps required for the JMS demos are listed in the jms_configuration.txt file that is part of the Queuing_Utilities JDeveloper project.
• In JDeveloper modify the existing Database Connection to point to your DOAG12_JMSUSER database schema.
• In JDeveloper modify the GenerateInvoice File Adapter of the Billing SOA Composite to write the invoice file to an existing directory of the SOA Suite runtime.
• In JDeveloper modify the WriteSensorToFile File Adapter of the ProcessSensors SOA Composite to write sensor values to an existing directory of the SOA Suite runtime.
• Use Enterprise Manager to create a partition in SOA Suite called doag2012_eventing.
• Deploy the following SOA Composites to the doag2012_eventing partition of the SOA Suite runtime using JDeveloper, Enterprise Manager or scripts: Order2Cash, Dunning, Billing, and ProcessSensors.
• Deploy the Java application CRM to the WebLogic Server.

Advanced Queuing (AQ)
The first demo shows the queuing capabilities of AQ and shows how to integrate SOA Composites with AQ queues:

1. Insert a record in the ORDERS table in the DOAG12_WEBSHOP schema. 
2. The BIT_ENQUEUE_NEW_ORDER_EVENT trigger will execute the ENQUEUE_NEW_ORDER_EVENT procedure that enqueues a message to the NEW_ORDER_QUEUE. 
3. You should see a new event in the NEW_ORDER_QT queue table of the DOAG12_JMSUSER schema.
4. If the Order2Cash SOA Composite is deployed you should see a new instance in Enterprise Manager. This process is started by receiving events from the NEW_ORDER_QT queue using an AQ Resource Adapter.

Java Message Service (JMS)
The next demo shows the publish/subscribe capabilities of JMS and shows how to integrate SOA Composites with JMS:

1. Complete the Book Order Human Task of the Order2Cash process instance that was started in the previous step.
2. The Order2Cash instance will use a JMS Adapter to publish an event to the JMS topic DOAG12_BillingTopic. You can inspect the topic in WebLogic Console and see that an event has been published. There are two subscribers to the JMS topic: the Billing SOA Composite (using a JMS Adapter) and the CRM Java application (using a Message-Driven Bean or MDB).
3. In Enterprise Manager you should see a new instance of the Billing SOA Composite that is started through the JMS Adapter. The instance should have written an invoice file using the File Adapter.
4. In the SOA Suite log file (e.g. user_projects/domains/[domain]/servers/[server]/logs) you should see a log statement produced by the CRM application that indicates that the event is received.

Event Delivery Network (EDN)
This demo shows the publish/subscribe capabilities of EDN and shows how to use EDN from SOA Composites:

1. Wait for 5 minutes after the billing event has been published from the Order2Cash process instance using JMS. The OnAlarm branch will be activated and publish a DunningEvent using a Mediator component.
2. The Dunning SOA Composite is subscribed to the DunningEvent, a new instance of this SOA Composite will be started and can be inspected from Enterprise Manager.

A second usage of EDN is demonstrated using the following steps:

1. Start a new instance of the Order2Cash process by inserting a new order in the database. Complete the Human Task again. Only this time use Enterprise Manager to fire a PaymentEvent to EDN. This can be done by right-clicking the soa-infra node and selecting Business Events. Select the PaymentEvent and hit the Test button. You can use the PaymentExample.xml file from the Order2Cash JDeveloper project as example event payload. 
2. Using correlation the running Order2Cash process instance will receive an in-flight event and will continue without starting the Dunning process.

Sensors and Composite Sensors
The last demo shows the sensor and composite sensor capabilities of SOA Suite. Composite sensors are used to add metadata to running instances so they are easier to find in Enterprise Manager or via the Java APIs of SOA Suite:

1. Inspect the composite sensors of the Order2Cash SOA Composite in JDeveloper.
2. In Enterprise Manager navigate to the instances of the Order2Cash SOA Composite. Use the Add Fields button to search for instances based on functional data such as customer name and order id. 

Alternatively, composite sensors can also be published using JMS.

Sensors and monitoring objects can be used to feed runtime data of BPEL component instances into Oracle BAM, or publish this data to JMS, Database, or AQ in a non-intrusive way. Sensors are based on configuration instead of adding additional activities and components to your SOA Composites:

1. Inspect the sensor configuration of the Dunning BPEL component in JDeveloper. These sensors publish to the DOAG12_SensorQueue queue.
2. In Enterprise Manager navigate to the instances of the ProcessSensors SOA Composite. You should see new instances that are started based on the published sensor events.

Conclusion
The presentation and the demos in this blog give you an overview of the eventing capabilities of SOA Suite. The demo is stuffed with different eventing techniques (AQ, JMS, EDN), this should not be considered a best-practice ;-) Analyze what the best eventing implementation is for your specific scenario.

Some best-practices:

• Model events in your BPM and SOA projects;
• Use events to notify running processes;
• Expand the service registry to include events;
• Use events for additional decoupling between processes and services;
• Events is not just queuing, also consider publish/subscribe and event stream processing;
• There is not a single best technology for eventing implementation in SOA Suite.

DOAG 2012

This week the German Oracle User Group, or DOAG as it is called in German, held their yearly conference. Like other years, the location was the conference center in Nuremberg, a beautiful city in the south.

Vennster was well represented in the SOA/BPM Space, we did the following sessions:

• SOA Made Simple: service design (Ronald van Luttikhuizen)
• SOA Made Simple: creating a roadmap for your SOA (Lonneke Dikmans) 
• Effective Fault Handling in Oracle SOA Suite 11g (Ronald van Luttikhuizen)
• Introduction in Eventing in SOA Suite 11g (Ronald van Luttikhuizen)
• Using the B2B Adapter in a Dutch government project (Ronald van Luttikhuizen)
• Securing heterogeneous systems using Oracle WebServices Manager (Ronald van Luttikhuizen and Jens Peters)
• Deployment in Oracle SOA Suite and Oracle BPM Suite (Lonneke Dikmans)
• Stop generating your User Interface! Start designing it (Lonneke Dikmans)

You can find the slides by Ronald and me on slideshare:

• http://www.slideshare.net/lonneked
• http://www.slideshare.net/rluttikhuizen

Of course there were also other presentations by other presenters ;) DOAG is a big conference, with over 400 presentations. Most of them cover cases, others explain the latest developments. There is a number of tracks that are of interest if you are working in the 'middleware space': BPM, Middleware & SOA, development, Java and Strategy and Business.  The English spoken sessions are not as popular as the German language sessions, but both are well visited. 

I visited three sessions, one case study titled "Dynamische Benutzer-Workflows mit SOA und BPM-Suite" by Arne Brüning, one about the new developments in EclipseLink called "The Evolution of Java Persistence" by Doug Clarke and the last one was a session titled "NoSQL and SQL: Blending the Best of Both Worlds" by Andrew Morgan. All three happened to be presented by Oracle. They were very different in nature. The workflow session discussed a customer case. It was interesting from that point of view. I would have preferred more technical depth, but the presenter was well prepared and had an interesting story to tell. The session by Doug about Eclipse gave a nice overview of the latest developments and put them into perspective of the history of TopLink and EclipseLink. I think that this is a good strategy: it shows that EclipseLink is both proven and modern: it has been around for years and part of the original team is still working there PLUS they have solutions for new developments like JSON, REST services, NoSQL and multi-tenancy. The final presentation was an example how not to do that. The presenter put NoSQL in the title in an attempt to attract a crowd. But the session was really about MySQL clusters. A lot of people left the session while he was talking, because it was completely off topic. The presentation itself was not bad, but the title was misleading.  

Unfortunately I did not have time to see more sessions, because of all the presentations we were doing ourselves. There certainly was a lot more I would have liked to listen to and I hope we will be back next year!

Compensation Hello World

There were several questions about compensation after the Effective Fault Handling in SOA Suite 11g session that Guido Schmutz and I presented at Oracle OpenWorld last week. The presentation included a live demo that showed how compensation can be used to undo previously executed actions in processes and services that are implemented in SOA Suite. Based on the questions, this blog explains the concepts of compensation, gives you an overview of the demo, and provides a download link to the JDeveloper demo project so you can play around with the demo yourself.

Concept

Process and service activities can be divided into unit-of-works, called transactions that are either executed (i.e. committed) as a whole or undone as a whole (i.e. rollbacked) due to some error or an explicit rollback command. When the activities within the same unit-of-work span multiple resources (databases, Java components, JMS topics, and so on) these resources all need to support global (XA) transactions and need to enlist in the global transaction, to support commits and rollbacks on the transaction as a whole.

It is not always possible to use transactions as a means to commit, or rollback related activities as a whole. For example when the invoked services do not support global transactions. Next to this, long running processes execute several (implicit) commits since transactions shouldn't be kept open too long: this degrades performance and causes completed activities to remain invisible to others. Once a commit has been executed, the committed activities cannot be rollbacked. Consider an Order-to-Process in which we book the customer's credit card in advance and deliver the ordered goods a few days later. The process will dehydrate at some point causing a commit on the active transaction. If delivery of the ordered goods fails we cannot rollback the credit card booking.

Compensation provides a mechanism to undo already committed activities by means of invoking opposite activities (i.e. compensation) in reverse order. For example booking the opposite amount of money to the credit card that was initially charged. SOA Suite provides the following activities to support compensation:

• Compensation handler. Compensation handlers contain the activities that need to be executed as part of the compensation flow. These handlers are defined per scope, similar to catch blocks. Per scope you need decide if you need a compensation handler. 
• Compensate activity. The activity that triggers compensation for a SOA Composite. Executing this activity will cause the invocation of compensation handlers for all successfully completed scopes that have defined a handler, and are not yet compensated. Only compensation handlers of scopes that are directly enclosed by the scope that contains the compensate activity will be executed. The handlers are executed in reverse order, so the handler of the last completed scope is executed first. 

Before we jump to the demo some considerations when using compensation:

• Compensate activities can only be executed from catch blocks and compensation handlers.
• Compensation activities either trigger compensation for all enclosed and completed scopes using the compensate activity (supported in BPEL 1.1 and 2.0), or can trigger compensation for one specific scope using the compensateScope activity (only BPEL 2.0).
• Compensation handlers can only be defined on scope level, not on sequence level.

Demo

The demo consists of a SOA composite that implements a simplified Order-to-Cash process. The composite contains a single, asynchronous BPEL component called CompensationExample. After receiving the input message the process executes the following scopes and sequences:

• Scope Register_Order. This scope simulates the registration of the new order using an Empty activity called Dummy_Register_Order. The scope defines a compensation handler that simulates an undo activity by updating the order with a new status (e.g. cancelled).
• Scope Invoke_CreditCardService. This scope simulates the registration of the new order using an Empty activity called Dummy_Book_CreditCard. The scope defines a compensation handler that simulates an undo activity by charging the opposite amount to the credit card.
• Sequence Throw_Fault. This sequence is added for demonstration purposes and throws a fault in case "compensate" is passed as input to the SOA composite; otherwise the process will complete normally. The fault is caught by the Catch_All sequence.
• Sequence Catch_All. This sequence includes a Wait activity before triggering compensation using a Compensate activity. The wait enables you to inspect the process instance in Enterprise Manager before the compensation logic is executed. After the compensation is completed, the process will send a callback message to the client indicating a fault occurred for this process instance.

Process flow of the Compensation Hello World example

After the wait has finished the compensation handler of the Invoke_CreditCardService is executed. Finally, the compensation of the Register_Order scope is executed. Again, by including Wait activities you can inspect the sequence of events in Enterprise Manager.

The flow of events is shown in the following figures:

Process instance after fault is thrown, but before compensation is triggered

Process instance after compensation is triggered and the compensation handler for the Invoke_CreditCard_Scope is completed. The compensation handler for the  Register_Order scope is still active.

Process instance after all compensation handlers have been executed.

Process instance after compensation has finished and the callback message has been send to the client invoking the process.

Code

You can download the JDeveloper project for the Compensation Hello World example here.

Real-life

In real-life projects you wouldn't add Throw activities to your SOA composites to trigger compensation and test your compensation logic. In such cases you can best use the out-of-the-box SOA Suite Test Framework and BPEL Test Framework to simulate errors (e.g. unreachable services) in unit tests to inspect and test the compensation logic of your SOA composites.

Presentations at OpenWorld 2012

This blog contains a wrap up of our presentations at OpenWorld 2012.

Oracle Fusion Middleware Live Application Development (UGF10464)

In this three hour show moderated by Duncan Mills and Chris Muir, the audience could experience the dynamics between three different teams that are building an application based on Oracle Fusion Middleware:

1. User Interface in ADF;
2. Services in SOA Suite and Oracle Database; 
3. Business processes in BPM Suite.

Behind the scenes at the presentation

It's the fourth time Vennster participated in the Live FMW Development sessions after appearances at ODTUG Kaleidoscope, UKOUG, and OBUG. For OpenWorld 2012 the team that prepared the application consisted of Lucas Jellema, Luc Bors, Aino Andriessen, Guido Schmutz, Lonneke Dikmans, and Ronald van Luttikhuizen. This time we tried a different approach in which we pre-built the application and focused on explaining and demoing it in the first part of the session. After that we made several changes and deployed the improved software components.

OpenWorld 2012 - Oracle Fusion Middleware Live Application Development from Ronald van Luttikhuizen

Some best-practices the team discussed:

• Use Business Rules to allow for runtime modification of fast changing business logic instead of design time modifications and redeployment of services. Encapsulate useful Business Rules as separate services instead of adding them to existing SCA composites.
• Include a heartbeat operation for every Web Service (e.g. by using the Mediator's Echo activity) so you can verify that all technical layers of the Web Service work without triggering a functional side effect. 
• Invoke PL/SQL from DB Adapters instead of directly executing CRUD operations for additional decoupling.
• Decouple components and introduce additional reliability and robustness by using events.

Effective fault handling in SOA Suite 11g (CON4832)

In this co-presentation with Guido Schmutz we explored how the out-of-the-box frameworks, patterns, and tools that are available in Oracle Service Bus and Oracle SOA Suite can help you to implement fault prevention and handling capabilities.

360 view during the Fault Handling presentation

The session was pretty well attended, and with an extended Q&A sessions at the end. I never had so many questions after a presentation; don't know if that's a compliment or not ;-) Some of the questions raised:

• Wrapping asynchronous message exchanges as synchronous exchanges and vice versa. 
• Where to execute long-running and statefull processes: SOA Suite rather than OSB.
• Fault handling in fire-and-forget message exchanges: if there's no callback, implement fault handling in the service that is being called.
• Can the Fault Management Framework of SOA Suite be used to catch internal BPEL faults: use catch activities for that purpose.
• Transaction boundaries, dehydration points, and global transaction timeouts.
• Compensation versus rollbacks.
• Chaining exception policies using the Fault Management Framework.

The slides are available from Slideshare and answer some of these questions. A series of articles is underway that dives deeper into these subjects!

Effective Fault Handling in Oracle SOA Suite and Oracle Service Bus (OpenWorld 2012) from Ronald van Luttikhuizen

Running EclipseLink DBWS 2.4.0 on GlassFish 3.1.2

In a previous blog I explained how to create a web service using the EclipseLink DBWS utility. This blog explains how to deploy this to GlassFish 3.1.2.

The main challenge is getting the right classes to load. But let's start from the beginning: create the war file.

Create the war file

First we create the war file with the DBWSBuilder utility. I defined a builder file on the employees table in the standard hr schema as follows:

<?xml version="1.0" encoding="UTF-8"?>

<dbws-builder xmlns:xsd="http://www.w3.org/2001/XMLSchema">

<properties>

<property name="projectName">Employee</property>

<property name="driver">oracle.jdbc.OracleDriver</property>

<property name="password">hr</property>

<property name="url">jdbc:oracle:thin:@localhost:1521:ORCL</property>

<property name="username">hr</property>

<property name="platformClassname">org.eclipse.persistence.platform.database.oracle.Oracle11Platform</property>

<property name="logLevel">all</property> 

</properties>

<table schemaPattern="%" tableNamePattern="EMPLOYEES"/>

</dbws-builder>

Second, I created the war file with the DBWS utility:

$ECLIPSELINK_HOME/utils/dbws/dbwsbuilder.sh -builderFile EmployeeTableBuilder.xml -stageDir . -packageAs glassfish employee.war

NB: don't forget to set the paths in setenv.sh in $ECLIPSELINK_HOME/utils/dbws/ directory.

Add libraries to the war file

The war file that is created assumes that the correct version of the  eclipselink.jar is on the classpath of the server.  So first we add two libraries to the WEB-INF directory:

• eclipselink.jar
• javax.persistence_2.0.4.v201112161009.jar

I would have liked it a lot better if I would have had the option in the DBWSBuilder utility to set a flag to include the eclipselink.jar and the persistence jar in the WEB-INF/lib directory, or to set a flag to tell it not to do that. Similar to the 'provided' property that Maven offers.

If you forget this step, there will be an exception in your $GLASSFISH_DOMAIN/logs/server.log file stating:

javax.enterprise.webservices.org.glassfish.webservices|_ThreadID=21;_ThreadName=Thread-2;|Deployment failed
java.lang.TypeNotPresentException: Type ProviderHelper not present

Add libraries to your domain

The JDBC driver needs to be added to the lib/ext directory of your glassfish domain. 

cp $ORACLE_HOME/jdbc/lib/ojdbc6.jar $GLASSFISH_DOMAIN/lib/ext

If you skip this step, or put the jar in the WEB-INF lib, you will get a SOAP Fault that is caused by a ClassNotFoundException:

com.sun.enterprise.container.common.spi.util.InjectionException: Error creating managed object for class: class _dbws.DBWSProvider

....

...

Caused by: java.lang.NoClassDefFoundError: oracle/sql/TIMESTAMP

at org.eclipse.persistence.platform.database.oracle.Oracle9Platform.<clinit>(Oracle9Platform.java:109)

Test the service

Glassfish offers a tester for webservices. This does not work, because the container does not recognize the webservice. You get the following (incorrect) error:

Service {urn:EmployeeService}EmployeeService looks like a JAX-RPC based WebService.

Please note that the tester feature is supported for JAXWS based webservices only

Creating a project based on the deployed wsdl does work as expected.

Live Fusion Middleware Development Session @ OBUG Connect 2012

The MECC in Maastricht will host the 5th edition of the OBUG (Oracle Benelux User Group) Connect conference on tuesday the 24th of April. This conference will be the third time that a group of Fusion Middleware specialists from different companies such as AMIS Services, Vennster, Veriton, and Oracle Consulting Services team up to present a live application development session after successful performances at ODTUG Kaleidoscope 2011 and UKOUG 2011. The goal for these events is to build and demo a working application in just half a day using JDeveloper, ADF, Oracle Service Bus, Oracle SOA Suite, and Oracle BPM Studio. At the upcoming OBUG Connect 2012 conference the team will consist of Steven Davelaar, Lucas Jellema, Edwin Biemond, Luc Bors, Lonneke Dikmans, and Ronald van Luttikhuizen.

Attendees can walk in and out of the session, ask questions, interact and comment on the way developers build the application, and see how Fusion Middleware can be used to build an application based on SOA and BPM principles in a productive and iterative way. Thanks to Murphy and his law, at each development session unexpected things happen that need to be dealt with by the development team and session moderators in a short time frame. So come and see!

FMW development session at UKOUG 2011

Read more about the OBUG Benelux Connect 2012 program here.

Start Small, Grow Fast | Companion Checklist

A companion checklist that summarizes the Oracle SOA Suite best practices contained in the "Start Small, Grow Fast" Oracle whitepaper by Demed L'Her, Edwin Biemond, and Ronald van Luttikhuizen at bit.ly/soa-start-small.

Administrative considerations

• Involve operations teams from the beginning.
• Make sure your organization has dedicated middleware administrators.
• Prefer scripting for deployment of composites.
• Use SOA Suite configuration plans to capture all environment-specific information instead of hardcoding them in your composites.
• Avoid creating a separate configuration plan per composite per environment - consolidate these to a few or a single configuration plan per environment.
• Script as many configuration tasks as possible from the beginning.
• Document configurations in a well-known central location such as a team Wiki, readily available to all stakeholders.
• Consider using NFS shares for JCA adapter configuration plans.
• Use partitions to categorize composites and execute various tasks for multiple composites at once.
• Use composite sensors to enable the search of specific instances.
• Negative testing is an absolute requirement.

Infrastructure considerations

• Build a cluster from the start -even if only with a single node. Think of the following when configuring a cluster:
• Use individual IP addresses for all components (WebLogic Admin Server, Node Manager, and the Managed Servers).
• Use the JRockit JVM and Mission Control to detect problems and perform tuning.
• Leverage WebLogic channels to separate cluster traffic from production traffic.
• Use a load balancer to divide traffic between services and detect outages of servers in the cluster.
• Think about your domains. There is no such thing as a “one-size fits all” domain topology; however there are a few tips and guidelines in this area:
• Always setup Node Manager.
• Do not combine Admin and Managed Servers; except maybe in non-critical environments such as development.
• Deploy your software and services to Managed Servers.
• Leverage domains to partition environments with different lifecycles or significant functional differences.
• Dedicate the most performant hardware to Managed Servers.
• Linux x86 is a great entry-level platform for Oracle SOA Suite.
• Favor Oracle Database as infrastructure database.
• Perform realistic load tests.
• Define and test a purging procedure well before going live.

Design-time considerations

• Use MDS to centrally store artifacts and avoid duplication.
• Have a canonical model for your core objects to ensure future re-use and consistency.
• Consider taking a contract-first or meet-in-the-middle approach for interfaces exposed to the outside world.
• Formulate and adhere to a small set of service design guidelines and naming conventions:
• Language conventions
• Naming conventions for composites, services, references and components
• Naming conventions for BPEL and BPM components and activities
• Naming conventions for composite sensors
• Wrap frequently used SOA Suite APIs in simpler custom APIs.
• Use Domain-Value Maps (DVM) and Business Rules to improve flexibility and agility.
• Use fault policy files to separate exception and fault handling from “normal” process logic. Centrally store them in MDS to enable re-use across composites.
• Unit test your composites using SOA Suite’s test framework as you would unit test your Java code using e.g. JUnit.

Architectural considerations

• Use a simple and concrete service categorization.
• Spend some time thinking about the granularity of your services to avoid too frequent refactoring. Reusability is a key factor. Other factors are rate of change, availability, and ownership.
• Building asynchronicity through business events or messaging will improve loose-coupling, ease of deployment and administration and enable throttling.
• Consider supporting multiple versions of the same service in production to allow service consumers to upgrade at their own pace.
• SOA governance is needed but can start very simply with a wiki. Consider a full-fledge repository as you grown your SOA efforts.

Using SOA Gateway in EBS 12

A lot of organizations use a packaged application for their supporting processes, like Human Resources and Finances. Often you need to integrate with these systems. There are several use cases for this:

• Use data from the HR system to determine hierarchy;
• Use functionality of an ERP module in another process;
• Expose services for self service purposes.

Oracle E-Business suite has built-in support for these types of integration. In this post we will look at an example of the third use case: a self service application for expense reports, that stores the result (a pay element) in the E-Business Suite.
There is a lot of documentation available, so far I have used:

1. Oracle® Integration Repository User's Guide
2. Oracle® E-Business Suite Integrated SOA Gateway Developer's Guide
3. Oracle® E-Business Suite Integrated SOA Gateway User's Guide
4. Integrated SOA Gateway Implementation Guide
5. Oracle® Application Server Adapters for Files, FTP, Databases, and Enterprise Messaging User's Guide
6. Oracle Human Resources Management Systems Workforce Sourcing, Deployment, and Talent Management Guide

This can make it rather overwhelming (to put it mildly ;) ) This blog gives an overview of important steps and the documents that contain information about these steps.

Preparation
Like any other feature in the Oracle EBS, you need to have the right privileges to use the SOA Gateway.

Therefore, you have to create a user that has the correct permissions to generate and deploy a web service. The following table can be found in theOracle® E-Business Suite Integrated SOA Gateway User's Guide:

Privileges	System Integration Analyst	System Integration Developer	Integration Repository Administrator
View public interfaces	Yes	Yes	Yes
View private/internal interfaces	No	Yes	Yes
Generate/regenerate Web Services (WSDL)	No	No	Yes
Deploy/redeploy Web Services	No	No	Yes
Undeploy Web Services	No	No	Yes
Subscribe to business events	No	No	Yes
Create Grants	No	No	Yes
Download Composite services	No (configurable)	Yes	Yes

In my experience it is easiest to just use the Integration Repository Administrator role in the development environment. If you want to limit the number of people that can actually create the web services you can do that in the testing, staging and production environment.

In the Integrated SOA Gateway Implementation Guide it is explained in "Setting up Oracle E-Business Suite Integrated SOA Gateway" how you can set up these roles.

Generate the WSDL
To generate the WSDL, navigate to the PL/SQL API in the Integration Repository.  In the project we generated a WSDL for PAY_ELEMENT_ENTRY_API, the function CREATE_ELEMENT.

Oracle® E-Business Suite Integrated SOA Gateway Developer's Guide describes in "Using PL/SQL as Web services" how you can accomplish this. In the screen shot below, you see what it looks like for the HR_EMPLOYEE_API for the function CREATE_EMPLOYEE.  (NB: I use the HR_EMPLOYEE_API example in the screen shots, because in our environment we already generated the WSDL for the PAY_ELEMENT_ENTRY, so the screens look different).

Now you can view the WSDL in your browser, by selecting: "View WSDL".  The Web Service status is added and is set to "Generated"

Deploy the WSDL
Once you have generated the WSDL, you need to actually deploy the webservice. This will install the required files on the Oracle Application Server. 
In this example we use username tokens as authentication mechanism, but SAML tokens are also supported. 

Grant Access 
Now you have to grant access to the webservice by using the "Create Grant" option. The best way to handle this, is to create one user for example "selfsvc" that has access to the webservices on behalf of the users that use the self service application. 

Call the Web Service
SQLJUTL is used when generating the web services from PL/SQL. This is the same technology that is used by the Oracle Database adapter and Eclipselink DBWS.  This requires some specific knowlegde. For example a PL/SQL boolean is generated as an integer:
<element name="P_VALIDATE" type="int" db:index="1" db:type="INTEGER" minOccurs="0" nillable="true" /> 
Documentation about the mapping form PL/SQL to XML can be found in  Oracle® Application Server Adapters for Files, FTP, Databases, and Enterprise Messaging User's Guide: "Acceptable values for the original BOOLEAN parameter now that it is an INTEGER are 1 for true and 0 for false. Any value other than 1 is considered false. The generated wrapper procedure uses APIs from the SYS.SQLJUTL package to convert from INTEGER to BOOLEAN and vice-versa."

Calling the webservice requires both knowledge of the implementation of the EBS module and knowledge of XML and Web Services. For example, it expects an assignment_id and an element_link_id as a parameter, it uses date-tracking and has the option to validate the call, instead of actually executing (committing) the data. 
Information about this can be found in the "Oracle Human Resources Management Systems Workforce Sourcing, Deployment, and Talent Management Guide". This guide explains for example the assignment concept: 
"The assignment is the central concept that relates employees to the structures in which they work, and the compensation and benefits for which they are eligible.

In Oracle HRMS, many of the activities you undertake in human resource management, such as vacancy management and budget planning, are based around assignments and not people. In particular, you enter all earnings, deductions, and other pay-related elements for the employee assignment, rather than the employee. This makes it possible to give an employee two or more assignments when this is necessary.

For example, if an employee has more than one role within your enterprise, he or she can have a separate assignment for each role."

You can test the web service using SOAPUI, or the standard test client on the Oracle Application server.
To test it with SOAP UI, you need to import the WSDL. You can find the address of the WSDL when you click on "View WSDL". Just copy the url in your browser into SOAP UI. It will look something like this:
http://[host]:[port]/webservices/SOAProvider/plsql/pay_element_entry_api/?wsdl


To test the web service in your browser, put the link in your browser, without the '?wsdl'
http://[host]:[port]/webservices/SOAProvider/plsql/pay_element_entry_api/


This will give you a webpage to test the service with:

Conclusion
We needed to install a number of patches before we were able to successfully use the SOA Gateway, but once the whole thing is setup (patches, users privileges etc), creating the web service is trivial. A very important benefit is that it is visible in the integration repository that a Web Service is generated, based on the PL/SQL API. This is much harder to keep track of when you use a Database adapter, Eclipselink DBWS or the Oracle Apps adapter. 
Knowledge of the PL/SQL APIs in EBS is a must in your project. In our project we worked closely with the EBS functional and technical people to figure out how to use the PL/SQL APIs correctly and what APIs we needed. The biggest challenge in this type of project is to combine knowledge of EBS and PL/SQL with knowledge of Web Services and integration.  
What is seriously lacking in the PL/SQL APIs are simple 'read functions'. So if the first use case applies (using data from HR), you need to write your own APIs or call the database tables directly. The latter is a bad idea in my opinion. This mean that you always have to write some APIs yourself when interfacing with EBS.  In the Oracle® E-Business Suite Integrated SOA Gateway Developer's Guide you can find a description how you can add these custom interfaces to the repository and generate WSDLs from them in chapter "Creating and using custom integration interfaces".

Using DBWS to call PL/SQL APIs

This blog explains how to setup DBWS in Eclipse so you can generate webservices based on PL/SQL APIs in your database. This is a very convenient way of accessing existing logic in your database, without the need of heavy duty infrastructure like a SOA Suite or Service Bus.

DBWS is part of Eclipselink, the open source project lead by Oracle that transforms relational data to Java objects, and relational data to XML. 

I put the standard procedures in the HR schema, in a package.  In this example we want to expose the following PL/SQL API:

PACKAGE HR_PACKAGE AS

  PROCEDURE add_job_history

  (  p_emp_id          job_history.employee_id%type

   , p_start_date      job_history.start_date%type

   , p_end_date        job_history.end_date%type

   , p_job_id          job_history.job_id%type

   , p_department_id   job_history.department_id%type

   );

   PROCEDURE secure_dml;

END HR_PACKAGE;

SQLJUTL
Note that if you use the Oracle10g Express Edition database, you need to install SQLJUTL in the SYS schema. Susan Duncan wrote a blog about that back in 2006.

• Log into the database as sys (with the role sysdba) and run sqljutl.sql

DBWS builder file
The webservice is created based on a so called builder file. This file contains information about the database connection, and the procedures, tables and functions you want to expose as a webservice.  Here you see an example of the builder file that exposes the package we described above:

<?xml version="1.0" encoding="UTF-8"?>
<dbws-builder xmlns:xsd="http://www.w3.org/2001/XMLSchema">
<properties>
<property name="projectName">hr</property>
<property name="driver">oracle.jdbc.OracleDriver</property>
<property name="password">hr</property>
<property name="url">jdbc:oracle:thin:@localhost:1521:XE</property>
<property name="username">hr</property>
<property name="platformClassname">org.eclipse.persistence.platform.database.oracle.Oracle11Platform</property>
</properties>
<plsql-procedure
name="JOB_HISTORY"
    catalogPattern="HR_PACKAGE"
    procedurePattern="%"
    isSimpleXMLFormat="true"
  />
</dbws-builder>

If you have used JPA (or Eclipselink JPA), these properties might look familiar. There are a couple of things to note here:

• the database properties for DBWS are elements. In EclipseLink JPA these are defined as attributes, like this:

<persistence-unit name="default" transaction-type="RESOURCE_LOCAL">
    <provider>
        oracle.toplink.essentials.PersistenceProvider
    </provider>
    <exclude-unlisted-classes>false</exclude-unlisted-classes>
    <properties>
        <property name="javax.persistence.loglevel" value="INFO"/>
        <property name="javax.persistence.jdbc.driver" value="oracle.jdbc.OracleDriver"/>
        <property name="javax.persistenc.jdbc.url" value="jdbc:oracle:thin:@myhost:l521:MYSID"/>
        <property name="javax.persistence.jdbc.password" value="tiger"/>
        <property name="javax.persistence.jdbc.user" value="scott"/>
    </properties>
</persistence-unit>

• The database property names are different than then the database properties in EclipseLink JPA. You can find the list op properties in the documentation. 

After creating the builder file, you can generate the web service. You can run the dbws utility from the command line using the following command:

dbwsbuilder.cmd -builderFile dbws-builder.xml -stageDir stage -packageAs wls hr.war

Another option is to run the DBWSBuilder class from eclipse.

• Create a new webservice project 'jobhistory'
• Create a new Java project 'dbws-test'
• Create a variable that points to your eclipselink installation, and a variable that points to your jdbc driver:

• Create a Java class that runs the DBWSBuilder class:

package test;
import javax.wsdl.WSDLException;
import org.eclipse.persistence.tools.dbws.DBWSBuilder;
public class DBWSLauncherTest {
    public static void main(String[] args) {
        try {
            DBWSBuilder.main(args);
        }
        catch (WSDLException e) {
            e.printStackTrace();
        }
    }
}

• In the Run configuration you specify the arguments for the main class:

-builderFile ./dbws-builder.xml -stageDir /home/oracle/dbws/Jobhistory -packageAs eclipse

The utility will create a an exploded war file, that contains a WSDL, mapping files, and two Java classes.

• Deploy the webservice and insert a record in the job history table. 

I would like to thank Mike Norman, Team lead for the DBWS Eclipselink project, who helped me setup
Eclipse so I could run the DBWSBuilder class from within the IDE.

DOAG2011 Day 1: Quality

Today was the first day of the German Oracle User Groep (DOAG) conference. If you are interested in SOA and BPM, there is plenty to see: there is  a development track, a Middleware track and a BPM track.

When you develop SOA and BPM applications, quality and control is an important topic. Components are reused and processes span multiple departments. This means that you have to make sure that the processes you design perform the way you expect them to, and that the services you use, are functioning as expected.

These topics were all addressed one way or the other in the sessions I visited today. Let's start with the more technical aspects: the quality of the development process.

Continuous integration
Jurgen Broda, Martin Karmann and Daniel Kleine-Albers presented "SOA Continuous integration". In this session they explained the way they had used Hudson in their SOA (SCA-BPEL) project. For people who are familiar with test driven development and continuous integration in Java projects, the approach this team took was very similar to that. The only complication with SCA composites is you have to deploy them before you can test the code. In this project, they created a custom testing framework for integration testing of the composites. Note that the SCA composites testing framework can be used to unit test the composites.
The biggest advantages of continuous integration in a SOA projects are:

• Developers work on components that are part of services that are reused. Often this becomes very complex and it is hard to keep track of the functionality and quality. Using continuous integration breaks down the complexity in deployable units of code that run tests successfully. 
• If a change is being developed, there are regression tests available to make sure the rest of the services and processes did not break. 
• Requirements can be tracked if you tie the tests to the requirements, giving project managers a clear view of the progress. 

The thing the team found lacking is automated quality checking of the BPEL code. In the Java world we are use to analyze code using tools like FindBugs and CheckStyle. There is no such thing for BPEL, yet.  This was a very inspiring session; I used to think continuous integration was not too complicated in SOA projects. But now I know differently: in fact, I am going to propose to install Hudson and start doing continuous integration in our current project as soon as I get back!

AIA

Controlling the code you create using continuous integration is an important step towards quality and stability in your development process. But, creating SOA and BPM applications has additional requirements compared to 'regular' java or PL/SQL applications: code is supposed to be reused, and often you need to integrate existing packaged applications like Oracle eBusiness Suite or Siebel CRM. For this reason, Oracle provides a framework to integrate different packaged (Oracle) applications: Application Integration Architecture. In his session "Oracle AIA, does it deliver on it's integration promise" Ahmed Aboulnaga talked about his experience with Oracle AIA. Lowering risk and cost are the main reasons to get started with AIA. Using predefined process integration packs (PIPs) and the application integration foundation pack is not easy. But the architectural principles behind AIA are solid and this ensures that the integration code is reusable. Apart form that, a lot of time can be saved in analysis, because the data models (EBOs) are already defined. This is often a large part of the effort in a SOA project. Beware of customization though: this will become expensive very quickly. In fact, Ahmed stated that in AIA projects in general the initial investment is low (ROI is high), then in the middle (which can take up to two years) cost are high and finally it lowers again because of reuse of the integration code and good systems design. In the end it is worth it, but you have to know what you get into.

Process Simulation

Reusing architectural patterns, code and data models is a good way of ensuring quality in your projects. But every organization is different. For example, the type of staff you employ, the number of process instances you run etc. With BPM studio, process analysts have a simulation tool to make sure that the processes they design are efficient in terms of resource utilization and throughput. Michael Stapf showed in his session "Simulation von BPMN 2.0 Prozessmodellen mit BPM 11g" how to simulate a model that is created in BPMN 2.0 with Oracle BPM Studio 11g. The nice thing is, you don't have to deploy the model, nor do you need to implement all the services to run the simulation. The simulation engine checks the quality on a totally different level: it gives you the opportunity to monitor and control your process design. This is a very effective way of finding bottlenecks before you actually implement them in your organization. Changing a process that is in production is much harder than changing a design, obviously... A nice new feature in 11.1.1.5 (Feature pack) is the round trip simulation: you can add data from your running instances and feed that into the simulation models. 

Social BPM

So far I talked about ensuring code quality with Continuous integration and AIA and ensuring quality of design with AIA and process simulation. But a lot of quality gets injected in your process at runtime. People work together, make decisions, talk to each other, learn from similar cases etc etc. Social BPM is a term to indicate the application of Collaborative work to Business Process Management. Manas Deb explained in his SOA keynote "Social and collaborative BPM pushing organizational excellence" what this means for your organizations and how this helps you improve the quality of your organization. 

I really enjoyed this session. I graduated in Cognitive Science in the 90s and a large part of the curriculum at the time was CSCW or computer supported collaborative work. At that time, the tooling and connections (internet) was not as developed as it is now. It is good to see the application of these concepts in modern tooling like Oracle BPM. 

For me this was a very successful day. I enjoyed running into the "Oracle crowd", and attending sessions both in German and in English. I learned a lot. Unfortunately I did not learn enough German to do my presentations in German, so tomorrow I will talk in English about using the BPA Suite to generate BPEL and how User Experience and BPM fit together (using Fusion apps as an example). This fits nicely with the themes I have discussed so far.  I think the DOAG program is really well balanced, it is a nice location and after attending DOAG2011, everybody should be able to produce modern high quality applications ;)