The feared “demo-effect” – bluescreen just before our OOW session

Yesterday I arrived in San Francisco to attend and present on Oracle Open World 2008. After a really nice dinner with most of the ODTUG presenters, I quickly went to sleep. The day after -today as I’m writing this blog-, Lonneke and I were going to present the Oracle versus BEA shootout session in Moscone West. Exciting, moreover since the session was fully booked, with more than fifty people on the waiting list. So naturally we wanted to prepare, test, and fine-tune our demo’s this morning.

The day starts good. I had slept for more than 10 hours, my jetlag -that I really felt during the ODTUG dinner- was reduced to a minor disorientation. So far so good. I met with Lonneke to go over the demo’s. First thing that happens when I start my laptop is that I see the feared bluescreen of death telling me that a fatal core dump has occurred (sounds just like Star trek). Meanwhile the presentation was only a couple of hours away. Aarrrgghhh! To keep in Star trek terms, I only needed to realign the dilithium matrix to stabilize the warp field to fix this “coredump”. My laptop caught the demo-effect in its worst form: half our presentation is demo and my laptop is dead! That’s when my heart rate doubled and my blood pressure went to a new all-time high. Lonneke -normally making jokes to cheer you up when something goes wrong- kept quiet this time and looked at me alarmingly. Luckily, restarting computers when they’re broken or don’t do what you want can result in miracles: no bluescreen this time. Guess my computer had a jetlag too. After fixing our BEA demo’s and a quick rehearsal we left for Moscone. Luckily, the demo-virus stayed in the hotel and all demo’s worked perfectly during our presentation!

Later on this week we’ll post more on our Oracle Open World. We’ll also post the demo’s from our presentation later on.

Lonneke at OOW 2008.

Collect all the ribbons and become a fout-star general!

Remotely invoking clustered BPEL Worklist application

Oracle BPEL PM provides a Java API -the BPEL Worklist API- to connect to its Worklist application. There are several blogs that provide sample code on how to use it.

In production-like environments, components invoking the Worklist application can run on different servers for reasons of scalability and failover. This is also the case in one of our projects, where we have a SOA Suite cluster running ESB and BPEL, and another cluster running the ADF and WebCenter front-end. A custom front-end component presents task-related information to the users through a portlet. Tasks are queried and refreshed frequently and a user can have access up to several thousand tasks -not only user-assigned tasks are displayed but also tasks assigned to groups they belong to.

For remotely connecting to the BPEL Worklist application from the front-end components we considered the SOAP and RMI clients that are provided with the Worklist application. The type of client can be specified in the Java code accessing the Worklist application:

WorkflowServiceClientFactory.getWorkflowServiceClient(
WorkflowServiceClientFactory.REMOTE_CLIENT)

or

WorkflowServiceClientFactory.getWorkflowServiceClient(
WorkflowServiceClientFactory.SOAP_CLIENT)

Note that when using a SOAP client you have to add the statement Predicate.enableXMLSerialization(true) to make sure that predicate information for querying tasks is marshalled and sent to the Worklist application.

The actual configuration containing the information for the client on the Worklist web service endpoints and Worklist Java application is in the wf_client_config.xml file. That file is located in %ORACLE_HOME%/bpel/system/services/config.

One of the advantages of the SOAP client is that we can use WebCache as software load-balancer for HTTP requests. WebCache is already used in the project to load-balance and failover HTTP calls to external web services, ESB services, BPEL composite services and processes, etc. During stress-testing we found out that using a SOAP client -instead of RMI- roughly results in a 1.5 seconds response time penalty when querying 1500 tasks. That does not include other operations such as rendering tasks. The delay is most likely caused by XML marshalling and de-marshalling. RMI serializes object variables, but does not use the verbose XML format nor does it have to build a DOM tree in memory.

One of the questions that arises is whether failover is achievable using an RMI client. We don’t want a front-end component to be “tied” to only one SOA Suite runtime instance. It turns out this is possible by correctly configuring the wf_client_config.xml file. The serverUrl element in the configuration file defines the location of the Worklist application. You can use a comma-separated list of locations like you do when building an RMI initialcontext: java.naming.provider.url = "server1,server2".
The following piece of code from the wf_client_config.xml file achieves failover when using an RMI client to a clustered Worklist application:

<ejb>
<serverurl>
opmn:ormi://server01:6003:oc4j_soa/hw_services, opmn:ormi://server02:6003:oc4j_soa/hw_services
</serverurl>
<user>oc4jadmin</user>
<password>welcome1</password>
<initialcontextfactory>
oracle.j2ee.rmi.RMIInitialContextFactory
</initialcontextfactory>
</ejb>

For reasons of standardization a SOAP client is preferable. However low response times and other performance issues are important for good user-interaction. An additional delay of 1.5 seconds every time a user retrieves his or her tasks isn’t acceptable. Instead of using RMI -that is a less standardized protocol- to improve response times, you can also use the Worklist API to not query the entire collection of tasks, but only a certain range. This can then be used in combination with a SOAP client while maintaining acceptable reponse-times. The front-end component does need some modification though to handle navigation and iteration through these task subsets.

Experiences with Vista and Oracle software (2)

A few months ago I wrote this blog about installing (Oracle) software on my new Vista laptop. I already installed these components several times on Windows XP, so I thought it would be a walk in the park. In the end, it took me a couple of days instead of hours … :-(
There are lots of threads on the OTN forums -like this one- dealing with Vista. It turned out I wasn’t the only one struggling with it.

This blog is about installing Oracle SOA Suite 10.1.3.1 and patch 10.1.3.3 on Vista. Common errors thrown at me -some gave me nightmares :-) - were:

• OWSM configuration assitant fails.
• Error: Missing ormi[s]://host:port.
• This OC4J is configured to make JMX connections via RMIS and fall back to RMI if the RMIS port is not configured. The RMIS connection on the OC4J instance null on Application Server null is configured but a connection could not be established. The JMX protocol is specified by the oracle.oc4j.jmx.internal.connection.protocol property in opmn.xml.

I did three “batches” of steps to get it running:

• Perform the pre-installation steps as documented in the Oracle SOA Suite 10.1.3.1 and 10.1.3.3 installation guides. These contain the usual steps like configuring a loopback adapter, not using whitespace characters in directory names, having the right JDK, etc. These guides are bundled with the SOA Suite installation files when downloaded from OTN. Performing these steps is “business as usual”. However, these steps alone didn’t work.
• Perform some steps as documented in Metalink note 444112.1.
• Browse through the OTN forums for the remaining errors and stacktraces.

These last two “batches” consist of the following additional actions:

• Do not use underscores in your computer name. I had an underscore in it -love them since you can’t safely use whitespaces :-) - and I couldn’t run SOA Suite. See this forum post.
• IPV6 is not supported with Vista. There are some compatibility issues between Sun’s JDK and the new IPv6 protocol. I explicitly disabled IPv6 by removing “::1 localhost” from my hosts file, disabling Internet Protocol version 6 (TCP/IPv6) for all my network adapters in the Windows network configuration, and added the following registry value (DWORD type) set to 0xFF:
  HKEY_LOCAL_MACHINE, SYSTEM, CurrentControlSet, Services, Tcpip6, Parameters, DisabledComponents. This is documented in the Metalink note. You can also add the Java option “-Djava.net.preferIPv4Stack=true” to use IPv4 instead of IPv6.
• Install, start, and stop Oracle SOA Suite as administrator (e.g. right-click and choose “Run as administrator”). Somehow this causes the relative paths in the Start Menu shortcuts to not resolve to the correct locations anymore. I changed these locations to absolute file paths. To do this, edit the [SOA Suite home]/bin/runstartupconsole.bat file and change the line containing “set ORACLE_HOME”.
• Firewalls and virus scanners are great when they are not hogging your system and/or block wanted messages. The latter happened on my laptop. I disabled the pre-installed security software since it blocked certain requests and replies from and to my local Oracle SOA Suite.
• I did an advanced install and used Oracle 10g XE as dehydration store since the pre-installed Oracle Lite database (Oracle SOA Suite basic install) crashed every now and then.

These errors can be really frustrating. Especially when you successfully install the exact same software on another OS within the hour and all your “Linux” and “XP” colleagues have Hello World running on Oracle SOA Suite :-) Hope this helps.

Vote for sessions at Oracle OpenWorld 2008 (2)

Lonneke Dikmans already blogged about the possibility to submit and nominate sessions for Oracle OpenWorld 2008. This is one of the ways to get involved in the Oracle community through Oracle Mix.

I submitted an idea called "Putting SOA to Use". This session is based on experiences from a customer-case in which a Service-Oriented Architecture is implemented using Oracle technology such as BPEL, ESB and WebCenter. Read it and if you think it’s interesting vote for it!

Master/detail inserts and native sequencing in Oracle ESB

The SOA Suite forum on Oracle Technology Network (OTN) contains several posts on the use and configuration of the Database Adapter. This adapter can be used to perform database-related operations such as inserts, polling, invocation of PL/SQL, and so on in your SOA-environment. Some of these forum questions relate to inserting XML data into master/detail tables and how to use native database sequences to populate primary keys and automatically update associated foreign keys. I wrote an article Invoice Processing in a Service-Oriented Environment that contains a step by step tutorial on how to achieve this.

Database adapters, TopLink and Transformations

In one of our current customer SOA projects we’re using Oracle Enterprise Service Bus (ESB) to implement and expose services. One of these services involves transformation of inbound data and persisting this data in a database. We’re using XSL to transform the inbound data and Oracle Application Server’s database adapter to persist data into a relational database. JDeveloper, Oracle’s IDE, provides wizards to configure database adapters. TopLink (an object-relational mapping framework) mappings are generated as a result of this configuration. These mappings are XML files containing metadata such as the structure and format of the database tables. In our ESB flow data is transformed into nested XML format, which is persisted in multiple master-detail tables.

This week -after a modification to the database adapter configuration- data was still persisted, but lots of database records that were previously populated, were suddenly empty. The ESB console indicated all instances were valid and debugging the ESB flows showed that the input XML consumed by the database adapter was unchanged. After some more investigation it turned out that the order of tables in some of the generated TopLink mapping files (OurService_table.xsd, OurService_toplink_mappings.xml and OurService.RootTable.ClassDescriptor.xml) had altered; e.g. instead of table 1, table 2, table 3, the order of tables was table 1, table 3, table 2. The transformation activity still generated input XML data as table 1, table 2, table 3. Synching the order of tables in the transformation and mappings files resolved this issue. This means that runtime, the database adapter strictly follows the order of tables indicated in the TopLink mappings. You can argue whether this is too strict or not, especially since the same tools are very “tolerant” of other faults, omissions, validation, etc. Well, at least you would like to see some kind of error when the XML input format does not conform to the TopLink mapping definition.

Anyway, this was a tricky issue since the input XML seemed valid, the ESB console indicated all instances were valid, no error was logged and some data (but not all!) was persisted in the database.

Events and SOA

In my opinion events are just as important in a SOA as services. I think it’s not possible to achieve a “real” SOA without addressing events separately, just as we do for services. It can be somewhat confusing that events have there own acronym emphasizing their importance, namely Event Driven Architecture (EDA). Some coin this as the upgraded version of SOA, or SOA 2. How we love 3 (or 4 or 5) letter acronyms :-) Most of these different acronyms only add to the confusion, so I prefer sticking to SOA and emphasize events as integral part of it.

So why are events important?
A service-oriented approach should result in business-value (ROI), otherwise SOA only introduces technical complexity through additional middleware. One of the ways to achieve this is through business-IT alignment. Events play a major role in both of these worlds (business and IT).

Business
Businesses deal with (and are causing) events all the time: a customer moving to a different address, a new purchase order, receiving an invoice from a supplier, sending a bill to a partner, and so on. Entities in a business’ ecosystem such as employees, partners, suppliers and customers all react to these events. They initiate new processes, perform activities, propagate events, and so on. It is therefore logical to incorporate events when modeling business processes, which are at the heart of SOA. Just as in all modellng practices, one wants to model and focus on important aspects. In case of SOA this includes processes, services and events.

IT
From a technical point of view events can achieve asynchronicity and de-coupling. By using publish/subscribe and queueing mechanisms, software components are not required to know of each others existence. They simply subscribe to a topic and act based on received (and subscribed to) events. The other way around, if components have some (intermediate) result or state, they can share this with the rest of the world by publishing an event and then forget about it. These components don’t need to know what other components are interested in this information. Of course you’ll need some glue (i.e. middleware) to implement this.

So what does this mean concretely in a SOA-project? For one thing, besides trying to identify what types of services you have (business services, composite technical services, etc.), also investigate what types of events can be identified. There should be an event-registry besides a service-registry. This does not need to be a full-blown tool at first but can simply be documented using Excel in the beginning. Also define the relationship between processes, services and events. What services publish what type of events? What processes are initiated by what events? What running processes are influenced by what events?