Tuesday, December 27, 2011

Using DBWS to call PL/SQL APIs

This blog explains how to setup DBWS in Eclipse so you can generate webservices based on PL/SQL APIs in your database. This is a very convenient way of accessing existing logic in your database, without the need of heavy duty infrastructure like a SOA Suite or Service Bus.

DBWS is part of Eclipselink, the open source project lead by Oracle that transforms relational data to Java objects, and relational data to XML. 

I put the standard procedures in the HR schema, in a package.  In this example we want to expose the following PL/SQL API:

PACKAGE HR_PACKAGE AS
  PROCEDURE add_job_history
  (  p_emp_id          job_history.employee_id%type
   , p_start_date      job_history.start_date%type
   , p_end_date        job_history.end_date%type
   , p_job_id          job_history.job_id%type
   , p_department_id   job_history.department_id%type
   );
   PROCEDURE secure_dml;
END HR_PACKAGE;

SQLJUTL
Note that if you use the Oracle10g Express Edition database, you need to install SQLJUTL in the SYS schema. Susan Duncan wrote a blog about that back in 2006.
  • Log into the database as sys (with the role sysdba) and run sqljutl.sql
DBWS builder file
The webservice is created based on a so called builder file. This file contains information about the database connection, and the procedures, tables and functions you want to expose as a webservice.  Here you see an example of the builder file that exposes the package we described above:

<?xml version="1.0" encoding="UTF-8"?>
<dbws-builder xmlns:xsd="http://www.w3.org/2001/XMLSchema">
<properties>
<property name="projectName">hr</property>
<property name="driver">oracle.jdbc.OracleDriver</property>
<property name="password">hr</property>
<property name="url">jdbc:oracle:thin:@localhost:1521:XE</property>
<property name="username">hr</property>
<property name="platformClassname">org.eclipse.persistence.platform.database.oracle.Oracle11Platform</property>
</properties>
<plsql-procedure
name="JOB_HISTORY"
    catalogPattern="HR_PACKAGE"
    procedurePattern="%"
    isSimpleXMLFormat="true"
  />
</dbws-builder>

If you have used JPA (or Eclipselink JPA), these properties might look familiar. There are a couple of things to note here:
  • the database properties for DBWS are elements. In EclipseLink JPA these are defined as attributes, like this:
<persistence-unit name="default" transaction-type="RESOURCE_LOCAL">
    <provider>
        oracle.toplink.essentials.PersistenceProvider
    </provider>
    <exclude-unlisted-classes>false</exclude-unlisted-classes>
    <properties>
        <property name="javax.persistence.loglevel" value="INFO"/>
        <property name="javax.persistence.jdbc.driver" value="oracle.jdbc.OracleDriver"/>
        <property name="javax.persistenc.jdbc.url" value="jdbc:oracle:thin:@myhost:l521:MYSID"/>
        <property name="javax.persistence.jdbc.password" value="tiger"/>
        <property name="javax.persistence.jdbc.user" value="scott"/>
    </properties>
</persistence-unit>
  • The database property names are different than then the database properties in EclipseLink JPA. You can find the list op properties in the documentation
After creating the builder file, you can generate the web service. You can run the dbws utility from the command line using the following command:
dbwsbuilder.cmd -builderFile dbws-builder.xml -stageDir stage -packageAs wls hr.war

Another option is to run the DBWSBuilder class from eclipse.
  • Create a new webservice project 'jobhistory'
  • Create a new Java project 'dbws-test'
  • Create a variable that points to your eclipselink installation, and a variable that points to your jdbc driver:



  • Create a Java class that runs the DBWSBuilder class:
package test;
import javax.wsdl.WSDLException;
import org.eclipse.persistence.tools.dbws.DBWSBuilder;
public class DBWSLauncherTest {
    public static void main(String[] args) {
        try {
            DBWSBuilder.main(args);
        }
        catch (WSDLException e) {
            e.printStackTrace();
        }
    }
}
  • In the Run configuration you specify the arguments for the main class:
-builderFile ./dbws-builder.xml -stageDir /home/oracle/dbws/Jobhistory -packageAs eclipse

The utility will create a an exploded war file, that contains a WSDL, mapping files, and two Java classes.
  • Deploy the webservice and insert a record in the job history table. 

I would like to thank Mike Norman, Team lead for the DBWS Eclipselink project, who helped me setup
Eclipse so I could run the DBWSBuilder class from within the IDE.

Friday, December 23, 2011

Wednesday Wizardry - Small encore

Just a small encore on my blog post about the Wednesday Wizardry session at UKOUG 2011. Thanks to Simon Haslam we have a 180 degree view on the setting and team. Based on the frowning and dislayed code on the screen the team was in the final stage of the session.

180 degree view of the Wednesday Wizardry session at UKOUG 2011

Sunday, December 18, 2011

Vennster @ UKOUG 2011

The UK Oracle User Group (UKOUG) Conference 2011 in Birmingham is a wrap! Well, at least it was when I started writing this blog :-) Now it has been over a week since I returned on a bumpy flight from Birmingham from my first UKOUG conference which I really enjoyed! Good atmosphere, friendly people, quality presentations, lots of familiar people, and surprisingly good food (at some places at least).

Here's a short summary of the sessions that were hosted by Vennster. Due to last minute cancellations we were asked to fill in for some of the speakers that couldn't attend and presented some more sessions than expected.

BPA Suite to BPEL case study
Lonneke presented on how to model business processes in Oracle BPA Suite and how to transform business process definitions from BPMN to BPEL along with the advantages and disadvantages of this approach. You can find the slides and more background information in this previous blog post.

Effective Fault Handling in Oracle SOA Suite 11g
I presented on fault prevention and handling in SOA and BPM environments and the mechanisms Oracle SOA Suite 11g offers for fault handling. A fault can be defined as something that happened outside the expected operational activity or “happy flow”. Faults can be categorized into:

  • Technical faults (network errors, full tablespace, etc.)
  • Programming faults (nullpointer exception, cursor not closed, etc.)
  • Business faults (credit check failure, invoice amount higher than the ordered amount, etc.)
  • Faulty user input (return date of a flight before the departure date, wrong credit card number, etc.).

To prevent faults you can apply various techniques: configure a robust infrastructure to avoid technical errors (clustering, redundancy, high availability), use pair-programming and peer-reviews to avoid programming errors, identify business requirements and implement these to avoid and correctly deal with business faults, and apply user experience techniques to lower the chance of faulty user input and its impact. This session mostly focused on fault prevention and handling for technical and business faults.

Event-Driven SOA: Events meet Services
Lonneke and I presented (unexpectedly since the would-be presenter was unable to attend) the use of events and how events, services, and processes can work together to provide flexibility, decoupling, and realize business requirements. Events are important from a business as well as technology perspective. Oracle SOA Suite supports eventing through several implemenations such as Advanced Queuing (AQ), Java Message Service (JMS), and the Event Delivery Network (EDN).

I included the slides from my earlier eventing presentation at Kaleidoscope 2011 which was quite similar to the UKOUG session.

Overview of Eventing in Oracle SOA Suite 11g

Some background information can be found in this 2008 blog post explaining the importance of events.

Approach to SOA: Making This a Successful Endeavor for the Whole Organization
Lonneke was asked last-minute to present on why and when to use SOA and what will makes it successful using the (according to some) infamous "breakfast" example. At the end of this session, you will know what SOA means, when it is best used, and how you can get there. 

Wednesday Wizardry
Lonneke, Lucas, and Ronald showed how to rapidly build an enterprise application based on SOA and BPM principles using Oracle Fusion Middleware in only a couple of hours. Read all about it in this blog post.

Other sessions
Due to the unexpected sessions we needed to host I wasn't able to attend that many other presentations. However, I did visit and enjoyed the "WebLogic Server and Oracle RAC" session by Simon Haslam and Frances Zhao on the integration between Oracle WebLogic Server and the Oracle (RAC) Database; especially Active GridLink and the enhancements it offers.

Christmas in Birmingham: Lonneke in front of a square with a giant polar bear and snooker on the big screen

As for me, until next year at UKOUG 2012!

Thursday, December 8, 2011

Wednesday Wizardry

No, this is not the title of a new Harry Potter novel. If it were, there would be lines of people in front of the ICC Birmingham days ago where the UKOUG conference took place. Instead there was a modest but very interested group of spectators assembled who were interested in how to rapidly build an enterprise application based on SOA and BPM principles using Oracle Fusion Middleware in only a couple of hours.

Let's step back a little bit. This year at the ODTUG Kaleidoscope conference in Long Beach a team of Dutch ACEs and ACE Directors took part in a rapid software development session called Thursday Thunder. The format is to have a small team of experienced developers that each build components of an enterprise application in parallel. The application consists of a business process, services, and a frontend. Such an approach is possible since the application is developed in a SOA-fashion: based on loosely-coupled components with well-defined interfaces that can be easily integrated. The developers' laptops are hooked up to projectors so the crowd can watch whatever developer, programming, and tools they are interested in. You'll also experience the interactions between developers and the pros and cons of the decisions they make. Moderators will question these choices, take questions from the audience, explain the design choices, interview the developers, and act as (annoying) stakeholder to the developers to spice things up a little bit. After a conference filled with theory and small demos, a final live development session showing all these concepts and resulting in a concrete result (a working application) in real-life is fun to watch!

Due to the success of the session at Kaleidoscope the format was copied at DOAG by our German and Swiss fellow ACEs and ACE Directors and repeated for this years UKOUG conference in Birmingham: Wednesday Wizardry! Goal was to develop an application supporting a conference and its speakers by implementing a speakers business process (submit an abstract, review and accept or reject it, invite a speaker, upload the presentation, collect evaluations, and so on).

Now we had a small setback with the original team. We lost some developers and moderators due to an ice hockey accident and last minute meetings. Of the original team only Lonneke Dikmans (Vennster), Lucas Jellema (Amis), and Ronald van Luttikhuizen (Vennster) remained. Luckily, Simon Haslam and John King were willing to step up as moderators!

So what were the three of us supposed to build in a three hour window? The figure below shows an overview of the various components involved. Lonneke would implement the speakers business process (blue) using Oracle SOA Suite 11g, Lucas would realize the user interface (yellow) using ADF 11g, and Ronald would develop the services (red) using Oracle SOA Suite 11g as well.

Overview of Wednesday Wizardry case


The team prepared only the database (green) and the interface of the services (so its WSDLs and XSDs) in advance. The rest was build from scratch.

In the beginning we had Murphy pay us a visit: There were some network problems and the VM on which the server was running was not reachable. Thanks to Alex Gorbachev, tips & tricks from the audience, and help from the tech guys we managed to solve the connectivity issues. In somewhat more than the two hours that were left the team was able to develop almost all of the functionality shown in the image and do a live demo for the "manager" (Simon was kind enough to play for manager, it was scary how well he did that...) at the end of the session. All steps from submitting an abstract to the acceptance and indicating the session was done were successfully executed!

Wednesday Wizardry Team in action; from left to right: Lonneke, Ronald and Lucas


Some general key take-aways:

  • SOA helps in breaking up systems into well-defined services that can be easily integrated. Thereby reducing complexity of components, speeding up development, and enabling parallel implementation.
  • A bigger development team is not always the best choice. A small experienced team can realize software fast(er) with less communication overhead.
  • Live enterprise application development in a few hours is doable and fun for both audience as well as developers. Things can go wrong and will go wrong, but these are the same issues real software development teams run into.

Some specific and technical take-aways:

  • Use MDS to store your service interfaces (WSDLs and XSDs); even for small projects! We didn't use MDS as team (too little preparation time) and as a side-effect had some troubles with defining service references for Web Services that pointed to local artefacts that were unreachable from the developers' machine.
  • Use scripting for deployment for predictable and less error-prone deployment of software components.
  • The out-of-the-box worklist application is an excellent tool in testing a busines sprocess. You don't need a working user interface that interacts with the Human Workflow components to test the business process.
  • Never overwrite a database trigger that enqueues an event on AQ with a trigger that selects sysdate from dual 5 minutes before the final demo :-)

So, what will be the next live application development session after Thursday Thunder and Wednesday Wizardry? Friday Fantasy, Monday Maniac, Tropic Tuesday? Our team is ready for it :-)

Saturday, November 19, 2011

DOAG2011 Day 2: Vennster presentations

Wednesday was the day I presented on BPA Suite to BPEL and User experience in the Enterprise. As I mentioned in the previous blog, this fitted nicely with the sessions I attended on Tuesday.

BPA Suite to BPEL case study
The first session I presented talked about the use of the Oracle BPA Suite as the modeling tool for the business processes in a project. The Oracle BPA suite provides an option to automatically transform BPMN (Business process execution language) to BPEL (Business process execution language). This approach has some advantages and disadvantages, as I described in "Transforming BPMN to BPEL: Why and How" a couple of years ago.
So, not surprisingly, we ran into some issues in the project; both from a design perspective and from a development perspective. These findings were discussed, including some pointers to solve these issues:

Using Oracle BPM (the BPMN 2.0 designtime and runtime tool from Oracle) will resolve a lot of the translation and impedance mismatch issues. But other issues will remain; the most important things to keep in mind when using the models that are created by analyst for development are:
  1. Make sure the analyst is fluent in BPMN 2.0.
  2. Make sure the business process is not to detailed, so that the best practices from a development and user experience perspective can still be applied.
User experience in the enterprise
The second presentation concerned the second item: best practices from a user experience perspective. Often when Business processes are modeled, the analyst will model all the tasks and then a screen is generated for one task. This might seem like a cheap solution from a business perspective, but usually this leads to very clunky user interfaces and expensive changes to the process once it is deployed an in production.
It is important to clearly separate the user interface logic (flow) from the business process flow and apply common user interface practices and patterns in the application. This is exactly what happened in Fusion application: user experience was an integral part of the development process.

You can apply these techniques in different types of projects:

  1. Custom development projects
  2. Customization of packaged application
  3. Creating packaged application.
It will save you a lot of money in the life cycle of the application, and does not cost a lot to implement!

Tuesday, November 15, 2011

DOAG2011 Day 1: Quality

Today was the first day of the German Oracle User Groep (DOAG) conference. If you are interested in SOA and BPM, there is plenty to see: there is  a development track, a Middleware track and a BPM track.

When you develop SOA and BPM applications, quality and control is an important topic. Components are reused and processes span multiple departments. This means that you have to make sure that the processes you design perform the way you expect them to, and that the services you use, are functioning as expected.

These topics were all addressed one way or the other in the sessions I visited today. Let's start with the more technical aspects: the quality of the development process.

Continuous integration
Jurgen Broda, Martin Karmann and Daniel Kleine-Albers presented "SOA Continuous integration". In this session they explained the way they had used Hudson in their SOA (SCA-BPEL) project. For people who are familiar with test driven development and continuous integration in Java projects, the approach this team took was very similar to that. The only complication with SCA composites is you have to deploy them before you can test the code. In this project, they created a custom testing framework for integration testing of the composites. Note that the SCA composites testing framework can be used to unit test the composites.
The biggest advantages of continuous integration in a SOA projects are:

  • Developers work on components that are part of services that are reused. Often this becomes very complex and it is hard to keep track of the functionality and quality. Using continuous integration breaks down the complexity in deployable units of code that run tests successfully. 
  • If a change is being developed, there are regression tests available to make sure the rest of the services and processes did not break. 
  • Requirements can be tracked if you tie the tests to the requirements, giving project managers a clear view of the progress. 
The thing the team found lacking is automated quality checking of the BPEL code. In the Java world we are use to analyze code using tools like FindBugs and CheckStyle. There is no such thing for BPEL, yet.  This was a very inspiring session; I used to think continuous integration was not too complicated in SOA projects. But now I know differently: in fact, I am going to propose to install Hudson and start doing continuous integration in our current project as soon as I get back!

AIA
Controlling the code you create using continuous integration is an important step towards quality and stability in your development process. But, creating SOA and BPM applications has additional requirements compared to 'regular' java or PL/SQL applications: code is supposed to be reused, and often you need to integrate existing packaged applications like Oracle eBusiness Suite or Siebel CRM. For this reason, Oracle provides a framework to integrate different packaged (Oracle) applications: Application Integration Architecture. In his session "Oracle AIA, does it deliver on it's integration promise" Ahmed Aboulnaga talked about his experience with Oracle AIA. Lowering risk and cost are the main reasons to get started with AIA. Using predefined process integration packs (PIPs) and the application integration foundation pack is not easy. But the architectural principles behind AIA are solid and this ensures that the integration code is reusable. Apart form that, a lot of time can be saved in analysis, because the data models (EBOs) are already defined. This is often a large part of the effort in a SOA project. Beware of customization though: this will become expensive very quickly. In fact, Ahmed stated that in AIA projects in general the initial investment is low (ROI is high), then in the middle (which can take up to two years) cost are high and finally it lowers again because of reuse of the integration code and good systems design. In the end it is worth it, but you have to know what you get into.

Process Simulation
Reusing architectural patterns, code and data models is a good way of ensuring quality in your projects. But every organization is different. For example, the type of staff you employ, the number of process instances you run etc. With BPM studio, process analysts have a simulation tool to make sure that the processes they design are efficient in terms of resource utilization and throughput. Michael Stapf showed in his session "Simulation von BPMN 2.0 Prozessmodellen mit BPM 11g" how to simulate a model that is created in BPMN 2.0 with Oracle BPM Studio 11g. The nice thing is, you don't have to deploy the model, nor do you need to implement all the services to run the simulation. The simulation engine checks the quality on a totally different level: it gives you the opportunity to monitor and control your process design. This is a very effective way of finding bottlenecks before you actually implement them in your organization. Changing a process that is in production is much harder than changing a design, obviously... A nice new feature in 11.1.1.5 (Feature pack) is the round trip simulation: you can add data from your running instances and feed that into the simulation models. 

Social BPM
So far I talked about ensuring code quality with Continuous integration and AIA and ensuring quality of design with AIA and process simulation. But a lot of quality gets injected in your process at runtime. People work together, make decisions, talk to each other, learn from similar cases etc etc. Social BPM is a term to indicate the application of Collaborative work to Business Process Management. Manas Deb explained in his SOA keynote "Social and collaborative BPM pushing organizational excellence" what this means for your organizations and how this helps you improve the quality of your organization. 
I really enjoyed this session. I graduated in Cognitive Science in the 90s and a large part of the curriculum at the time was CSCW or computer supported collaborative work. At that time, the tooling and connections (internet) was not as developed as it is now. It is good to see the application of these concepts in modern tooling like Oracle BPM. 

For me this was a very successful day. I enjoyed running into the "Oracle crowd", and attending sessions both in German and in English. I learned a lot. Unfortunately I did not learn enough German to do my presentations in German, so tomorrow I will talk in English about using the BPA Suite to generate BPEL and how User Experience and BPM fit together (using Fusion apps as an example). This fits nicely with the themes I have discussed so far.  I think the DOAG program is really well balanced, it is a nice location and after attending DOAG2011, everybody should be able to produce modern high quality applications ;) 

Friday, October 21, 2011

SOA Partner Community Award for Dutch ACE team

Every year at the SOA Partner Community dinner at Oracle OpenWorld, Jürgen Kress announces the winners of that year's SOA Partner Community awards. These awards are a recognition of contributions to the Oracle community in specific areas such as BPM 11g, SOA 11g, and specialization. This year there was a special community award for the Dutch ACE team for their community contributions through blogging, articles, tweeting, presentations, books, and so on. Vennster's Lonneke and Ronald are part of that team. Lonneke accepted the award on behalf of the others.

For the complete list of awards and winners see the SOA Community Blog.

Friday, October 7, 2011

Magic OpenWorld pass

You ever had the disappointing experience that you weren't allowed in a session or hands-on lab cause you didn't register and the session was already fully booked? I certainly had that a couple of times. This year is different, and not because I already finished the Schedule Builder months ago. I don't know why but somehow I'm always allowed to enter sessions this year :-) Even those sessions I didn't register for and are fully booked. Every time the green checkbox mark appears after scanning. I don't know what it is, but I'm going to treasure and hang on to my OpenWorld pass, that's for sure. Beginning to hope the magic pass also works on the business class lounge at the airport and the
ATM :-)

Just for the fun of it, a picture of Duke & Vennster.

Wednesday, October 5, 2011

Oracle OpenWorld 2011, The Story So Far ... !

It doesn't matter how often you've visited Oracle OpenWorld before, it remains an overwhelming experience every year! The enormous crowds (over 45,000 this year), the gigantic amount of information to absorb, all the networking, business opportunities, social events and meetups with old friends and new people, all the new ideas that start bubbling in your head by listening, dicussing, asking, answering, and presenting.

Mix all of that with a proper jetlag and you'll get a feel of the spicy cocktail that OpenWorld is :-) Returning from OpenWorld gives you a tired yet really energetic feel!

Tuesday feels a bit like the calm before the storm. Everyone digested a lot of information and attended a lot of events these last few days and is now prepping for the final encore on wednesday with Larry's keynote and the appreciation event that evening at Treasure Island.

To give you an idea of the things going on at OOW, I've compiled a short list of activities we participated in so far:
  • ACE Director Briefing at Oracle HQ. Product management gives a preview of the things to be announced at OpenWorld and presents roadmaps of Oracle's products and tools.
  • BPM and SOA Partner Advisory Council. Every year Jurgen Kress organizes a partner council at OOW in which product management present a detailed roadmap for BPM and SOA products such as Oracle SOA Suite, Oracle BPM Suite, Oracle Enterprise Repository, AIA, and so on. Partners are encouraged to provide feedback for product improvement. It's good to see that the Oracle products in this space such as OSB and SOA Suite are mature and future versions will build on the same programming models and will be made even more stable. 
  • SOA & BPM Partner Social Event. This year Jurgen took us eBiking in San Francisco and Sausalito. Beautiful weather, beautiful scenery, and a lovely dinner at Sausalito. Excellent event. This was followed up by the SOA & BPM Community Reception at Gordon Biersch later this week. Again a great event. And on behalf of the Dutch, thanks for the "active community" award :-)
  • Loads of sessions and hands-on labs on BPM, SOA, NoSQL, Java/JEE, and so on. This year I had a chance to play with Coherence, BAM, CEP, ADF Mobile, BPM, and some other cool three letter acronyms.
  • Lonneke presented on how to make SOA successful - session 28720 "Approach to SOA: Making This A Successful Endeavor For The Whole Organization". Full room!
  • Annual ACE Dinner. Yet another great event to meet up with the OTN team and your Oracle peers!
  • More good stuff from the OTN team who make a lot of this stuff possible! The OTN Lounge and OTN Night Party. Excellent hangsouts for Oracle developers! Go to the Howard Street Tent. Always something good going on in the OTN Lounge with techcasts, meetups with the OTN team, chatting with others from the OTN community.
  • ODTUG social event in the French Bistro. Already lots of discussions on Kaleidoscope 2012.
  • Oracle Publishers Seminar. Another heads up from product management on all the various products. We were invited thanks to Packt, who is publisher of the SOA Made Simple book we're writing.
  • The Fusion Apps User Experience Advocates program had a short meeting to discuss the UX topics for the upcoming period.
  • Demogrounds. Almost all Oracle products are demoed at Moscone. Go there to learn more about products and ask questions to Oracle staff. 
  • Oracle BeNeLux event at Ruby Skye.


What's there to come?
  • Meetup with Bob Rhubart (OTN Architect Community) for a podcast recording on our new book "SOA Made Simple".
  • Larry's keynote of course: bright lights, loud music, cool intro, and hopefully some big news!
  • Appreciation event at Treasure Island with Sting!
  • Sessions, and even more sessions!
So, I can only say this. Come to OpenWorld next year if you're not here this time!

Friday, September 30, 2011

Warships and Architecture

Not so long ago I visited Stockholm and its Vasa Museum. The Vasa is a legendary Swedish warship from the 17th century that was meant to be one of the most fearsome warships at that time. The Vasa sank outside Stockholm’s harbor in 1628 after only a few kilometers on its maiden trip. Most likely reason was instability due to faulty weight distribution causing the top-heavy ship to sink. The Vasa has been salvaged in 1961 and restored in the following decades. As of 1990 the ship is displayed in the spectacular Vasa museum that has been visited by millions of people ever since.

The Vasa from the Bow by Javier Kohen

Walking around in the museum and reading why the Vasa sunk felt like reading about failed IT projects. Since we love analogies in IT, I will compare the construction of the Vasa to an IT project and see how architecture can help us in realizing an IT project successfully. Before we begin, a disclaimer on the accuracy of the history and construction of the Vasa in this blog is in order since I'm not an historian and neither a shipbuilder.

Good design doesn't mean good architecture: Why design alone isn't enough
You can hire experienced craftsmen to design and build a ship. Those people can be top of the line in their respective area such as armament, rope-making, carpeting, and sail-making. Putting together such a team doesn’t automatically result in a good ship however. For example, while each of the ships canons could have been of great quality and have an enormous range, placing too much heavy canons on the upper deck and having too little weight in the bottom of the ship can cause instability.

Architecture and design are often intermixed. Architecture is about the overall structure of a system and the rules and principles that govern the system as a whole. Design is more about the detailed structure of a particular subsystem or smaller component. Suppose we create an IT system to support the Customer Care department. The system can be made up of a Document Management System, Web Portal, and Case Management System. While all the different components can work really well themselves from both functional and technical perspective, you need to make an effort to make these components work together and also in such a way that supports Customer Care. Bad design will probably mean a bad solution. Good design however can still result in a bad solution. We need overall structure and coherence between the parts of an IT system to make them work together optimally and create a valuable solution. The various IT systems in an organization need to work together to support the enterprise. This is what architecture is about.

Using best-practices: Applying reference architecture
In the time period in which the Vasa was built, knowledge and experience were mostly in the heads of masters and passed on to apprentices. Knowledge and experience wasn't as broadly and readily available as it is nowadays. Also there weren't many ships built in Sweden like the Vasa that had multiple gun decks. When the main ship-builder Henrik Hybertsson died and his less experienced successor took over, he didn't have much best-practices or blueprints available for completing the Vasa.

In IT and other fields we have reference architectures that contain best-practices and guidelines to implement systems. Examples of reference architectures are eTOM for telcos and NORA for Dutch governments. Such blueprints speed up design and help you avoid common pitfalls.

On a side note, I recently participated in a podcast on reference architectures hosted by Bob Rhubart. One of the statements was that you shouldn’t apply reference architectures in an organization or project without first validating the applicability of the guidelines. Reference architectures have their merits, but can be too abstract, too elaborate, or too complex for a particular project or particular organization. However, Reference architectures are useful tools that can be applied to solve particular problems.

Not only managers, but also architects need to have an opinion on how and when to meet requirements
King Gustavus Adolphus of Sweden had some pretty strong requirements for the Vasa. It should have massive firepower and be constructed in a short time period. Requirements such as dimensions were frequently changed over time. It's interesting to review these requirements when we take a step back. Requirements are often derived from a particular goal or strategy. If the goal is to achieve naval superiority in the Baltic region, an alternative to a few big battleships like the Vasa could be the deployment of a larger number of smaller war vessels that can be created with the same total budget and within the same timeframe. Perhaps these vessels have less firepower but can cover a larger part of the Baltic region.

Architects can help make such cost/benefit analysis in IT projects or within the enterprise as a whole and align projects to the overall strategy.

If project managers say no to requirements it is usually because of time or budget constraints. Managers often need to balance quality, price and time. Besides budget and time one can also validate the realization of requirements based on compliance with architectural guidelines. If Java is the defacto standard then building a perfect PHP system that realizes all requirements and is delivered in time and within budget can still cause serious problems. For example when an enterprise doesn’t have the knowledge to correctly administer PHP systems and has a policy of not outsourcing system administration. Architects should look at the feasibility of requirements from the structures, principles, and boundaries they have set up in an architecture. Architects should propose alternatives that better fit the architecture. Architects should also stand up to managers if requirements don’t fit enterprise policies.

Also test architecture, not only functional and technical requirements
It is said that the stability test of the Vasa (men running up and down the deck) was cancelled shortly after it began because the ship almost tumbled over. Still construction and launch of the Vasa continued.

IT projects test systems to validate that functional and technical requirements are met. They furthermore test to verify that changes to the IT system do not introduce faults and cause regression. It is somewhat less obvious to test the architecture or architectural constraints and principles a IT system should adhere to. It can still be that all functional and technical requirements are met, but that e.g. applying changes after taking an IT system in production takes way too long. This can be an indicator of a flaw in the IT systems architecture that has big impact.

For example, a solution architecture could state that services should be reusable by future consumers. At first these services could only be used by a single consumer. We can actually test if these services are reusable (or whether they might be too specific to be used by future consumers) by investigating the service contract and interface.

To summarize: good architecture helps making IT projects successful, good design alone isn’t enough, we can learn lessons from other fields for applying architecture, and finally make sure you visit the Vasa museum when you’re in Stockholm!

Monday, July 11, 2011

Personas on the wall help the development team to focus

Recently I helped an insurance company to improve the User Experience of a website that had been specifically designed to sell their new internet service to young people with no kids.

The problem

A year after the launch of this service, the site had generated fewer sales amongst young people than expected. On top of that, older people and families, formerly regular customers, (not part of the target group) used the website to exchange their current service policy for this new (cheaper) internet service. That had explicitly not been the companies intention. They wondered what they could do about that.

The cause

I reviewed the current website and noticed that it was mostly designed from a business point of view.

  • All technical details about the service, the preconditions, the costs and the coverage were incorporated in the site, however not easy to find.
  • The language used to present the product was very formal and contained terminology hard to understand by young people, who are not familiar with the insurance business.
  • The range of services and appliances, offered by the site, was based more on availability (‘we have it in our assortment, so why not offer it!’) then on the needs of the target group.

When designing a website it is important to reckon with the business goals. In this case those goals are very clear, well contemplated and researched. However to develop a good and suitable User Experience it takes more than just business goals.

The solution

Know the user

The company acknowledged that they didn’t know much about the visitors of their website. Therefore I defined the personas[1] which represent the target group for their service offer.

Based on analysis of the target group, I found answers to questions like:

  • What are the user’s goals and priorities?
  • How do they behave?
  • Which attitudes do they have?
  • Their education
  • Their knowledge about the domain
  • Their use of a computer and internet
  • Their income and expenses

I segmented the information and created the personas. Per persona, I had their characteristics, their name (Fabio & Maggie) and a representative photo, printed at poster size.

Don’t lose sight of the user

Everywhere, where Fabio and Maggie were presented to the different project team members, I attached their posters on the wall. By doing this, the personas are visible during the whole presentation. I saw a look of recognition among the audience. They saw the characteristics and priorities of the target group and surprisingly noticed that these are not similar to their own.

From that moment on, no one in the team talked about ‘personas’ anymore. They all now call them by their fictive name as if Fabio & Maggie themselves are part of the team.

I advised the company to put the posters on the wall at each department which is involved in developing the website. By doing this, Fabio and Maggie will always be in their mind (“What would Fabio & Maggie think | do | want | need?” ) when taking any crucial decisions during the development process.

Involve the user

In the next steps of the project I wrote user scenarios based on Fabio & Maggie. I completed the requirement specification with user requirements. Till then the requirement list mainly contained technical and functional requirements.

I did a cardsort research involving the target group, in order to structure the content of the website based on the user’s perspective.

While Fabio & Maggie were hanging on the wall, I designed the website navigation structure in wireframes in close corporation with the graphic designers who gave the website a look & feel which suited both Fabio and Maggie.

The website developers are currently working on the end result. They also are familiar with Fabio & Maggie. It would be good if they’d run some usability tests during the development process with real users of the target group. By doing that in an early stage of the process, there is still time left to improve the design in favor of the user, before it is launched.


[1] A persona is a realistic character sketch representing one segment of a website’s targeted audience | Source: The User is Always Right by Steve Mulder with Ziv Yaar.

Saturday, June 25, 2011

User Experience in Fusion apps

Historically, Oracle is a technical company. Historically, Oracle is a database company. Once they started to offer development tools, modeling tools and applications, the whole concept always centered around data. User interfaces were things that you should generate based on a table definition, not something you should spend time on. So when I heard about the Fusion Applications User Experience efforts, I was skeptical...
...To put it mildly.
My view of Oracle and User Experience has changed drastically today. I attended the Fusion Apps User Experience training with a couple of other Ace Directors that are at Kaleidoscope 2011 in Long Beach.

The message
Companies buy Enterprise applications for support of their business. This is different than consumer facing applications like Amazon or Facebook; in Enterprise applications, users need to be able to accurately and efficiently complete their tasks. Fusion applications enables this because it takes the context of the user/customer into account. The business context consists of four things:
  • Who you are
  • What you are doing
  • Where you are
  • What information you need to complete the task
Research
Instead of looking at the data and generating a lot of screens on top of those data, Oracle has done extensive research to define the roles and persona's, the processes and tasks that need to be supported and the requirements these different users have.
They applied various user experience techniques like contextual inquiry, persona's and scenario's, card sorting, tasks analysis and qualitative methods like eye tracking and keystroke level modeling.

Analysis and modeling
This research resulted in prototypes and different solutions.
There are four key concepts in fusion apps:
  1. The right information at the right time. The information that is needed for a task is present, so the user does not have to navigate out of a screen to get to required information. Analysis of what information is needed at what time is key here. There are different patterns that are applied: showing information when hovering over an item, context sensitive actions that are available when clicking on an item, etc etc.
  2. Different types of navigation: search, dashboards, worklists (to-do lists), watch lists and bookmarks are examples of this. This caters different user preferences, and different types of tasks at different times.
  3. Collaboration and communication. People are a source of information, just like stored data. So collaboration and communication are an integral part of the fusion apps experience.
  4. Increased productivity and ease of use. By analyzing tasks, the actual productivity of the users is increased considerably.

Testing
The results were tested with customers to make sure that it actually increased the productivity and the overall experience.

The result
The approach Oracle took is very much task oriented with an emphasis on task analysis and quantitative research. This makes sense, considering the earlier statement about enterprise applications and the need to support the claims with data.
However, in addition to that, adding concepts like persuasive design into the mix to actually influence users in an organization would make it even more compelling. This is particularly true for areas like CRM or the so-called 'self service' or employee tasks like filling out expense reports, questionnaires, and updating resumes.

As I said in the beginning of this blog, my view of Oracle User Experience has changed drastically today. I think the Fusion apps people have solved the 'silo' problem that traditional ERP and CRM systems have. They did this by applying user centered design principles and of course have the architecture setup in a way that is supports this.

I look forward to hearing more from fusion apps in general and from this group in particular!

Friday, June 24, 2011

Security-per-environment using config plans

Software artifacts normally flow through several environments; for example development, test, integration, acceptance, and production. Some piece of software may be developed locally on your laptop, can be deployed to a central test environment by a developer, scheduled for deployment to the integration environment by the build manager, and then formally promoted to acceptance and production by system administrators after successful testing. These various environments rarely look the same. While a production environment might consist of a clustered and load-balanced configuration with multiple servers running on Linux, your development environment may consist of a bunch of laptops all running a single integrated server on Windows.

Not only sizing, server versions, hardware specs, and OS specifics can vary between these environments, also security configuration. It could be that the production environment enforces SSL/TLS for all internal and external Web Service calls, only uses official certificates issued by a trusted CA, applies WS-Security based message encryption for outbound Web Service calls, uses WS-Security SAML Tokens for authentication, and an appliance for SSL offloading instead of the application server itself. Maybe in the integration environment self-signed certificates and WS-Security UserName Tokens are used, while the development environment enforces no security measures whatsoever.

One way to deal with environment-specific settings is to manually configure these settings while promoting software items from one environment to the other. Maybe this is manageable for a small and simple application, this approach will soon be error-prone and high-maintenance when the size of the application increases. This is especially the case in SOA systems, where there is a high(er) number of different software components involved. A different approach is to use scripting and automated builds in which settings can be configured per environment.

Oracle SOA Suite 11g uses so-called config(uration) plans for this purpose. Environment-specifics like Web Service endpoint locations invoked by an SCA composite can be extracted from the SCA composite and stored in an XML config plan per environment. So we could have a MyComposite_cfgplan_dev.xml that indicates the endpoint of MyWebService is located at http://localhost:7001/MyWebService, while the endpoint is configured as https://some-server:8011/external/MyWebService in the MyComposite_cfgplan_prod.xml config plan for the production environment.

Oracle Web Service Manager (OWSM) is used by Oracle SOA Suite to secure services, references, and components of an SCA composite. You can for example apply the out-of-the-box oracle/wss_username_token_client_policy enforcing a WSS UserName Token to be included in the invocation of an external Web Service. While there are numerous examples online that explain the use of config plans and other examples explaining design-time addition of OWSM policies using JDeveloper, there is less information on how to include and configure OWSM-specific settings in config plans for SOA Suite 11g.

Adding OWSM policies to an SCA composite at designtime in JDeveloper

Adding OWSM policies to an SCA composite at designtime in JDeveloper

OWSM policies can be applied and configured per reference, service, or component. When applying security to a reference and configuring it for a specific environment, the OWSM settings need to be placed as wsp:PolicyReference element between the attribute and property elements of the reference in the config plan. The wsp prefix refers to the http://schemas.oracle.com/ws/2006/01/policy namespace.

For example (snippet from a configuration plan):


<reference name="MyExternalService">
  <binding type="ws">
    <attribute name="location">
      <replace>http://server:8011/SomeService-1.3?wsdl</replace>
    </attribute>
    <wsp:PolicyReference 
      orawsp:category="security" 
      orawsp:status="disabled"    
      URI="oracle/wss_username_token_client_policy"/>
    <property name="csf-key">
      <replace>BPMS_USER</replace>
    </property>
  </binding>
</reference>


In the above example, the oracle/wss_username_token_client_policy is applied on the MyExternalService reference but disabled during deployment. It furthermore indicates that the credentials stored in the BPMS_USER key that is located on the server should be used in constructing the WS-Security UserName Token for the outbound SOAP call. While this may be an appropriate configuration for the test environment, for production you might want the policy to be enforced during deployment, perhaps use another policy that enforces SSL, and possibly use another CSF key. In order to do this, you can add these OWSM-specifics to the reference element in the configur plan for production.

For example (snippet from a configuration plan):


<reference name="MyExternalService">
  <binding type="ws">
    <attribute name="location">
      <replace>http://prodserver:8012/Service-1.3?wsdl</replace>
    </attribute>
    <wsp:PolicyReference 
      orawsp:category="security" 
      orawsp:status="enabled"    
      URI="oracle/wss_username_token_over_ssl_client_policy"/>
    <property name="csf-key">
      <replace>PROD_PROXY_USER</replace>
    </property>
  </binding>
</reference>

Friday, June 3, 2011

The User Experience of the Dutch Chamber of Commerce (KvK)

Almost everyone who owns a business in the Netherlands has to deal with the Dutch Chamber of Commerce, in Dutch the KvK (Kamer van Koophandel). The Dutch Chamber of Commerce manages the trade register. Its other tasks are to provide Dutch entrepreneurs with information and stimulate regional trade & industry. It targets its services at Dutch businesses across all sectors.

The Five Channels of the KvK
Every BV yearly pays the KvK a fixed amount of money. In return the Chamber gives you information and answers your questions regarding business matters by means of 5 channels:

  1. Website
  2. Call centre
  3. Email
  4. Fax
  5. Post

The information and service given by all five channels determines the User Experience of the KvK.

The Experience of Defining an Annual Account
This year my BV, although sleeping, celebrated its 1st birthday. Therefore I had to fulfill al kind of legal obligations. One of them is the publication of the BV’s annual account. Now I myself am a User Experience consultant, and therefore I don’t know much about annual accounts.

OK, of course I could hire all kind of experts to help me, but come on… a sleeping BV… how difficult can it be? So with the help of a friend and some information (hard to read, in a difficult formal language) from the KvK site, I managed to formulate the account. But now what?

The Experience of Registering an Annual Account
I checked the 1st channel of the KvK, its website and tried to figure out how I could send my annual account to them. I found a FAQ: “How to register (deponeren) an annual account?” I would never have thought of the word ‘deponeren’ but this apparently was the information I needed.

Answer: “You can send it by post (6), by fax (5), by Email (4) or online (1) to your regional KvK.”

OK clear! That sounded good. Many options available! I clicked the link to find out contact information about my regional KvK. Then the confusion started. The website now first reveals different phone numbers, a visiting address and at the end of the page an E-mail address to send in a question or a complaint. The online-way was explained nowhere.

I tried the Email form, a waste of time because there is no possibility to attach a file. I saw no other option then to set up a call to the KvK. Therefore I first had to look up the telephone number of my regional KvK. I made the call, I passed a voice response system. The moment my call was answered by a receptionist, she transfered me to the KvK-expert. I finally could pop the question. The answer seemed simple, because in no time I received an odd Email address where I could send my annual account to. “No need for an addressee or a cover letter, just send the account to this address!”.

So I did. In half an hour I received an automatically generated Email from the KvK telling me that they received my mail and that my annual account will be published in due time. Great! Pfff I did it! The whole action took me an hour, a phone call and the help of some KvK employees.

Conclusion Regarding the Overall User Experience

  • Online I couldn’t find what I needed to know; therefore I had to consult the phone-channel.
  • The info on the website was not self-explanatory, the language too formal.
  • The whole process took me too much time.



What about Channel Control?
Channels which involve people (visitor center, call center, mailroom) cost money and they don’t always offer the most suitable service. Employees are kept away from their actual work by customers asking for Email addresses.

When promoting channels, start with the most cost effective ones. But watch out for the pitfall: Make sure that this channel fulfills the needs of its users.

Leading customers to the most (cost) effective and suitable channel and making sure this channel fulfills their user needs, assures you of:

  • happy customers
  • happy employees who can do their actual work more efficiently
  • cost reduction for your organization