Showing posts with label architecture. Show all posts
Showing posts with label architecture. Show all posts

Monday, March 25, 2013

SOA Made Simple


The book SOA Made Simple is published! SOA Made Simple is written by Vennster's Lonneke Dikmans and Ronald van Luttikhuizen. You can download a sample chapter and order the book from the Packt Publishing website

SOA Made Simple is a concise and indispensable handbook for finally understanding exactly what Service Oriented Architecture is. Split into three clear sections, in this book you’ll learn from both theory as well as step-by-step implementation examples to aid in your understanding of this often poorly- articulated industry term.

A short abstract for SOA Made Simple:
SOA is an industry term which is often preached like a religion rather than taught like a technology, and over time, grasping the concept has become unnecessarily difficult. Many companies proclaim that they don’t know where to begin with SOA, while others have begun their SOA effort but haven’t reaped the benefits they were convinced it would bring. “SOA Made Simple” explains what SOA is in simple terminology and by using real-life examples. Service-orientation is already a very natural way of thinking for business stakeholders that want to realize and sell services to potential clients, and this book helps you to realize that concept both in theory and practice. After reading “SOA Made Simple” you will have a clear understanding ofwhat SOA is so you can implement and govern SOA in your own organization. If you are an architect who wants to be completely clear in your understanding of what SOA is, then this book is essential. In fact, anyone (designer, developer, administrator or team lead) who is implementing or about to implement an architecture in an IT environment should not miss out on “SOA Made Simple”. 

Saturday, January 28, 2012

Start Small, Grow Fast | Companion Checklist

A companion checklist that summarizes the Oracle SOA Suite best practices contained in the "Start Small, Grow Fast" Oracle whitepaper by Demed L'Her, Edwin Biemond, and Ronald van Luttikhuizen at bit.ly/soa-start-small.



Administrative considerations

  • Involve operations teams from the beginning.
  • Make sure your organization has dedicated middleware administrators.
  • Prefer scripting for deployment of composites.
  • Use SOA Suite configuration plans to capture all environment-specific information instead of hardcoding them in your composites.
  • Avoid creating a separate configuration plan per composite per environment - consolidate these to a few or a single configuration plan per environment.
  • Script as many configuration tasks as possible from the beginning.
  • Document configurations in a well-known central location such as a team Wiki, readily available to all stakeholders.
  • Consider using NFS shares for JCA adapter configuration plans.
  • Use partitions to categorize composites and execute various tasks for multiple composites at once.
  • Use composite sensors to enable the search of specific instances.
  • Negative testing is an absolute requirement.


Infrastructure considerations

  • Build a cluster from the start -even if only with a single node. Think of the following when configuring a cluster:
    • Use individual IP addresses for all components (WebLogic Admin Server, Node Manager, and the Managed Servers).
    • Use the JRockit JVM and Mission Control to detect problems and perform tuning.
    • Leverage WebLogic channels to separate cluster traffic from production traffic.
    • Use a load balancer to divide traffic between services and detect outages of servers in the cluster.
  • Think about your domains. There is no such thing as a “one-size fits all” domain topology; however there are a few tips and guidelines in this area:
    • Always setup Node Manager.
    • Do not combine Admin and Managed Servers; except maybe in non-critical environments such as development.
    • Deploy your software and services to Managed Servers.
    • Leverage domains to partition environments with different lifecycles or significant functional differences.
    • Dedicate the most performant hardware to Managed Servers.
  • Linux x86 is a great entry-level platform for Oracle SOA Suite.
  • Favor Oracle Database as infrastructure database.
  • Perform realistic load tests.
  • Define and test a purging procedure well before going live.


Design-time considerations

  • Use MDS to centrally store artifacts and avoid duplication.
  • Have a canonical model for your core objects to ensure future re-use and consistency.
  • Consider taking a contract-first or meet-in-the-middle approach for interfaces exposed to the outside world.
  • Formulate and adhere to a small set of service design guidelines and naming conventions:
    • Language conventions
    • Naming conventions for composites, services, references and components
    • Naming conventions for BPEL and BPM components and activities
    • Naming conventions for composite sensors
  • Wrap frequently used SOA Suite APIs in simpler custom APIs.
  • Use Domain-Value Maps (DVM) and Business Rules to improve flexibility and agility.
  • Use fault policy files to separate exception and fault handling from “normal” process logic. Centrally store them in MDS to enable re-use across composites.
  • Unit test your composites using SOA Suite’s test framework as you would unit test your Java code using e.g. JUnit.


Architectural considerations

  • Use a simple and concrete service categorization.
  • Spend some time thinking about the granularity of your services to avoid too frequent refactoring. Reusability is a key factor. Other factors are rate of change, availability, and ownership.
  • Building asynchronicity through business events or messaging will improve loose-coupling, ease of deployment and administration and enable throttling.
  • Consider supporting multiple versions of the same service in production to allow service consumers to upgrade at their own pace.
  • SOA governance is needed but can start very simply with a wiki. Consider a full-fledge repository as you grown your SOA efforts.

Saturday, November 19, 2011

DOAG2011 Day 2: Vennster presentations

Wednesday was the day I presented on BPA Suite to BPEL and User experience in the Enterprise. As I mentioned in the previous blog, this fitted nicely with the sessions I attended on Tuesday.

BPA Suite to BPEL case study
The first session I presented talked about the use of the Oracle BPA Suite as the modeling tool for the business processes in a project. The Oracle BPA suite provides an option to automatically transform BPMN (Business process execution language) to BPEL (Business process execution language). This approach has some advantages and disadvantages, as I described in "Transforming BPMN to BPEL: Why and How" a couple of years ago.
So, not surprisingly, we ran into some issues in the project; both from a design perspective and from a development perspective. These findings were discussed, including some pointers to solve these issues:

Using Oracle BPM (the BPMN 2.0 designtime and runtime tool from Oracle) will resolve a lot of the translation and impedance mismatch issues. But other issues will remain; the most important things to keep in mind when using the models that are created by analyst for development are:
  1. Make sure the analyst is fluent in BPMN 2.0.
  2. Make sure the business process is not to detailed, so that the best practices from a development and user experience perspective can still be applied.
User experience in the enterprise
The second presentation concerned the second item: best practices from a user experience perspective. Often when Business processes are modeled, the analyst will model all the tasks and then a screen is generated for one task. This might seem like a cheap solution from a business perspective, but usually this leads to very clunky user interfaces and expensive changes to the process once it is deployed an in production.
It is important to clearly separate the user interface logic (flow) from the business process flow and apply common user interface practices and patterns in the application. This is exactly what happened in Fusion application: user experience was an integral part of the development process.

You can apply these techniques in different types of projects:

  1. Custom development projects
  2. Customization of packaged application
  3. Creating packaged application.
It will save you a lot of money in the life cycle of the application, and does not cost a lot to implement!

Tuesday, November 15, 2011

DOAG2011 Day 1: Quality

Today was the first day of the German Oracle User Groep (DOAG) conference. If you are interested in SOA and BPM, there is plenty to see: there is  a development track, a Middleware track and a BPM track.

When you develop SOA and BPM applications, quality and control is an important topic. Components are reused and processes span multiple departments. This means that you have to make sure that the processes you design perform the way you expect them to, and that the services you use, are functioning as expected.

These topics were all addressed one way or the other in the sessions I visited today. Let's start with the more technical aspects: the quality of the development process.

Continuous integration
Jurgen Broda, Martin Karmann and Daniel Kleine-Albers presented "SOA Continuous integration". In this session they explained the way they had used Hudson in their SOA (SCA-BPEL) project. For people who are familiar with test driven development and continuous integration in Java projects, the approach this team took was very similar to that. The only complication with SCA composites is you have to deploy them before you can test the code. In this project, they created a custom testing framework for integration testing of the composites. Note that the SCA composites testing framework can be used to unit test the composites.
The biggest advantages of continuous integration in a SOA projects are:

  • Developers work on components that are part of services that are reused. Often this becomes very complex and it is hard to keep track of the functionality and quality. Using continuous integration breaks down the complexity in deployable units of code that run tests successfully. 
  • If a change is being developed, there are regression tests available to make sure the rest of the services and processes did not break. 
  • Requirements can be tracked if you tie the tests to the requirements, giving project managers a clear view of the progress. 
The thing the team found lacking is automated quality checking of the BPEL code. In the Java world we are use to analyze code using tools like FindBugs and CheckStyle. There is no such thing for BPEL, yet.  This was a very inspiring session; I used to think continuous integration was not too complicated in SOA projects. But now I know differently: in fact, I am going to propose to install Hudson and start doing continuous integration in our current project as soon as I get back!

AIA
Controlling the code you create using continuous integration is an important step towards quality and stability in your development process. But, creating SOA and BPM applications has additional requirements compared to 'regular' java or PL/SQL applications: code is supposed to be reused, and often you need to integrate existing packaged applications like Oracle eBusiness Suite or Siebel CRM. For this reason, Oracle provides a framework to integrate different packaged (Oracle) applications: Application Integration Architecture. In his session "Oracle AIA, does it deliver on it's integration promise" Ahmed Aboulnaga talked about his experience with Oracle AIA. Lowering risk and cost are the main reasons to get started with AIA. Using predefined process integration packs (PIPs) and the application integration foundation pack is not easy. But the architectural principles behind AIA are solid and this ensures that the integration code is reusable. Apart form that, a lot of time can be saved in analysis, because the data models (EBOs) are already defined. This is often a large part of the effort in a SOA project. Beware of customization though: this will become expensive very quickly. In fact, Ahmed stated that in AIA projects in general the initial investment is low (ROI is high), then in the middle (which can take up to two years) cost are high and finally it lowers again because of reuse of the integration code and good systems design. In the end it is worth it, but you have to know what you get into.

Process Simulation
Reusing architectural patterns, code and data models is a good way of ensuring quality in your projects. But every organization is different. For example, the type of staff you employ, the number of process instances you run etc. With BPM studio, process analysts have a simulation tool to make sure that the processes they design are efficient in terms of resource utilization and throughput. Michael Stapf showed in his session "Simulation von BPMN 2.0 Prozessmodellen mit BPM 11g" how to simulate a model that is created in BPMN 2.0 with Oracle BPM Studio 11g. The nice thing is, you don't have to deploy the model, nor do you need to implement all the services to run the simulation. The simulation engine checks the quality on a totally different level: it gives you the opportunity to monitor and control your process design. This is a very effective way of finding bottlenecks before you actually implement them in your organization. Changing a process that is in production is much harder than changing a design, obviously... A nice new feature in 11.1.1.5 (Feature pack) is the round trip simulation: you can add data from your running instances and feed that into the simulation models. 

Social BPM
So far I talked about ensuring code quality with Continuous integration and AIA and ensuring quality of design with AIA and process simulation. But a lot of quality gets injected in your process at runtime. People work together, make decisions, talk to each other, learn from similar cases etc etc. Social BPM is a term to indicate the application of Collaborative work to Business Process Management. Manas Deb explained in his SOA keynote "Social and collaborative BPM pushing organizational excellence" what this means for your organizations and how this helps you improve the quality of your organization. 
I really enjoyed this session. I graduated in Cognitive Science in the 90s and a large part of the curriculum at the time was CSCW or computer supported collaborative work. At that time, the tooling and connections (internet) was not as developed as it is now. It is good to see the application of these concepts in modern tooling like Oracle BPM. 

For me this was a very successful day. I enjoyed running into the "Oracle crowd", and attending sessions both in German and in English. I learned a lot. Unfortunately I did not learn enough German to do my presentations in German, so tomorrow I will talk in English about using the BPA Suite to generate BPEL and how User Experience and BPM fit together (using Fusion apps as an example). This fits nicely with the themes I have discussed so far.  I think the DOAG program is really well balanced, it is a nice location and after attending DOAG2011, everybody should be able to produce modern high quality applications ;) 

Friday, September 30, 2011

Warships and Architecture

Not so long ago I visited Stockholm and its Vasa Museum. The Vasa is a legendary Swedish warship from the 17th century that was meant to be one of the most fearsome warships at that time. The Vasa sank outside Stockholm’s harbor in 1628 after only a few kilometers on its maiden trip. Most likely reason was instability due to faulty weight distribution causing the top-heavy ship to sink. The Vasa has been salvaged in 1961 and restored in the following decades. As of 1990 the ship is displayed in the spectacular Vasa museum that has been visited by millions of people ever since.

The Vasa from the Bow by Javier Kohen

Walking around in the museum and reading why the Vasa sunk felt like reading about failed IT projects. Since we love analogies in IT, I will compare the construction of the Vasa to an IT project and see how architecture can help us in realizing an IT project successfully. Before we begin, a disclaimer on the accuracy of the history and construction of the Vasa in this blog is in order since I'm not an historian and neither a shipbuilder.

Good design doesn't mean good architecture: Why design alone isn't enough
You can hire experienced craftsmen to design and build a ship. Those people can be top of the line in their respective area such as armament, rope-making, carpeting, and sail-making. Putting together such a team doesn’t automatically result in a good ship however. For example, while each of the ships canons could have been of great quality and have an enormous range, placing too much heavy canons on the upper deck and having too little weight in the bottom of the ship can cause instability.

Architecture and design are often intermixed. Architecture is about the overall structure of a system and the rules and principles that govern the system as a whole. Design is more about the detailed structure of a particular subsystem or smaller component. Suppose we create an IT system to support the Customer Care department. The system can be made up of a Document Management System, Web Portal, and Case Management System. While all the different components can work really well themselves from both functional and technical perspective, you need to make an effort to make these components work together and also in such a way that supports Customer Care. Bad design will probably mean a bad solution. Good design however can still result in a bad solution. We need overall structure and coherence between the parts of an IT system to make them work together optimally and create a valuable solution. The various IT systems in an organization need to work together to support the enterprise. This is what architecture is about.

Using best-practices: Applying reference architecture
In the time period in which the Vasa was built, knowledge and experience were mostly in the heads of masters and passed on to apprentices. Knowledge and experience wasn't as broadly and readily available as it is nowadays. Also there weren't many ships built in Sweden like the Vasa that had multiple gun decks. When the main ship-builder Henrik Hybertsson died and his less experienced successor took over, he didn't have much best-practices or blueprints available for completing the Vasa.

In IT and other fields we have reference architectures that contain best-practices and guidelines to implement systems. Examples of reference architectures are eTOM for telcos and NORA for Dutch governments. Such blueprints speed up design and help you avoid common pitfalls.

On a side note, I recently participated in a podcast on reference architectures hosted by Bob Rhubart. One of the statements was that you shouldn’t apply reference architectures in an organization or project without first validating the applicability of the guidelines. Reference architectures have their merits, but can be too abstract, too elaborate, or too complex for a particular project or particular organization. However, Reference architectures are useful tools that can be applied to solve particular problems.

Not only managers, but also architects need to have an opinion on how and when to meet requirements
King Gustavus Adolphus of Sweden had some pretty strong requirements for the Vasa. It should have massive firepower and be constructed in a short time period. Requirements such as dimensions were frequently changed over time. It's interesting to review these requirements when we take a step back. Requirements are often derived from a particular goal or strategy. If the goal is to achieve naval superiority in the Baltic region, an alternative to a few big battleships like the Vasa could be the deployment of a larger number of smaller war vessels that can be created with the same total budget and within the same timeframe. Perhaps these vessels have less firepower but can cover a larger part of the Baltic region.

Architects can help make such cost/benefit analysis in IT projects or within the enterprise as a whole and align projects to the overall strategy.

If project managers say no to requirements it is usually because of time or budget constraints. Managers often need to balance quality, price and time. Besides budget and time one can also validate the realization of requirements based on compliance with architectural guidelines. If Java is the defacto standard then building a perfect PHP system that realizes all requirements and is delivered in time and within budget can still cause serious problems. For example when an enterprise doesn’t have the knowledge to correctly administer PHP systems and has a policy of not outsourcing system administration. Architects should look at the feasibility of requirements from the structures, principles, and boundaries they have set up in an architecture. Architects should propose alternatives that better fit the architecture. Architects should also stand up to managers if requirements don’t fit enterprise policies.

Also test architecture, not only functional and technical requirements
It is said that the stability test of the Vasa (men running up and down the deck) was cancelled shortly after it began because the ship almost tumbled over. Still construction and launch of the Vasa continued.

IT projects test systems to validate that functional and technical requirements are met. They furthermore test to verify that changes to the IT system do not introduce faults and cause regression. It is somewhat less obvious to test the architecture or architectural constraints and principles a IT system should adhere to. It can still be that all functional and technical requirements are met, but that e.g. applying changes after taking an IT system in production takes way too long. This can be an indicator of a flaw in the IT systems architecture that has big impact.

For example, a solution architecture could state that services should be reusable by future consumers. At first these services could only be used by a single consumer. We can actually test if these services are reusable (or whether they might be too specific to be used by future consumers) by investigating the service contract and interface.

To summarize: good architecture helps making IT projects successful, good design alone isn’t enough, we can learn lessons from other fields for applying architecture, and finally make sure you visit the Vasa museum when you’re in Stockholm!