Developing geometry-based Web Services for WebLogic | Part 3

Part 1 of this blog gives an overview of the geometry-based Web Services we developed. Part 2 dives into some of the interesting details for the implementation of such Web Services. This part discusses the deployment specifics of such Web Services.

Deployment

There is a catch to deploying an application to WebLogic that uses the EclipseLink StructConverter to convert geographical data into JGeometry objects. When packaging the dependent JAR files sdoapi.jar en sdoutl.jar into the application (e.g. WAR or EAR file) you get the following error when running the application:

java.lang.NoClassDefFoundError: oracle/spatial/geometry/JGeometry at org.eclipse.persistence.platform.database.oracle.converters.JGeometryConverter.<clinit>(JGeometryConverter.java:33)

This issue is caused by classloading specifics of EclipseLink on Oracle WebLogic. You can resolve this by adding the sdoapi.jar and sdoutl.jar files to the system classpath of WebLogic: 

• Copy the JAR files to the [WEBLOGIC_HOME]/wlserver_10.3/server/lib directory; 
• Add these JAR files to the WEBLOGIC_CLASSPATH variable in the commEnv.cmd or commEnv.sh file located in [WEBLOGIC_HOME]/wlserver_10.3/common/bin;
• Restart the WebLogic Managed Server after these steps.

Also see the following OTN forum post for the NoClassDefFoundError error.

We use Maven to build the Web Service. For EclipseLink you can add the following repository to the pom.xml file:

<repository>
 <id>EclipseLink</id>         <url>http://download.eclipse.org/rt/eclipselink/maven.repo</url>
</repository>

For the Oracle Java Spatial libraries you can add the following dependencies. You can place the associated JAR files into your local Maven repository:

<dependency>
  <groupId>com.oracle</groupId>
  <artifactId>ojdbc6</artifactId>
  <version>11.2.0</version>
  <scope>provided</scope>
</dependency>
<dependency>
  <groupId>com.oracle</groupId>
  <artifactId>sdoutl</artifactId>
  <version>11.2.0</version>
  <scope>compile</scope>
</dependency>
<dependency>
  <groupId>com.oracle</groupId>
  <artifactId>sdoapi</artifactId>
  <version>11.2.0</version>
  <scope>compile</scope>
</dependency>

Summary

This blog series shows how to create geometry based Web Services that can be deployed on Oracle WebLogic Server. Oracle provides functionality in several of their products and frameworks to ease development of such applications. These include geographical data types in the Oracle Database and specific geographical converters in EclipseLink.

Developing geometry-based Web Services for WebLogic | Part 2

Part 1 of this blog gives an overview of an end-to-end example of a geometry-based Web Service. This part dives into some of the interesting details for the implementation of such Web Services. The details are discussed per component that was named in part 1.

Database schema

This one is pretty straightforward. When defining a table column that needs to store geographical data, you can use the MDSYS.SDO_GEOMETRY type. For example:

CREATE TABLE RESIDENCE(
  RESIDENCE_ID INT NOT NULL,
  NAME VARCHAR2(100) NOT NULL,
  GEOMETRY SDO_GEOMETRY,
  CONSTRAINT PK_RESIDENCE PRIMARY KEY (RESIDENCE_ID)
);

You can use the SDO_UTIL package to insert GML into SDO_GEOMETRY types using the SDO_UTIL.FROM_GMLGEOMETRY and SDO_UTIL.FROM_GML311GEOMETRY functions.

See the Oracle Spatial Developer's Guide for more information on the SDO_GEOMETRY type.

ORM layer

In the ORM layer we map database table rows to POJOs and vice versa using JPA. JPA implementations such as EclipseLink provide out-of-the-box mappings between most common Java data types and database columns. To map more exotic and user-defined Java objects you can use Converters in EclipseLink. You can either use an out-of-the-box converter that is shipped with EclipseLink, or code one yourself by implementing EclipseLink's Converter interface. For more information, see this blogpost by Doug Clarke.

In this case we need to map the SDO_GEOMETRY database objects to some sort of Java geometry object. Luckily, EclipseLink ships with an out-of-the-box Converter that maps the SDO_GEOMETRY type to a JGeometry object. The JGeometry class provides all kinds of convenience methods and attributes for working with geographical data. This class is part of the Oracle Spatial Java functionality. It can be used only for Oracle Spatial's SQL type MDSYS.SDO_GEOMETRY and supports Oracle JDBC Driver version 8.1.7 or higher.

To implement the mapping for geographical data we need to do the following:

• Add the required JARs to the classpath; 
• Annotate the JPA entities and attributes. 

The JGeometry class and associated Java classes are contained in the sdoapi.jar and sdoutl.jar files. They can be found in the library directory of your Oracle RDBMS installation. Also add the ojdbc JAR to the classpath.

Add a Convert annotation to the geometry attributes in your JPA entities that need to map to the SDO_GEOMETRY database types:

@Column
@Convert("JGeometry")
private JGeometry geometry;

Next, add the StructConverter annotation to the JPA entities containing geometry attributes. The StructConverter is a specific type of EclipseLink converter that provides out-of-the-box mappings to Oracle RDBMS struct types.

@Entity
@Table(name = "RESIDENCE")
@StructConverter(name = "JGeometry", converter = "org.eclipse.persistence.platform.database.oracle.converters.JGeometryConverter")
public class Residence implements Serializable

The org.eclipse.persistence.platform.database.oracle.converters.JGeometryConverter provides the actual mapping logic. The name attribute of the StructConverter needs to be same as the attribute value for the Convert annotation. 

Web Service layer

Since GML is an XML format we can use JAXB to generate Java classes for the GML elements that are part of the input and output values of the Web Service operations. There are several ways to generate the JAXB classes including Maven plugins for JAXB or the command-line tool xjc. A simple example of running xjc is shown by the following command:

xjc -d [target dir of generated classes] [XSD root directory] 

In our use case, we had a predefined Web Service interface and used a top-down approach to generate Java classes based on the existing interface. You can use the wsimport tool to generate the JAX-WS artifacts including the Java WebService class from the WSDL.

Note that in this end-to-end scenario the service is exposed as SOAP Web Service. It is simple to expose the same functionality as a RESTful service. You can use JAX-RS annotations instead of JAX-WS annotations to create a RESTful service that exposes geographical data in GML format. See the following example that shows how JPA and JAX-RS can be combined to create RESTful services.

Business logic layer

This layer, among others, provides the logic to map between the JAXB generated classes for the GML elements and the JGeometry objects.

For the conversion from JGeometry objects to JAXB generated classes for GML elements this involves:

• Use the static methods of the oracle.spatial.util.GML3 class to generate a String containing the textual GML representation of the geographical object;
• Unmarshall the GML String into the JAXB generated classes.

This is shown in the following code snippet:

JAXBContext jaxbContext = JAXBContext.newInstance("net.opengis.gml");

Unmarshaller unmarshaller = jaxbContext.createUnmarshaller();

JAXBElement jaxbGeometry = null;

String gml = GML3.to_GML3Geometry(jGeometry);

ByteArrayInputStream bais = new ByteArrayInputStream(gml.getBytes());

JAXBElement jaxbGeometry = (JAXBElement) unmarshaller.unmarshal(bais);

The GML3 class and supporting code can also be found in the sdoapi.jar and sdoutl.jar files.

For the conversion from JAXB generated classes for GML elements to JGeometry objects you need to retrieve the geographical data from the GML elements and use the static methods of the JGeometry class to instantiate the proper JGeometry object. For example:

JGeometry geometry = JGeometry.createLinearPolygon(coordinates, spatialDimension, spatialReference);

Read about the deployment specifics for this geometry-based Web Service on Oracle WebLogic Server in part 3 of this blog.

Developing geometry-based Web Services for WebLogic | Part 1

In a recent project we developed Web Services that expose geographical data in their operations. This blog explains the use case for the service, gives an overview of the software architecture, and briefly discusses GML as markup language for geographical data. Part 2 of this blog provides pointers on the implementation of the service while part 3 discusses the deployment on Oracle WebLogic Server.

Use Case

The "BAG" (Basisregistratie Adressen en Gebouwen) is a Dutch national database containing information on all addresses and buildings in the Netherlands, and is maintained by Dutch municipalities. For several object types the BAG also maintains the associated geographical location and shape; for example for premises and cities.

Often organizations have their own GIS/GEO systems and databases for analysis and decision-making purposes. In this particular project we created Web Services to retrieve and modify the geographical data in these GIS/GEO systems based on updates from the national BAG database.

There are of course numerous use cases for geo-based services such as creating mashups with maps and viewers, and offering geo-based public APIs for your customers to integrate with.

Software Architecture

The following figure shows an overview of the software architecture for the Web Services we developed.

Overview of the software architecture for the Web Services 

These services consist of the following components:

• Database schema containing the geographical, and related administrative data. The schema is located in an Oracle RDBMS 11g Enterprise Edition instance. The geometry is stored as SDO_GEOMETRY type; an Oracle spatial object type that provides methods for storing, accessing, and altering geographical attributes.
• Object-Relational Mapping (ORM) layer that provides access to the geographical data by converting SDO_GEOMETRY database object types into JGeometry Java objects. The persistency layer is implemented using the JPA (Java Persistence API) standard and EclipseLink as persistency provider. JGeometry is a Java class provided by Oracle as part of the Oracle Spatial Java functionality.
• Web Service layer that exposes the service and its operations to consumers using SOAP. The operations expose geographical elements as GML (Geography Markup Language) data types. GML is an XML grammar for expressing geographical features that is maintained by the Open Geospatial Consortium (OGC). The Web Service layer is implemented using JAX-WS and JAXB.
• Business logic layer that contains Java logic for validation and transformation. As part of the transformation logic, this component converts GML elements into JGeometry objects and vice versa.

GML

The following XML snippet shows an example of a GML element. Such elements are part of the input and output of the Web Service operations. In this case, the element denotes a single polygon containing a hole. The polygon is defined by the exterior element, and the hole is defined by the interior element.

<gml:Polygon srsName="urn:ogc:def:crs:EPSG::28992" xmlns:gml="http://www.opengis.net/gml">
  <gml:exterior>
    <gml:LinearRing>
      <gml:posList srsDimension="2">82862.708 436122.616 ... (more coordinates) ... 82862.708 436122.616</gml:posList>
    </gml:LinearRing>
  </gml:exterior>
  <gml:interior>
    <gml:LinearRing>
      <gml:posList srsDimension="2">82832.967 436145.273 ... (more coordinates) ... 82832.967 436145.273</gml:posList>
    </gml:LinearRing>
  </gml:interior>
</gml:Polygon>

Note the following:

• There are several versions of GML available. The accompanying XML Schemas for GML can be found on the OGC website.
• GML doesn't define a default Coordinate Reference System (CRS) from which the absolute position of the defined GML elements can be determined. Instead you have to define what coordinate reference system should be used. This is done by specifying the srsName attribute. In the above example, the EPSG::28992 (European Petroleum Survey Group) reference system is used which has the Dutch city of Amersfoort as reference for GML elements. You can find several reference systems at http://spatialreference.org and http://www.epsg-registry.org. 
• You need to define the dimension of the GML elements using the srsDimension attribute. In the above example all elements are two-dimensional.
• GML defines several geographical types. In our use case, a residence or municipality is represented by a point, simple polygon, or multi surface. A multi surface can include multiple polygons, for example when defining two buildings that are not attached but together form one residence. The available geographical types for GML can be found on the OGC website as well.

Part 2 of this blog will dive into some interesting implementation aspects of the Web Service.

SOA Made Simple

The book SOA Made Simple is published! SOA Made Simple is written by Vennster's Lonneke Dikmans and Ronald van Luttikhuizen. You can download a sample chapter and order the book from the Packt Publishing website. 

SOA Made Simple is a concise and indispensable handbook for finally understanding exactly what Service Oriented Architecture is. Split into three clear sections, in this book you’ll learn from both theory as well as step-by-step implementation examples to aid in your understanding of this often poorly- articulated industry term.

A short abstract for SOA Made Simple:

SOA is an industry term which is often preached like a religion rather than taught like a technology, and over time, grasping the concept has become unnecessarily difficult. Many companies proclaim that they don’t know where to begin with SOA, while others have begun their SOA effort but haven’t reaped the benefits they were convinced it would bring. “SOA Made Simple” explains what SOA is in simple terminology and by using real-life examples. Service-orientation is already a very natural way of thinking for business stakeholders that want to realize and sell services to potential clients, and this book helps you to realize that concept both in theory and practice. After reading “SOA Made Simple” you will have a clear understanding ofwhat SOA is so you can implement and govern SOA in your own organization. If you are an architect who wants to be completely clear in your understanding of what SOA is, then this book is essential. In fact, anyone (designer, developer, administrator or team lead) who is implementing or about to implement an architecture in an IT environment should not miss out on “SOA Made Simple”. 

From the Trenches 2 | Patching OSB and SOA Suite to PS5

In my previous blog I talked about a recent upgrade from Oracle Fusion Middleware 11g PS2 to 11g PS5. This blog continues with two post-installation steps that were required to complete the upgrade.

Patching Oracle B2B for ebMS-based services

In this particular project we implemented ebMS-based services using Oracle B2B and Oracle Service Bus 11g. Besides "plain" SOAP Web Services, ebMS is part of the Dutch government standards to exchange messages and expose services between organizations in the public sector. You can read more about implementing ebMS-based services using Oracle B2B, Oracle Service Bus and Oracle SOA Suite in this presentation.

The ebMS standard makes extensive use of SOAP headers to facilitate features such as guaranteed delivery and to avoid duplicate messages. The following snippet shows part of an ebMS message header.

One of the identifiers used in the ebMS message exchange is the manifest id. According to the ebMS specification maintained by OASIS this needs to be XML NCName type. This type has some restrictions; for example that its values cannot start with a number. In Oracle B2B 11g PS2 the manifest id value is prefixed with the text oracle. This prefix is removed in 11g PS5 resulting in the following error from the B2B trading partner at runtime:

09:06:00 Local Listener(8914) Result: "Error" "Fault sent:<SOAP:Fault><faultcode>SOAP:Client</faultcode><faultstring>org.xml.sax.SAXParseException: cvc-datatype-valid.1.2.1: '0A1A1A1A1A1A1A1A1A1A1A1A1A1A1A1A' is not a valid value for 'NCName'.</faultstring></SOAP:Fault>"

Luckily there is an easy-to-apply patch that solves this problem; see article 1497168.1 on Oracle Support. After applying the patch, the manifest id is prefixed with a text value again.

Changing namespaces in WSDLs and XSDs of JAX-WS Web Services

The environment that was patched contains several Java applications running on WebLogic Server. These applications expose Web Services using JAX-WS. A meet-in-the-middle approach was used to create them: the business logic implemented in Stateless Session Beans and JPA (EclipseLink) is integrated with the Java classes generated from the predesigned WSDLs and XSDs.

Depending on the developer that created the Web Service, deployment descriptors such as webservices.xml and weblogic-webservices.xml were added to the application. Descriptors are used for configuration, overriding default settings, and adding metadata. For Web Services this can be the endpoint, port configuration, linkage of the Web Service to EJB components, and so on. When deployed, the WSDL location of Web Services is listed in the WebLogic Console and the WSDL can be retrieved at runtime.

After the patch we noticed that these artifacts weren't identical to the original WSDLs and XSDs. More specifically, the namespaces of the XSD elements in the request and response message definitions were changed to the namespace of the service itself. At runtime however, the service accepted requests and responses as defined by the original contract. This makes it difficult to test these Web Services using clients that inspect the runtime WSDL; for example when creating a default project in soapUI.

This issue was resolved by removing the webservices.xml and weblogic-webservices.xml deployment descriptors from the Java archive and redeploying the Web Services to WebLogic Server. The WSDL that can be retrieved at runtime matches the original designed WSDL again.

Using custom code in Oracle Integrated SOA Gateway

In a previous post I discussed how to expose seeded APIs in Oracle EBS 12 as a Web Service.

But, as you can guess, seeded APIs are not always the easiest way to access EBS. For example, the PAY_ELEMENT.CREATE_ELEMENT_ENTRY procedure expects the programmer to take care of EBS specific arguments like element_link_id.
Our self service application has logic to submit, approve and check expenses. To do that, it uses employee ids and business rules about submitting, approving and checking steps. Users can view the state, get tasks assigned to them and can save their expenses in the application. This is setup using BPM, business rules, Java and ESB technologies. This application is blissfully ignorant about element_entry_link_ids, date tracking implementations etc. On top of that, EBS does not provide "select" APIs to get data from the application. We don't want to create SQL statements in our application, we like to hide the implementation and use APIs to communicate with EBS.

The good news is, Oracle E-Business Suite 12 provides a way to make custom code part of the integrated SOA Gateway.

This way you have the best of both worlds: monitoring and control of the services with the Integration repository and usable APIs for external applications. Let's see how that works.

Overview

The following steps have to be executed to accomplish this:

1. Write your custom API
2. Test it
3. Annotate Custom APIs
4. Validate Annotated Files
5. Upload Validated Files
6. Create Security Grants
7. Generate Web Services
8. Deploy Web Services

From: Integrated SOA Gateway Developer's Guide

Writing and testing PL/SQL code is beyond the scope of this blog and the last three steps were described in the previous post, so I will skip those steps here.

Documentation and roles
There are two roles involved: a system integration developer and a system integration administrator. The 'Integrated SOA Gateway Developer's Guide' describes how the annotations should be added to the package specification in the Chapter 'Creating and Using Custom Integration Interfaces'.  Appendix A 'Integration Repository Annotation Standards' describes the annotations in detail. The 'Integrated SOA Gateway Implementation Guide' describes how to validate and upload the files.

Annotate Custom APIs
A system integration developer annotates the API that needs to be loaded in the integration repository. The integration repository parses the source files to populate itself.
So to populate the integration repository with your own Custom interfaces, you only need to annotate the package specification, not the package body.  The syntax is similar to JavaDoc annotations: @NameSpace:TypeName keyString
First of all, you need to annotate the package, with a scope, the product name, the displayname and the category. Secondly, you annotate the procedures that you want to expose as a Web Service. The annotations for the procedure should be placed between the definition and ';'.

create or replace package xxgr_soa_pay_elements is
/* $Header: xxgr_soa_pay_elements $ */
/*#
* This custom PL/SQL package can be used to add pay elements. 
It hides some particulars of the PAY_ELEMENT_ENTRY_API
* @rep:scope public
* @rep:product PER
* @rep:displayname Create Pay element entry
* @rep:category BUSINESS_ENTITY CREATE_ELEMENT_ENTRY
*/
procedure create_element_entry
( p_employee_number        in number
, p_element_name           in varchar2
, p_element_eff_start_date in varchar2
, p_element_eff_end_date   in varchar2
, p_entry_type             in number
, p_input_value_name1      in varchar2
, p_entry_value1           in number
, p_input_value_name2      in varchar2 default null
, p_entry_value2           in number   default null
, p_input_value_name3      in varchar2 default null
, p_entry_value3           in number   default null
, p_date_earned            in varchar2 default null
, p_subpriority            in number   default null
, p_effective_start_date   out varchar2
, p_effective_end_date     out varchar2
, p_element_entry_id       out number
, p_object_version_number  out number
, p_create_warning         out number
)

/*#
* Use this procedure to create a single entry for an expense. NB
* this procedure needs to be optimized to use an array for the flex fields.
* Date formats are dd-MM-YYYY
* @param employee_number employee number that the pay element applies to
* @param element name name of the pay element, type of expense
* @param  p_element_eff_start_date  effective start date of the pay element. 
* @param p_element_eff_end_date effective end date of the pay element
* @param  p_entry_type a constant 'E' for earnings
* @param p_input_value_name1  label of the first entry
* @param p_entry_value1  value of the first entry, the amount that is expensed
* @param p_input_value_name2  label of the second entry optional
* @param p_entry_value2  value of the second entry, optional
* @param p_input_value_name3  label of the third entry, optional
* @param p_entry_value3 value of the third entry, optional
* @param p_date_earned  date that the value is earned. Optional
* @param p_subpriority priority of the pay element, optional
* @param p_subpriority            in number   default null
* @param p_effective_start_date returns the effective start date
* @param p_effective_end_datereturns the effective end date
* @param p_element_entry_id  the number of the pay element (expense) that is 
* created
* @param p_object_version_number the version number of the object
* @param p_create_warning flags whether warning is created.1=true, 0=false
* @rep:displayname Create a pay element (expense)
* @rep:category BUSINESS_ENTITY 
* @rep:scope public
* @rep:lifecycle active
*/
;
end xxgr_soa_pay_elements;

Validate Annotated Files
The integration repository administrator validates the annotated files before uploading them to the repository. This is done using the Integration Repository Parser, a standalone design time tool. While executing the parser, the annotated source files are validated based on the interface type supported for customization. If no error occurs, an Integration Repository loader file (iLDT) will be created. However, before you can start, you need to setup the environment: install Perl modules and possibly install some patches (depending on the EBS12.x version you are using).
Once you installed the necessary Perl modules, you execute the command:

$IAS_ORACLE_HOME/perl/bin/perl $FND_TOP/bin/irep_parser.pl -g -v
-username=<a fnd username> <product>:<relative path from product
top>:<fileName>:<version>=<Complete File Path, if not in currect
directory>

Upload Validated Files
If the annotated files were valid, they can be uploaded to the integration repository by someone with the rol integration repository administrator.

Not that for an object (or class) that is already present in the Integration Repository, the Integration Repository Loader program reloads the new definition of that object ONLY if the new version is greater than the current version. Make sure you increment the Header version of the target source file.
Note that there is no GUI for uploading validated files. You can yse Telnet to have command access to the Oracle E-Business Suite Release 12 instanc and use the following command to upload the iLDT file:

$FND_TOP/bin/FNDLOAD <db_connect> 0 Y UPLOAD $fnd/patch/115/import/wfirep.lct <ildt file>


Conclusion
We showed a PL/SQL example here, but the same procedure applies to Java classes, or other supported interface types. Check the developers guide and the implementation guide for a list of supported interface types.

As I stated in my previous post, in order to successfully interface with an Oracle E-Business Suite instance, it is critical to cooperate with the developers and administrators of the Oracle E-Business Suite. That way you ensure that the interfaces are both manageable for the people responsible for running and maintaining the Oracle E-Business suite implementation and maintainable and usable for the developers of the self service application. When executed this way, Oracle E-Business Suite becomes a first class citizen of the 'SOA-World' we live in nowadays.

Oracle, Webservices, and PL/SQL

There are many ways you can publish or consume Web Services or XML Documents in an Oracle environment. The other day an organization asked me to evaluate the solution they had chosen to expose PL/SQL packages as Web Service and to consume Web Services from PL/SQL. Not because they experienced problems, but because they wanted to know the 'current state of affairs' in Oracle and XML and evaluate if their way of solving this was the most efficient way.

Not surprisingly, the possibilities are endless, depending on your environment. In this blog, I have listed the possibilities I know of for PL/SQL and added some details about when you would like to use the feature.

Publishing Web services
The following frameworks are available to publish a PL/SQL procedure or function as a web service:

Framework	Infrastructure	Features
Native Database web services	Oracle Database 11g	Oracle XML DB 11g
PL/SQL Webservices	Oracle Applicatieserver	JPublisher, SQLUTL
Eclipselink-DBWS	JEE Applicatieserver	DBWSBuilder (DesignTime) JAX-WS (Runtime)
Oracle DB Adapater	Oracle SOA Suite	Toplink
Integrated SOA Gateway	Oracle E-Business Suite 12	Oracle SOA Suite 10g

The Integrated SOA Gateway is only a viable option, if you are exposing PL/SQL APIs that are part of EBS 12.

If you don't have an JEE Application server (for example because you are in a .NET programming environment), Native Database web services are a viable option.

Of course, you can also program a web service in .NET or Java and call the PL/SQL procedure or function using Object Relational mappings.

Consuming Web services
The other way around, calling a Web Service from the database is also a common use case if you have a lot of logic in your database.

Framework	Infrastructure	Features
PL/SQL	Oracle Database 11g	UTL_HTTP
Java	Oracle Database 11g	Java stored procedures, JAX-WS
JPublisher	Oracle Database11g	UTL_DBWS

Although I think that BPEL, Java and .Net frameworks are a better fit for this type of thing, it sometimes is not feasible to change the entire software architecture. Using a utility like UTL_DBWS and JPublisher is very convenient and less error prone than coding all the XML by hand in the database.
If you are planning to move to JEE or SOA, using PL/SQL Webservices, EclipseLink or the Oracle database adapter is a good step to start this endeavor.