Using External REST Services with JDeveloper Part 3

In this blog we look how we can use an external REST service with JDev 12.2.1.2. To make things more interesting we don’t use an ADF based REST service and we look how to get nested data into the UI.

For this sample we like to create an application which allows to search for music tracks and show the results in a table or listview. To get the music data we use a REST service and to display the data we use ADF faces application.

In Part 1 we create the application and the project for the REST Data Control. In Part 2 we started creating the UI using the REST Data Control. In this final part we are enhancing the UI by using nested data from the REST Web Service and add this as column to the search result table. Before we start we look at the use case again.

Use Case

Before we begin implementing something which uses the external REST service we have to think about the use case. We like to implement a music title search using the external MusicBrainz REST service. A user should be able to enter a music title or part of a music title and as a result of the search she/he should get a list of titles, the artist or artists, the album and an id.

 

Handling nested Data

The use case demands that we add the artist and the album the music track is on to the same table. A look at the table in it’s current layout, make this understandable.

First of all we need to identify the dat a we want to add to the table in the response we get from the service.

Let’s investigate the JSON data, part of it, we get from the service for the search for the track ‘yesterday’


 

{
   "created": "2017-08-02T12:42:48.815Z",
   "count": 5985,
   "offset": 0,
   "recordings": [
       {
           "id": "465ad10d-97c9-4cfd-abd3-b765b0924e0b",
           "score": "100",
           "title": "Yesterday",
           "length": 243560,
           "video": null,
           "artist-credit": [
               {
                   "artist": {
                       "id": "351d8bdf-33a1-45e2-8c04-c85fad20da55",
                       "name": "Sarah Vaughan",
                       "sort-name": "Vaughan, Sarah",
                       "aliases": [
                           {
                               "sort-name": "Sarah Vahghan",
                               "name": "Sarah Vahghan",
                               "locale": null,
                               "type": null,
                               "primary": null,
                               "begin-date": null,
                               "end-date": null
                           },
...
                       ]
                   }
               }
           ],
           "releases": [
               {
                   "id": "f088ce44-62fb-4c68-a1e3-e2975eb87f52",
                   "title": "Songs of the Beatles",
                   "status": "Official",
                   "release-group": {
                       "id": "5e4838fa-07f1-3b93-8c9d-e7107774108b",
                       "primary-type": "Album"
                   },
                   "country": "US",

I marked the info ne need in blue in the data above. We see that the artist name is inside a list of name ‘artist_credit’ and that there can be multiple artists inside the ‘artist_credit’. This is a typical master/detail relationship.

The same is true for the album name which is an attribute inside a list of ‘releases’. The big question now is how do we get the nested data into the table as column.

When we expand the MusicBrainz Data Control we see the same structure we saw in the JSON data

So, the data is there, we only need to get to it. The data is structured like a tree and ADF is capable of accessing data in a tree structure (e.g. using an af:tree component). However, we like to use a simple table and don’t want to use a af:tree or af:treeTable. To get to the data, we first have to add the nested structure to the recordings binding we already use to display the current two columns of the table.

Right now we see the first level of the tree, the ‘recodrings’. Click the green ‘+’ sign to add the next level ‘artist_credit’

Add all attributes to the right side

As the artist name is still one level down, click the green ‘+’ sign again and add the ‘artist’ level

And shuffle the id and name attribute to the right side

Finnally we need to add the ‘releases’ level to get to the album name. For this select the ‘recordings’ level (the first) and click the green ‘+’ sign

And shuffle the id, title and track_count to the right side

Now all related data we need can be accessed via the ‘recordings’ binding.

We start with the artist column. Select the af:table in the structure window and open hte properties window

Click the green ‘+’ sign twice in the columns section to add two columns

Select the first added column (score in the image) and change the display label to ‘Artist’ and the component To Use’ to ‘ADF Output Text’. The second added column we change the display label to ‘Album’ and the ‘Component To Use’ again to ‘ADF Output Text’

We change the ‘Value Binding’ in the next step.

To get to the data for the artists we need to traverse two levels of sub rows. First level is the ‘artist_credit’, the second level is the artist itself. Here we have to keep in mind, that there can be more than one artist. In this case we have to join the names into one string for the table. As the ‘artist_credit’ itself can occur more than once, at least that’S what the data structure is telling us, we use an iterator to get the data.

The value property points to the current row and selects the ‘artist_creadit’. Each item we get from this iterator we access via the var property. So the item inside the iterator can be addressed as ‘artists’.

The artists can be one or more so we need another iterator to get to the artist data.

<af:iterator id="i2" value="#{artists.artist}" var="art" varStatus="artStat">

The value property for this iterator points to the artist we got from the outer iterator and is addressed as #{artists.artist}. To access attributes inside the artist data structure we use the var property and set it to ‘art’.

Now we have to somehow joint multiple artist names together if a track has more than one artist. The MusicBrainz Web Service helps us here by providing a ‘joinphrase’ which can be used to build one string for all artists. This ‘joinphrase’ can be .e.g a ‘&’ or a ‘,’. The full column code for the artist looks like

<af:iterator id="i2" value="#{artists.artist}" var="art" varStatus="artStat">

Here is some sample data for a search for the track ‘Something Stupid’ (to make it more readable I removed some attributes

"recordings": [
 {
  "title": "Something Stupid",
  "artist-credit": [
   {
    "joinphrase": " duet with ",
    "artist": {
     "name": "The Mavericks",
    }
   },
   {
    "joinphrase": " & ",
    "artist": {
     "name": "Raul Malo",
    }
   },
   {
    "artist": {
     "name": "Trisha Yearwood",
    }
   }
 ]

This data will be translated into the artist: “The Mavericks duet with Raul Malo & Trisha Yearwood”.

For the album column it’s easier. This too needs an iterator, but we don’t have to go down another level and we don’T have to join the data we get from the iterator. The column code for the album looks like

<af:iterator id="i1" value="#{row.artist_credit}" var="artists">
 <af:iterator id="i2" value="#{artists.artist}" var="art"
                    varStatus="artStat">
   <af:outputText value="#{art.name}#{artists.joinphrase}" id="ot5"/>
 </af:iterator>
</af:iterator>

The whole table for the search results look like

With this the page is ready and we can run the application. After start we see the page

Now entering a search term ‘something stupid’ into the search field will show

or trying the search with ‘dave’ will show

This concludes this mini series about how to use external REST Services and build an ADF UI from it.

The source code for this sample can be loaded from GitHub BlogUsingExternalREST. The sample was done using JDeveloper 12.2.1.2 and don’t use a DB.

Advertisements

Using External REST Services with JDeveloper (Part 2)

In this blog we look how we can use an external REST service with JDev 12.2.1.2. To make things more interesting we don’t use an ADF based REST service and we look how to get nested data into the UI.

For this sample we like to create an application which allows to search for music tracks and show the results in a table or listview. To get the music data we use a REST service and to display the data we use ADF faces application.

In Part 1 we create the application and the project for the REST Data Control. In Part 2 we start create the UI using the REST Data Control. Before we start we look at the use case again.

Use Case

Before we begin implementing something which uses the external REST service we have to think about the use case. We like to implement a music title search using the external MusicBrainz REST service. A user should be able to enter a music title or part of a music title and as a result of the search she/he should get a list of titles, the artist or artists, the album and an id.

Implementing the UI

In Part 1 we implemented the REST Data Control which we now use to build a small UI. Let’s look at the REST Web Service Data Control in the JDeveloper IDE

Above we see the data control ‘MusicBrainzJSONDC’ with it’s only resource recording, the input parameter names ‘query’ and the return data structure which was created using the sample JSON data we used when creating the REST Web Service Data Control.

When we query the resource we get back a complex data structure which give us information about how many results where found for the query and a list of ‘recordings’ which holds the artist names and the album names as ‘releases’.

To build the result table which should show the title id, the artist or artists and the album we have to go through all the nested data.

Setting up the search page

We start by adding a view the unbounded task flow adfc-config.xml which we name ‘MunicBrainz’ and create the page with a quick layout from the list

Make sure that you have selected to use ‘Facelets’! This will create a starter page with the layout selected. When the page is created it opens up in JDev like

We add an outputText component to the header and set the value to ‘MusicBrainz Test’

The resulting code looks like

For the layout we want to archive (search part and table to show the results) we need another grid row in the panelGridLayout. We drag a GridRow component from the ‘Component palette’ onto the panelGridLayout component in the structure window. You can use the source window too if you like. Dropping a new gridRow in the design isn’t recommended as it’s difficult to control the point where to insert the component.

Now we adjust the height of the rows and set the first row to 50 pixel, the second one to 100 pixel and leave the remaining height to the third gridRow:

Next we add the panelFormLayout holding the search field and the button to search for music tracks. For this we simply drag the ‘recording(String)’ operation from the MusicBrainzJSONDC data control onto the second grid row and drop it as ‘ADF Parameter Form’

we get a dialog showing us the methods parameter. Here we can bind the field to any other data field we like. However, in this case we leave it as is and just click OK

The framework wires everything up for us and we get the page as

Here we change the text on the button to ‘Search’

To see how things are wired up we look at the pagedef for the page

Here we see the method ‘recording’ and can expand it by clicking on the pencil icon

Where we see the details like where the parameter ‘query’ gets it’s value from (#{bindings.query.inputValue}). The ‘query’ binding is defined right above the recording method:

When we select the binding for ‘query’ wee see that the binding points to a variable defined in the pagedef (see Creating Variables and Attribute Bindings to Store Values Temporarily in the PageDef) which holds the value the user enters into the field. The recordings binding and the other stuff we talk about later.

Next up is creating the table with the results returned from the method call. For this we drag the recordings from the methodReturn binding onto the page and drop it as ADF Table into the third gridRow

To get the next dialog

Where we remove every attribute but the ‘id’ and the ‘title’ by selecting the rows and clicking the red ‘x’ icon. We set the row selection to single and make the table ‘read only’

The resulting page looks like

If we run the application now the UI comes up, but we’ll get an exception

Why’s that?

If we look into the servers log we see the error:-


<oracle.adf.view> <Utils> <buildFacesMessage> <ADF: Adding the following JSF error message: JBO-57001: Invocation of service URL used in connection failed with status code 400 Unable to parse search:tnum:.> 
oracle.adf.model.connection.rest.exception.RestConnectionException: JBO-57001: Invocation of service URL used in connection failed with status code 400 Unable to parse search:tnum:.
    at oracle.adf.model.connection.rest.RestConnection.getResponseCheckingStatus(RestConnection.java:783)
    at oracle.adf.model.connection.rest.RestConnection.getResponse(RestConnection.java:629)
    at oracle.adfinternal.model.adapter.ChildOperation.getJerseyResponse(ChildOperation.java:1167)
    at oracle.adfinternal.model.adapter.ChildOperation.makeServerCall(ChildOperation.java:977)
    at oracle.adfinternal.model.adapter.JSONChildOperation.invokeOperationInternal(ChildOperation.java:2056)
    at oracle.adfinternal.model.adapter.ChildOperation.invokeOperation(ChildOperation.java:542)
    at oracle.adf.model.adapter.rest.RestURLDataControl.invokeOperation(RestURLDataControl.java:247)
    at oracle.adf.model.bean.DCBeanDataControl.invokeMethod(DCBeanDataControl.java:512)
    at oracle.adf.model.binding.DCInvokeMethod.callMethod(DCInvokeMethod.java:269)
    at oracle.jbo.uicli.binding.JUCtrlActionBinding.doIt(JUCtrlActionBinding.java:1742)
    at oracle.adf.model.binding.DCDataControl.invokeOperation(DCDataControl.java:2371)
    at oracle.adf.model.bean.DCBeanDataControl.invokeOperation(DCBeanDataControl.java:628)
    at oracle.adf.model.adapter.AdapterDCService.invokeOperation(AdapterDCService.java:316)
    at oracle.jbo.uicli.binding.JUCtrlActionBinding.invoke(JUCtrlActionBinding.java:803)
    at oracle.jbo.uicli.binding.JUMethodIteratorDef$JUMethodIteratorBinding.invokeMethodAction(JUMethodIteratorDef.java:175)

 


Which doesn’t tell us more. What we see is that an ‘invokeMethod’ is the root cause. The reason is that when the pages loads, the iterators in the executable section of the pagedef are fired. As we saw we have two executables and those are giving us the errors.

As the field is empty the recordings method is called without a query parameter. If you mimic this in Postman with the query

http://musicbrainz.org/ws/2/recording/?fmt=json&query=

we get

Exactly the same error, only this time as html.

To solve this problem we have to avoid calling the service without a parameter. This can easily be archived by adding an expression to the executable RefreshCondition property

This we have to both executables in the pagedef. After that running the application will get us

 

This ends part 2 of this series, due to the length and the number of images in this post. The remaining part 3 will cover how to use the nested data and to add it to the search result table and provide the link to the sample application.

Blog Using External REST Servies (Part 1)

Using External REST Services with JDeveloper 12.2.1.2 (Part 1)

In this blog we look how we can use an external REST service with JDev 12.2.1.2. To make things more interesting we don’t use an ADF based REST service and we look how to get nested data into the UI.

For this sample we like to create an application which allows to search for music tracks and show the results in a table or listview. To get the music data we use a REST service and to display the data we use ADF faces application.

In part 1 we create the application and the project for the REST Data Control. Part 2 we will create the UI using the REST Data Control.

Setting up an external REST services

Let’s start be selecting a REST service which is available for public use without the need to get a key first. We use such a service to make it easier for you to run the sample and to look at the code. If we would need a key to use the API, you would need to register yourself with the service before you can run the sample.

There are a couple of such REST services like Spotify, iTunes or MusicBrainz which offer search APIs for music data as public REST service. Spotify we have to eliminate from the list as this service requires an API key since Mai 2017, meaning that it’s not public available without you register yourself before using it. ITunes REST API allows public access and the data structure returned is very simple. The result for a search get you everything in a flat structure. This will make things easy, too easy 🙂

For this sample where we like to show how to work with more complex data structures returned by a REST service. So, the final vote for this blog goes to MusicBrainz (MusicBrainz Rest API).

MusicBrainz

Musicbrainz REST API comes in different versions (V1 and V2). The current version v2 is what we are interested in as V1 is already deprecated. The documentation tell us, that the service is an XML style REST service. However, there is a JSON style REST service available too. This JSON style RSET service is what we use for the sample.

Before we implement the REST service calls we need to find out how to search for the data we like to show. For this a tool like Postman is a great help. Postman allows you to enter calls to REST services in a browser like UI. You can set all kind of headers, e.g. below we see a sample of the Postman UI (in the result a couple of sub structures are folded to show the relevant data). The query searched for recordings named ‘yesterday’ and asked for a result in JSON format:

To learn more about the possible searches refer to Web Service Search.The data structures and their meaning are described in in the MusicBrainz Data Structure.

Use Case

Before we begin implementing something which uses the external REST service we have to think about the use case. We like to implement a music title search using the external MusicBrainz REST service. A user should be able to enter a music title or part of a music title and as a result of the search she/he should get a list of titles, the artist or artists, the album and an id.

Creating a REST Web Service Project

After we looked at the REST Service and the data it returned we have identified the data we need to get from the REST service. The first step is to create a project which communicate with the REST service.

We create a normal Fusion Web Application which will create a ADF model project and a view controller project. If you need a script on how to do this you can look at Writing Reproducible Test Cases: Why and How. The model project we don’t need for this sample. You can delete it or just leave it empty.

For the access to the MusicBrainz data we create a new REST Web Service Project inside the application:

Name it and go through the rest of the wizard

Before we create the web service data control, we need to create a REST Connection from the resource pallete we create a new IDE connection of REST type

Then we later need data returned from the REST service which the wizard uses to produce the data structure. A simple way to get such data is to use e.g. Postman to call the REST service:

Copy the result (all of it!) and save it to a file. Next we create a new Web Service Data Control from the gallery

Select the ‘WebService Data Control (SOAP/REST)’

And fill out the wizard. Select the REST connection created before

In the next image click on the green ‘+’ sign

And change the path to ‘/ws/2/recording’, select JSON as data format and checkmark the GET method to enter ‘recording’ into the field.

In the next screen we need to select ‘Parse from Sample Code’ and copy the content of the file we saved from Postman into the textarea

And finally test the Web Service Data Control

The finish the wizard. Now we can test the data contron by finding the DataControl.dcx file in the project and right click on it. Choose ‘Run’ from the context menu:

In the dialog window right click hte data control and choose ‘Operations’

Fill in the fields and click the execute button

The result should look like

You can copy the return value into an editor to fully see it. If you don’t get the successful result, check the steps against the ones in the blog.

This concludes part 1 of this series. In part 2 we develop the UI for the application using the Web Service Data Control we created in this part. The source of the sample can be downloaded from GitHub. The link to it will be provided with part 2.

Enable Oracle JCS to access External REST API

Enable Oracle JCS to access External REST API

For a training I’m preparing I had to implement a POC on how to access an external REST API and to make it available in an ADF application running in the Oracle Java Could Service.

This task sounds pretty easy, nevertheless it’s best to see this working before starting a training which in the end will not work.

I decided to use Spotify public available REST API at https://api.spotify.com for this task. I started by creating a simple Fusion Web Application using JDev version 12.2.1.2.0. To this project I added a custom model Project which I later used to add the REST DataControl pointing to the Spotify track search API.

As this post isn’t about how to create such a project and use it in your normal ADF application, I spare the details here and write this up in another blog post.

Implementing a REST DataControl is pretty straight forward and the sample application was set up quickly. The first problem you might run into, even on the local development machine is this

null

error you get when you try to access the public API from inside the WebLogicServer. To make this error searchable for other users here is a part of the stack trace

javax.net.ssl.SSLKeyException: Hostname verification failed: HostnameVerifier=weblogic.security.utils.SSLWLSHostnameVerifier, hostname=api.spotify.com.
at weblogic.security.SSL.jsseadapter.JaSSLEngine.doPostHandshake(JaSSLEngine.java:686)
at weblogic.security.SSL.jsseadapter.JaSSLEngine.doAction(JaSSLEngine.java:757)
at weblogic.security.SSL.jsseadapter.JaSSLEngine.unwrap(JaSSLEngine.java:133)
at weblogic.socket.JSSEFilterImpl.unwrap(JSSEFilterImpl.java:644)
at weblogic.socket.JSSEFilterImpl.unwrapAndHandleResults(JSSEFilterImpl.java:541)
at weblogic.socket.JSSEFilterImpl.doHandshake(JSSEFilterImpl.java:99)
at weblogic.socket.JSSEFilterImpl.doHandshake(JSSEFilterImpl.java:78)
at weblogic.socket.JSSESocket.startHandshake(JSSESocket.java:240)
at weblogic.net.http.HttpsClient.New(HttpsClient.java:574)
at weblogic.net.http.HttpsClient.New(HttpsClient.java:545)
at weblogic.net.http.HttpsURLConnection.connect(HttpsURLConnection.java:236)
at weblogic.net.http.HttpURLConnection.getInputStream(HttpURLConnection.java:685)
at weblogic.net.http.SOAPHttpsURLConnection.getInputStream(SOAPHttpsURLConnection.java:41)
at weblogic.net.http.HttpURLConnection.getResponseCode(HttpURLConnection.java:1545)
...

This problem can easily sorted out by changing the ‘Hostname Verification’ to custom and to specify ‘weblogic.security.utils.SSLWLSWildcardHostnameVerifier’ as ‘Custom Hostname Verifier’. Here are the detailed steps:

  1. Go to the WebLogic admin console -> Environment -> Servers -> Server -> Configuration -> SSL
  2. Under advanced options , change “Hostname Verification” from “BEA Hostname Verifier” to “Custom Hostname Verifier”.
  3. Set “Custom Hostname Verifier” to weblogic.security.utils.SSLWLSWildcardHostnameVerifier
  4. Click “Save” and then “Activate Changes”
  5. Restart your server.

After this the sample application will run on the local development machine or an stand alone webLogic Server.

Now the fun part begins, making the application run in the Oracle JCS. Here I started with setting up a new Java Cloud Service with a WebLogic Server of version 12.2.1.2 and deployed my local running application to this new JCS.

The application did start OK, but when I came to the point where the application tried to call the external REST API all I got is: NOTHING

I only saw a spinning cursor, no error message on the UI regardless of my exception handler. It turned out, that I did not wait long enough for the error message to come up. The REST call timed out eventually providing more info in the servers log file

Exception in invoking HTTP method GET from Rest data control. Cause: javax.ws.rs.ProcessingException: java.net.ConnectException: Tried all: 3 addresses, but could not connect over HTTPS to server: api.spotify.com port: 443

It looks like the external REST call is not allowed. Here is the REST call

https://api.spotify.com/v1/search?q=sorry&type=track

If you write this into your browser you’ll get a JSON string in return, something like

{
 "tracks" : {
 "href" : "https://api.spotify.com/v1/search?query=sorry&type=track&offset=0&limit=20",
 "items" : [ {
 "album" : {
 "album_type" : "album",
 "artists" : [ {
 "external_urls" : {
 "spotify" : "https://open.spotify.com/artist/1uNFoZAHBGtllmz…

telling me that the problem is somewhere with the JCS. I ask for help in the OTN Java Cloud Service space and got an answer that there must be a rule missing. This is true to some point as I tried to access the REST API with curl from the JCS command shell

null

which did not work either. The interesting part is that you can do the same from the DBCS command shell and get the right answer.

I tried to add a rule to allow the access but looking at the possible source and destinations lists this did not work. Only

  • OTD — The Oracle Traffic Director load balancer VMs
  • WLS_ADMIN_SERVER — The WebLogic Server Administration Server VM
  • WLS_MANAGED_SERVER — The WebLogic Server Managed Server VMs

are allowed as destinations and my rule should allow the managed server (source) to access the PUBLIC_INTERNET or allow access to https protocol port 443.

After some more reading and testing I found a solution, however I’m not sure if this is the best way to handle this. Anyway, for others users who run into the same problem here it is:

First you have to create a ‘Security List’ Which you name e.g. outbound_wlsms_https_traffic which denies incoming packages and allows outgoing packages

null

Next a ‘Security Rule’ can be created like

null

with source set as the managed server and destination the new security list. After that the access to the external REST API works.

Summary of Day 4 at the Oracle Open World 2016

Late, but not forgotten, here is the summary of day four. It was too late yesterday, after the appreciation event to write it all down,

Wednesday was a somehow slow day for me as I attended two sessions only. Most of the day was reserved for meetings around my other activities in the OTN network like moderation and the German ADF Community which will soon relaunch their community page on OTN.

The first session was about testing web applications with Selenium ‘Testing Java Web Applications with Selenium: A Cookbook‘ by Jorge Hidalgo and Vicente Gonzalez Arellano, over at the Java One. It turned out that the Selenium Webdriver for JDev ADF is better working than the one showed in the demo in this session. The JDev Webdriver abstracts all the tricky stuff like waiting for ajax calls or finding the right component away from the developer. This make the job really easy. Summary: nothing new learned.


After a nice working lunch with my peer OTN moderators and the Queen of Moderators I attended a session about developing applications with Oracle JET and ADFbc REST services ‘Oracle Application Development Framework and Oracle JavaScript Extension Toolkit in the Cloud‘ by Sherry Yu, Shray Bansal and Abhinav Shroff. This session was interesting to see as it used REST services generated from ADFbc. This kind of REST services offer many usages

ADFbc REST Services Usage

ADFbc REST Services Usage

and allow some very nice features out of the box like pageable collections, rich set of meta data, list of values, attribute types and validation and resource discovery

ADFbc REST Services Functions

ADFbc REST Services Functions

During run time you can tailor the payload by only retrieving the attributes you need, execute batch transactions, sort the results and have build in security.

ADFbc Run Time Features

ADFbc Run Time Features

Simple queries can be added to the REST calls. These are working like ‘Query by Example’ in ADF tables. This set of features allows for many different use cases

ADFbc REST Use Cases

ADFbc REST Use Cases

like back end for OracleJET based applications, mobile friendly UIs, integration with other services and as REST solution for SaaS.


The remaining part of the day I spend on multiple events like hte OTN Blogger Meetup, OTN Happy Hour and finally the Oracle Appreciation Event featuring Sting and Gwen Stefani.