Topic Archive: SOA

Is It Time To Stop Pushing the Browser Around?

Dietrich Kappe, Thursday, April 19, 2007 @ 7:38 am

In my first ever real grownup consulting engagement, back in 1990, I was called in to improve the data cleaning operations of a small testing company. This was back in the days when even companies heavily dependent on computer technology relied on a single "computer guy." This particular guy had rigged up a data cleaning system on a set of state-of-the-art 80386 machines. He had set up a folding table in one corner of the office ("Machine room? What's a machine room?") and taped "Do not touch!!!" signs to the monitors. The guy would take the tapes containing maybe 20-30MB of data and run then for 6 days through his cleaning environment. If something blew up or a new defect was identified in the source data, boom! another 6 days.

When I probed a little bit about what he was using to perform this magic, I found out, to my horror, that he had implemented the whole data cleaning shebang as a set of WordPerfect macros! Ouch! Talk about the wrong tool. I recommended they buy a little Sparcstation (sorry, x86 unix was kind of expensive and unreliable back then) and solved their problem with a modest amount of Perl4.'

So what does WordPerfect have to do with Javascript in the Browser? As people have been testing the limits of Ajax, we see more sophisticated SOA and messaging concepts and abstractions making their way into the picture. Coach Wei, a clever and insightful man, has an article up now on the Nexaweb's IMB (Internet Messaging Bus).

It's a clever concept of how to introduce MOM/ESB into a browser environment (Nexaweb also supports Java and Desktop clients, as you can see in the article above). I've implemented a number of distributed apps using JMS as well as a few using ESB's such as WebMethods, Tibco and Mule. It's an eleagant way to design apps, but requires a somewhat different approach and outlook than OOP (not that you can't include OO software in the mix as a component). See Enterprise SOA: Service-Oriented Architecture Best Practices for a pretty good read on what's involved in building SOA systems.

I checked out a few of the applications mentioned in his article, and they we slick, responsive, elegant. Nice work by the Nexaweb folks.

So again, what does this have to do with WordPerfect macros? Using a technology in an inappropriate way will come to bite you. All that infrastructure and encapsulation and tunneling comes at a price, espectially in a still somewhat wobbly execution environment like Javascript on the browser. Sure enough, those Nexaweb apps start out nice, but over time start to peg my CPU at 100% on both IE and Firefox. Don't even think of running more than one of these apps on your machine at one time. Maybe you can put a "Do Not Touch!!!" graphic over all of the other application icons on your desktop. I don't think this is Nexaweb's fault. Javascript in the browser still has a long way to come before you can write truly reliable, complex and layered systems.

Further, lots of these boundary pushing technologies make use of characteristics of the browser that are more side-effects than features. Look at Comet, for example. Common Comet techniques such as keeping the response from the server to the browser open (in an Iframe, etc.) or streaming a multipart HTTP message via XHR don't work the same in all browsers, are not part of any browser requirements, and could change with the next release. This is no way to implement serious applications.

Now I'm not saying that Comet is a bad idea, or that Nexaweb is peddling SOA/MOM junkware. Quite the contrary. I like what I see and hope to see more. What I object to is that all of this is being done via the back door. If we are going to implement fancy communication layers in the browser, let's do it by the front door, by requesting a more robust Javascript execution environment perhaps with threading and better networking control built into the browser.

Today's browsers are not that far removed from the Netscape Navigator with Livescript (later renamed to Javascript) that I used in 1995 to toggle a few heirarchical checkboxes. We are asking them to do things they were not designed to do. As we push the envelope of what a web app is and can do, we should do so in a smart way. Otherwise we are just like that guy who took WordPerfect macros -- a tool designed to perform simple textual and menu operations -- and wrote an ETL system. Yes, it can be done, but why would you?

Technorati : ajax, coach wei, esb, jms, mom, nexaweb, soa

• Digg
• Reddit
• del.icio.us

Comments (2)

agile ajax

101 Ideas for JSONP - Idea #3: Scraping HTML With TagSoup and XQuery

Dietrich Kappe, Monday, March 19, 2007 @ 6:20 am

Now that we have two ways (here and here) to pull XML data into our Ajax apps via JSONP, it's time to talk about how we can get our hands on more interesting data. Our two techniques allow us to proxy services that already produce valid XML, but not everything is available in this form. Lots of groovy data is still locked away in tradition web applications.

You can see an example of this problem even over at the leader in JSONP web services, Yahoo!. While Yahoo! provides an excellent delayed stock quote service, one thing that is lacking is a corresponding ticker lookup service. They do however have a ticker lookup web app: http://finance.yahoo.com/lookup?t=S&m=US&s=IBM

Transforming this app into a service involves the usual calisthenics -- transform the crappy HTML into well formed XHTML and then extract and transform our desired data into valid XML. Now we could use XSLT to crack this nut, but every time I break out the XSLT I get flashbacks to my Recursive Function Theory course and think that I'm doing a proof. Truth be told, I've never been a big fan of functional programming; FLWR expressions just seem so much easier to write. Yes, there are times where performance dictates not using XQuery, but for this example I'm sticking with my preference.

Below is a query that will perform our desired extraction and transformation. (XQuery syntax is a little beyond the scope of this post, but for a quick tutorial see here.) Like all screen scraping, it is a little brittle. If Yahoo! decides to change the format or even the color of the header row in this return page, the thing breaks. There are other queries that will do the trick and maybe be a little less brittle, but you should build some sort of regression testing and notification into these sorts of screen scrapers for precisesly these reasons. When it breaks, you know about it and you can fix it quickly.

<entities>{     for $t in //*:table[*:tr[@bgcolor="dcdcdc"]]     for $r at $position in $t/*:tr[count(child::*:td)>1]     where $position != 1     return     <entity>     <symbol>{data($r/*:td[1])}</symbol>     <name>{data($r/*:td[2])}</name>     <market>{data($r/*:td[3])}</market>     <industry>{data($r/*:td[4])}</industry>     </entity> } </entities>

To apply this to the Yahoo! ticker symbol lookup, we use a combination of the Nux XML processing toolkit and the TagSoup parser. TagSoup is the weapon of choice for converting gnarly HTML to well formed XHTML, and Nux gives us the XQuery machinery we need. Here is the code snippet for scraping the ticker symbol information from the results page:

    private String applyScript(String uri) {         InputStream in = null;         GetMethod get = new GetMethod(uri);         get.setFollowRedirects(true);         try {             httpClient.executeMethod(get);             in = get.getResponseBodyAsStream();         } catch (IOException e) {             log.error("Problem getting uri feed " + uri, e);             return JSONIdeasConstants.ERROR_XML;         }         try {             XMLReader parser = new org.ccil.cowan.tagsoup.Parser(); // tagsoup parser            Document doc = new Builder(parser).build(in);            Nodes results = XQueryUtil.xquery(doc, xqueryScript);            if (results.size() < 1) {                return JSONIdeasConstants.ERROR_XML;            }            return results.get(0).toXML();        } catch (ValidityException e) {            log.error("Problem getting uri feed " + uri, e);            return JSONIdeasConstants.ERROR_XML;        } catch (IOException e) {            log.error("Problem getting uri feed " + uri, e);            return JSONIdeasConstants.ERROR_XML;        } catch (ParsingException e) {            log.error("Problem getting uri feed " + uri, e);            return JSONIdeasConstants.ERROR_XML;        }    }

Here xqueryScript is a String containing the above XQuery script text. The ERROR_XML is just a constant string, "<Error></Error>", that we send back to the caller in case of an error. (You may want to send back a more informative error message to the browser. We'll tackle that in a later post.) This code will end up transforming the following app output...

...into the following xml

  <entities>     <entity>        <symbol>IBM</symbol>        <name>INTL BUSINESS MACH</name>        <market>NYSE</market>        <industry>Diversified Computer Systems</industry>     </entity>     <entity>        <symbol>HZK</symbol>        <name>CORTS TR VI IBM DEB</name>        <market>NYSE</market>        <industry>N/A</industry>     </entity>     <entity>        <symbol>HZD</symbol>        <name>6.40% CORPORATE-BACK</name>        <market>NYSE</market>        <industry>N/A</industry>     </entity>     <entity>        <symbol>KVM</symbol>        <name>STR PD 7.0 CORTS IBM</name>        <market>NYSE</market>        <industry>N/A</industry>     </entity>     <entity>        <symbol>GJI</symbol>        <name>SYNTHETIC FXD INC</name>        <market>NYSE</market>        <industry>N/A</industry>     </entity>  </entities>

Submitting the query is just the same old thing we've done before, just grab the input field contents and pass it to our servlet via an insert script operation:

  JSONPIdeas.lookupSymbol = function() {      var query = $("#lookup").val();      JSONPIdeas.addScript("http://labs.pathf.com/JSONIdeas/TickerLookup?callback=JSONPIdeas.renderSymbols&query=" + query);  }

Give the example below a try. You can see the Javascript source code involved in this here. Note that if there is only one entity, the JSON lib that converts the XML into JSON doesn't return an array, so we check for that.

I hope I've inspired you to write some of your own JSONP web services. Next time I'll tackle some of the security issues that JSON and JSONP raise.

 
  Technorati : ajax, jsonp, nux, tagsoup, xml, xquery

• Digg
• Reddit
• del.icio.us

Comments (1)

agile ajax

SOA meets Ajax - APIlitAx: an Alternative Interface for Google AdWords

Dietrich Kappe, Wednesday, November 15, 2006 @ 9:49 am

In the deluge of social bookmarking, weather ticker, and notepad applications, it's always nice to see an application come along that isn't a toy. APIlitAx (ugly name, I know) is an alternative interface to the Google AdWords system. It seems to have been developed by some folks at google and has been released on Sourceforge. As an application, it is very different from most of the stuff coming out of Google these days, which seems focused around the GWT technology. This applications doesn't use an existing Javascript framework, i.e. it's all custom Javascript, and uses PHP on the back end to proxy requests to AdWords via the APIlity library (PHP API for AdWords).

The developers have been careful to make the application non-blocking, i.e. you can interact with parts of the interface while another part updates. This is probably an emerging best practice for Ajax apps: don't just make lots of small blocking requests. Design your apps so that you can make small, non-blocking requests.

Technorati : PHP, SOA, adwords, ajax, google

• Digg
• Reddit
• del.icio.us

Comments (1)

agile ajax

If SOA and AJAX is the Killer App, Who is it Killing?

Dietrich Kappe, Friday, October 13, 2006 @ 12:45 pm

Over at SOA Digest, AJAX + SOA is hailed as the new Killer App:

We have business functionality represented in a reusable form - SOA services. We have ubiquitous connectivity - the Web. We have what's turning out to be the new application container - the browser. We have a programming model in the application container/browser - JavaScript. What I really want is a user-based composite application, not a middleware-based composite application.

I'm seeing this more and more, the idea that the browser becomes not just the interface from some middleware powered app that orchestrates the services, but that the browser, by means of Ajax, becomes the orchestrator of services, cutting out the middleman. I see this trend in the stated desire of Tibco to have it's GI product drive the demand for services in the enterprise. It is a trend that worries me.

I think the author of the post misunderstood the original article he was referring to:

What I really want is a user-based composite application, not a middleware-based composite application. To make this happen, I need a development and runtime platform that not only speaks AJAX and SOA, but governs the two as well. Some vendors promote the concept of AJAX applications calling WSDL-based Web services directly from the browser, essentially making SOAP calls. This approach even has a name - "client/SOA." This may be fine for simple non-enterprise or pure consumer applications, but it's a no-go for the enterprise. Why? Because when you call Web services directly from the browser, governance is left to the browser - which is another way of saying, there's no governance.

That article goes on to talk about a "service gateway," a fuzzy notion that sounds a little like a web proxy with some security. Even if workable, that still doesn't solve the drip, drip, drip at the heart of Ajax+SOA: orchestrating and composing services is a form of the dreaded Business Logic Leak. You are leaking both data (from the services) and logic (how you are composing them). This past April I described why it was a bad idea:

There are two kinds of information leak -- what and how. "What" is your customer data, your orders, your accounts receivable, your employee database. The "how" is your secret sauce, the special way of pricing or quoting or doing that gives you a competitive advantage. Reports can leak the "what," and many businesses can survive a leak of quite a bit of "what," but leak the "how," your secret sauce that makes you money, and you're cooked. Javascript can leak the "how" if you let it.

Even Tibco will tell you that it's GI product is intended only for the corporate desktop (where the ability to control access to data and logic is much greater) and not for the great unwashed on the open Web.

For the actual service providers there is a risk too. If you publish an SOA interface of some kind (SOAP, REST, etc.), your competitors can clone that interface and undercut your offering. See the Google Maps clone for down under for evidence that this is already happening. Building in switching costs becomes a difficult task which runs counter to the whole principle of SOA.

Finally, cross domain Ajax is still a problem with either JSONP (essentially JSON with a callback function) or a web proxy being the only viable solutions for orchestrating web services across domains in the browser. Right now, Yahoo! is the only major service provider to offer JSONP.

In my cynical view, Ajax + SOA is just another way of saying "mashup" -- a clever if innocuous pairing of two or more web services. And right now, given the nature of things, that's all it will be until the browser platforms change in some major way.

Technorati : ajax, soa

• Digg
• Reddit
• del.icio.us

Comments (0)

agile ajax

Framework Watch - Jayrock

Dietrich Kappe, Sunday, October 8, 2006 @ 12:19 pm

Nice and simple, that's what I like in my dotnet frameworks. Jayrock certainly fits the bill there. It's somewhat similar to the Java-based DWR, in that it automatically generates a Javascript proxy class from your ASP.NET HTTP Handler. From the project page:

Jayrock is a modest and an open source (LGPL) implementation of JSON and JSON-RPC for the Microsoft .NET Framework, including ASP.NET. What can you do with Jayrock? In a few words, Jayrock allows clients, typically JavaScript in web pages, to be able to call into server-side methods using JSON as the wire format and JSON-RPC as the procedure invocation protocol. The methods can be called synchronously or asynchronously.

This framework could be used as the basis for a more full featured framework (like WebWork on top of DWR and Dojo). I'd like to see them follow DWR's lead and add Comet support.

Technorati : ajax, c#, web services

• Digg
• Reddit
• del.icio.us

Comments (0)

agile ajax

Web Serice and Mashup Pros and Cons and the First Google Clone

Dietrich Kappe, Thursday, September 28, 2006 @ 10:46 pm

Over at Web Directions South, a good collection of pros and cons regarding web services and mashups from Kevin Yanks and Cameron Adams. Some choice things from the list

• Having an API allows external programmers to access your data in a smart way. Page scraping is so Web 1.0
• Programmableweb.com acts as an encyclopaedia of what APIs are available and what people are doing with them.
• The amount of data mining capable through APIs is potentially frightening; especially if you're the paranoid type.
• It's not all about maps - TagTV, Viral Video Chart, BlueOrganizer, Salesforce Adwords.
• Cross Domain AJAX is a security risk that has come along for the ride. You can load images, CSS and JavaScript from other domains, but cannot load HTML or XML. JSON-P gets around this by disguising the new data as JavaScript. The potential problem here is that your data provider could very easily inject malicious script if they wanted to. JSON-P only supports GET requests and fails silently if you get the API URL wrong.

One thing I was not aware of was that a company down under -- ZoomIn -- is providing a google maps-like interface. I guess google maps for New Zealand and Australia leave something to be desired, leaving an opening to an upstart that looks a lot like google maps. You can include maps on your site, and the Javascript API looks just like the Google one.

What does this all mean? It means that in Australia and New Zealand if you don't like google or it's terms of service, you can use an equivalent service with a minimal change. Is this the future of online services, or are we about to see a raft of API lawsuits? Hard to say.

 
  Technorati : google clone, mashups, soa

• Digg
• Reddit
• del.icio.us

Comments (1)

agile ajax