SNTT: Web Services Primer

hootsThursday_BLU.jpg

Its been a while, but I've been doing some really interesting web services work recently. In both cases, the customers had not yet upgraded to Notes/Domino 7 and therefore we couldnt use the Domino 7 LotusScript web services interface. In one case, in order to deploy BlackBerry MDS applications, and in another case to use Acrobat to add annodations to a PDF File.




So - what to do ? In my various presentations (especially the BlackBerry MDS Studio primer one), I state that a backup approach is to write a Servlet. However, the servlet engine in Domino is fairly old, and definitely not recommended. Can an agent be written to mimic a web service ? An agent written in LotusScript ? Well, it transpires - yes - this is possible.


So what this article gives you, dear reader, is a little bit of a primer on how to deal with this issue and choose the "best" approach.






























































Detail ND7 Web Service Servlet Agent
Deployment Type "Web Service" Servlet loaded with http Domino Agent
Language LotusScript Java LotusScript
Domino Version ND7+ ND5 + ND5 + (home grown XML)

ND6+ (LotusScript XML)
Ease of Deployment Simple - code is updated Files copied into Domino\servlet path and http process restarted Simple - code is updated
Scalability (Note 1) Medium Large Small
Security Simple - Uses Domino Security with user context Hard - uses Server context. You have to hand-roll your own security in terms of differentiating between different users needs Simple - Uses Domino Security with user context
Adding new Web Service functions Simple - All handled by nd7 Web service interface Medium - the code has to be updated to deal with new web services (forcing redeployment of files) Medium - the code has to be updated with the new web services



Notes:


  1. Scalability.

    • The ND7 Web service is quite robust - I've certainly written Web services that deal with thousands of requests per hour. However, I would have to say that I couldnt see it handling more than 10 or so concurrent requests at any time.

    • I have personally seen servlet-driven approaches that are very scalable - the persistent nature of servlets means that the whole Notes Object Model support isnt being switched in and out of memory every time a single request is processed.

    • The agent approach relies on doing meaningful work during a web-triggered agent instantiation. Those who have designed and built huge domino web-based sites have seen that whilst web-agents are remarkably fast, certainly there appears to be a limit of 8-10 concurrent queries of the same type. So it might not scale as well on a single server approach.







Domino Servlet Overview


This isnt an exhaustive walk through of writing a Domino Servlet - that would take some time. However, this should get you up and running quickly.


  • Install Eclipse

  • Install Java v131 (for Domino v6.x) and set that as the base java version. Remember, this servlet has to be written to use the same version of Java as Domino.

  • Create a new Java project.

  • Copy the Notes.Jar (and possibly the XML4.jar files if you want XML4 to do the XML parsing) to the project directory and add them as External JAR files.

  • Remember that the DoGet function in your servlet deals with normal brower based queries, as well as the WSDL request from your web service consumer. So put code in here that will trap this and return the WSDL. And remember to start off with "Content type: text/xml" in order to inform the target client that this is an XML stream.

  • In terms of the WSDL, remember that the web services URL is passed as part of the WSDL - you might have to make this dynamically computed so that it returns the correct host/database URL.

  • The doPost in your servlet deals with POST requests - web service clients will push actual Web service requests to you, and pass correctly formed XML. You have to parse this XML and decide which web service request is being asked for, and parse out any parameters. Now this isnt trivial. I tried (unsuccessfullly) to use SAX parsing to do this, and after serveral days beating my head against the screen, switched to DOM parsing, which is far far more verbose.

  • You then decide (based on the XML) which request and parameters, and at this stage, go off and do some work on this. You then pass back correctly formed XML with the results of this operation, again remembering to set the content return type to text/xml.

  • Another trick with any Domino Java programming is to remember to run "recycle()" on any domino object that supports it. You see the Java NOI layer interfaces with an underlying C++ Notes Object Model factory, and you have to somehow inform the underlying C++ layer to release that memory. Since the java garbage collector is "lazy", this often never happens, and you land up allocating huge amounts of memory. Forcing the "recycle" operation cures this.

  • You then put the servlet class file into the domino data directory, in a subdirectory called "domino/servlet" (by default).

  • Place a well formed "servlet.properties" file into the domino data directory.

  • Switch on the Domino servlet engine on the server configuration document "Domino Web Engine" configuration section.

  • Remember that to update this servlet, you have to drop the http server task, update the files, and reload http. This operation will of course disrupt any other web users you have on this server (and is the achillies heel of Servlet programming). This may have to happen out of hours on your production servers..




Domino Agent Overview


Our challenge here is to come up with a Domino agent which responds to http requests and mimics a Web service. How hard could this be ?


  • Write a domino agent that responds to HTML requests.

  • Use the "Document Context" document to gain access to the incoming stream of data. The "REQUEST_CONTENT" item on this document will hold HTML for any query that was sent using a POST command. In other words, if REQUEST_CONTENT is empty, its a request for the WSDL, so just return that. Remember to set the content type to "text/xml".

  • Now parse the contents of REQUEST_CONTENT into a DOM parser (again, I prefer using DOM against SAX for relability reasons) and parse out the Web service operation and any parameters.

  • Based on the operation and parameters, do some meaningful work.

  • Now pass back some well-formed XML to describe the results.


Compared to the servlets above, this doesnt sound as hard.



Using SoapUI


An invaluable tool I found was SoapUI. Using this tool, you can point it at an existing web service, capture the WSDL, and then have it construct test cases for all operations. You can then pass in meaningful data into each operation and see the incoming as well as outgoing XML. This made the construction of this a breeze, and really cut down debugging time.



So how did I approach this?


My approach was:


  • I Constructed a web service in ND7 using web services, and used this to construct and debug the code in LotusScript that I wanted to port. It also meant that when the customers finally upgraded, they had a web service all ready to go.

  • I used SoapUI to grab the WSDL, and all incoming and outgoing XML pieces from the existing Domino 7 web service.

  • I then constructed either a Servlet or Agent based on this WSDL and XML.

  • I then used SoapUI to test the new Web service as I was building it, in order to check at each stage that it was working correctly.



So here's the kicker. Which one would I choose?


Its down to how many requests you think this service will receive, versus the time in coding. The servlet approach is far "crustier" (in terms of using a very old version of Java, v2.0 of the servlet interface, its lack of recommendation, and of course the restarting of http every time you wish to deploy new code), however it can be made far more scalable. It might be worth just doing the agent version and seeing how far it can take you, and then taking the decision to either upgrade to ND7 (recommended) or biting the bullet and dealing with it all in a servlet.



So as you can see - nothing superhuman. Now you've just had three weeks worth of analysis (the last three weeks of my life, in fact!), coding and implementation on three different types of web service implementation described - surely NOW there is no excuse for you NOT to do this in your applications ?



This is here so that you can understand my thought processes - this does NOT meant that any other approach is incorrect - quite the reverse. This is here so you can see my mistakes, and possibly save you some time, as well as helping you make your mistakes faster. And yes, I know its not Thursday.. ;-)