Category Archives: Open Source

Update to embedded Axis application in Tomcat

I got a great email from Tamás in response to my last post who pointed out that the straight copy of deploy.wsdd to server-config.wsdd doesn’t cut it. More importantly, he mentioned that there is a utility that ships with Axis that allows you to generate server-config.wsdd from your deploy.wsdd (or from multiple deploy.wsdd if you have multiple web service end points). From the command line it looks like this:

> java -cp axis.jar;jaxrpc.jar;commons-logging.jar;commons-discovery.jar;saaj.jar;
org.apache.axis.utils.Admin server dir1\deploy.wsdd dir2\deploy.wsdd

But if you’re using the Ant build.xml I provided in the previous example, you’d use this:

<java
  classname="org.apache.axis.utils.Admin"
  fork="true"
  failonerror="true"
  classpathref="compile.classpath"
  dir="${basedir}\WEB-INF\">
   <arg value="server" />
    <arg file="${basedir}\deploy.wsdd" />
</java>

I updated the source code example (embeddedaxis.zip), you can download it here.

NOTE: The source code for the Admin class is available here, where you can see (but the documentation doesn’t mention) that the Admin class accepts multiple WSDD files from the command line.

Embeddding an Apache Axis application in Tomcat

One of the applications I’ve been heading up at my 8-to-5 needed a SOAP API that fronted a Java application deployed on Tomcat. If you’ve spent any time with Axis you know that it’s not the simplest thing to deal with; in fact it’s downright complex if you want do anything more than the simplest thing using SOAP. The simplest? Write your source code, rename the .java file to have a .jws extension and then copy the file into a public directory inside your servlet container (see ‘Deploy a Java Class as a Web Service’). Easy. But this method leaves alot to be desired: you can’t use packages in the source code and the code is compiled at runtime which means you don’t find out about compilation errors until after deployment (one way around this would be to first compile the .java file to make sure that it works and then use Ant to copy / rename the .java file to a .jws) and you can’t specify custom type mappings, among other things. Lucky for you, the jws method isn’t the only way you can do it.

The next two options give you flexibility with the additional cost of complexity. The first method is well covered in the documentation: Axis comes packaged with a web application that you can deploy to your servlet container and then add your custom services using a remote administration client also provided with the war file that you deploy. The downside (at least in my environment) is that this means you now have to maintain and deploy two separate applications: my Java based web application would be deployed to Tomcat and then the same business logic would be deployed to the Axis engine running inside of Tomcat. I felt that it would be simpler to maintain to instead deploy the web application and the SOAP application together as one application in one war file. That of course, is not well documented (in fact other than a PDF file that’s part of the ‘Java Development with Ant‘ written by Erik Hatcher, there is no mention of deploying your web application alongside an Axis application without using the Axis AdminClient). Hence the article you’re reading now.

step 1: setup application / dependent libraries: I’ll make the assumption that you already have a web application that you want to expose SOAP webservices with which means you probably have a file structure that looks like something like this:

index.jsp
  - WEB-INF
    web.xml
    -classes
    -lib

You’ll need to add the following libraries to your WEB-INF/lib directory:
* axis.jar
* axis-ant.jar
* jaxrpc.jar
* wsdl4j.jar
* commons-logging.jar
* commons-discovery.jar
* saaj.jar

If you don’t, you can download the sample application I wrote that exposes a single hello world webservice.

Alright, so either you’ve downloaded the sample application or you’ve got your own application and you’ve added the above libraries to your WEB-INF/lib directory. Now you’re ready to write some script to expose your existing classes.

step 2: use WSDL2Java / Java2WSDL to generate the server side wrapper & deployment descriptors for the classes you want to expose. Write a interface:

package net.cephas.soap;
public interface HelloWorld extends java.rmi.Remote {
  public java.lang.String sayHello(java.lang.String in0)
    throws java.rmi.RemoteException;
}

and then a corresponding implementation which should be named $InterfaceName + SoapBindingImpl

package net.cephas.soap;
import java.rmi.RemoteException;
public class HelloworldSoapBindingImpl implements HelloWorld {
  public String sayHello(String name) throws RemoteException {
    return "hello " + name;
  }
}

Next, you can use Ant (but you could easily run this from the command line as well) to create the WSDL and generate the server side wrapper & deployment descriptors. You’ll need to define two tasks in Ant:

<taskdef name="axis-java2wsdl" classname="org.apache.axis.tools.ant.wsdl.Java2WsdlAntTask">
  <classpath refid="compile.classpath" />
</taskdef>
<taskdef name="axis-wsdl2java" classname="org.apache.axis.tools.ant.wsdl.Wsdl2javaAntTask">
  <classpath refid="compile.classpath" />
</taskdef>

and then you can generate the WSDL:

<axis-java2wsdl
  classname="net.cephas.soap.HelloWorld"
 style="rpc"
  namespace="urn:soap.cephas.net"
  location="http://localhost:8080/${project.distname}/soap/helloworld"
  output="${basedir}\generated\helloworld.wsdl">
</axis-java2wsdl>

and generate the server side wrappers:

<axis-wsdl2java
  output="${basedir}\generated"
  serverside="true"
  skeletondeploy="true"
  url="${basedir}\generated\helloworld.wsdl"
  verbose="true"
  debug="true">
</axis-wsdl2java>

Copy the generated web service deployment descriptor (deploy.wsdd) file to WEB-INF/server-config.wsdd:

<copy file="generated/net/cephas/soap/deploy.wsdd"
  tofile="WEB-INF/server-config.wsdd" />

and the *SoapBindingSkeleton.java file to your source tree:

<copy todir="src/net/cephas/soap" includeEmptyDirs="no">
  <fileset dir="generated/net/cephas/soap">
    <patternset>
    <include name="*SoapBindingSkeleton.java" />
    </patternset>
    </fileset>
</copy>

Finally, compile your source and either jar it up to the WEB-INF/lib directory or deploy the compiled classes to the WEB-INF/classes directory.

step 3: configure web.xml with the appropriate servlet mappings. The last thing you need to do is to map a resource path in your application to Axis, you accomplish this by adding a servlet and servlet-mapping element to your web.xml:

<servlet>
  <servlet-name>AxisServlet</servlet-name>
  <display-name>Apache-Axis Servlet</display-name>
  <servlet-class>
    org.apache.axis.transport.http.AxisServlet
  </servlet-class>
</servlet>
<servlet-mapping>
  <servlet-name>AxisServlet</servlet-name>
  <url-pattern>/soap/*</url-pattern>
</servlet-mapping>

You can see that I’ve selected ‘/soap/’ as the resource path which in combination with the server-config.wsdd means that I’ll invoke the SOAP web service using a URL like this:

/embeddedaxis/soap/helloworld?wsdl

where embeddedaxis is the name I’ve the sample application. It will probably be different for your application.

Phew! That’s an awful lot of scripting and configuration to deploy hello world, but at least now you don’t have to rely on .jws files or deploying your application separately from the SOAP service. If you’re trying to do the same thing and something you read above doesn’t make sense, ping me.

Download embeddedaxis.zip.

XML characters, smart quotes and Apache XML-RPC

I’ve been eating my own dogfood with the deliciousposter project (as you can see from my daily links). A couple days ago I posted a some links to del.icio.us and expected them to show up automatically the next day… except they didn’t. I traced it down to an errant smart quote that I copied from the Internet Alchemy Talis, Web 2.0 and All That post, which caused the Apache XML-RPC library to throw this error:

java.io.IOException: Invalid character data corresponding to XML entity ’

I worked under the assumption that the smart quote was an invalid XML character for quite awhile, but it looks like it actually is according to the XML 1.1 specification, the following characters are allowed in an XML document:

#x9 | #xA | #xD | [#x20-#xD7FF] | [#xE000-#xFFFD] | [#x10000-#x10FFFF]

I then checked the source code for the XmlWriter which has this method for writing character data:

...
if (c < 0x20 || c > 0xff) {
  // Though the XML-RPC spec allows any ASCII
  // characters except '<' and '&', the XML spec
  // does not allow this range of characters,
  // resulting in a parse error from most XML
  // parsers.
  throw new XmlRpcClientException("Invalid character data " +
  "corresponding to XML entity &#" +
  String.valueOf((int) c) + ';', null);
} else ..

which turns out to be a tad aggressive. It also turns out that the above code snippet and the version of the Apache XML-RPC library I was using are out of date. The chardata(String text) has been updated in the latest version of the Apache XMl-RPC library to include a new method called isValidXMLChar(char c) which is much more lenient:

if (c == '\n') return true;
if (c == '\r') return true;
if (c == '\t') return true;
if (c
and not coincidentally, is compliant with the specification.

I'll be updating deliciousposter to use the latest version of the Apache XML-RPC library soon. In the meantime, if you're using the Apache XML-RPC library, you should probably download the latest version to take advantage of the new XML character validation method.

eBay Java / C# SOAP Examples

At the start of this year I worked with some guys at eBay to further develop their code samples. Some, but not all of the twelve examples I wrote went live in the recently launched Community Codebase. You can download all the Java examples (of which I wrote three) or browse the Subversion repository. I wrote the SOAPAddItem, SOAPGetItem, and SOAPGetUser items in the Java source tree.

The examples I wrote were different from the majority of the (then) existing examples in that I didn’t make any IDE assumptions (most of the Java examples have JBuilder .jpx files and Eclipse .project files, the .NET projects contain the Visual Studio artifacts.. tsk tsk.) and as such, all the examples I wrote contain comprehensive Ant / NAnt build files which means you can get up and running without having to setup your fancy schmancy IDE. But the biggest difference was that all the examples I wrote used either Ant (with WSDL2Java) or NAnt (with NAntContrib) tasks to conditionally download the eBay WSDL, generate the client stub(s), and compile the resulting code, which makes for a prettier source code repository (generated stubs aren’t checked into source) and gives you compile time checking of your code against the API.

If you’re interested in how you can use Ant or NAnt in a build environment where you access SOAP services, you should check it out!

del.icio.us poster

Last month I wrote about a quick utility which uses the delicious-java and Apache XML-RPC libraries to grab posts from my del.icio.us account and post them to my Metaweblog enabled (in my case Movable Type) blog. I’m not sure that it’s useful unless you have your own server or always on desktop PC, but I wasn’t satisfied with the original product, so I updated it with better documentation, a license (GPL), an Ant build file and the ability to edit the format of the posts using a Velocity template. Oh, and it’s all in Subversion now. You can read all about it by checking out the del.icio.us poster project page.

AWStats Installation Notes

I tried installing AWStats a couple months ago on my server, got frustrated after an hour or two of reading the documentation and trying to figure out permissions problems, gave up and then tried again about two weeks ago, this time with more patience. Maybe after reading my notes (on RedHat Linux) someone else won’t have to exercise as much patience.

First step: download AWStats, unzip (I unpacked to /usr/local/awstats/) and then execute the following script to start the setup process:

perl awstats_configure.pl

After you complete that step, you’ll have a couple AWStats directives in your Apache configuration. Since some of the sites I’ve written use servlet filters to do URL rewriting and because I wanted to minimize the number of AWStats entry points (AWStats was recently hacked), I setup a virtual host in Apache as a reporting domain. I then moved the AWStats directives inside this reporting virtual domain.

Next, using the sample awstats.mysite.com.conf, you’ll need to create the a domain configuration file for every site that you want to report on and place each one in /etc/awstats/ (read more about the domain configuration files here, scroll down to step #4 and #5). So I ended up with something like:

# ls /etc/awstats
awstats.cephas.net.conf
awstats.blackberryblog.com.conf
...

After you’ve got all your configuration files setup, you’ll want to run awstats against all your old log files (assuming that you’ve got old log files that you want to process). There’s no easy way of getting all your old log files processed, I had to modify the configuration file to point to the zipped up log file:

LogFile="gzip -d </usr/hosts/yoursite.com/logs/200205.zip | "

run awstats:

perl awstats.pl -config=yoursite.com -update

and then modify the configuration file to point to the next month. If I was better at shell scripting I’m sure I could have worked out some way of automating this, but I didn’t have that many log files laying around so it wasn’t a big deal. After you’ve processed all your old log files, you’ll want to point AWStats to look at your current log files. I configured my machine so that it always processes the log file based on yesterday’s date, so my configuration looked like this:

LogFile="/usr/hosts/yoursite.com/logs/%YYYY-24%MM-24.log"

and then I setup a cron job to execute AWStats every night by saving this snippet of code as a script and placing it in your /etc/cron.daily/ folder (cron.daily runs based on the settings in /etc/crontab)

#!/bin/bash
cd /usr/local/awstats/tools
perl awstats_updateall.pl now

So now you’ve got your configuration files, you’ve got a script that processes your logs automatically every night and you want to view the reports from a web browser. So you point your web browser to:

http://reporting.yoursite.com/awstats/awstats.pl?config=yoursite.com

and if you’re like me at this point you’ll run into an error message that makes no sense at all:

[Fri Jun 03 15:52:42 2005] [crit] [client xxx.xxx.xxx.xxx] (13)Permission denied: /usr/local/awstats/.htaccess pcfg_openfile: unable to check htaccess file, ensure it is readable

The funny thing was that I didn’t have an htaccess file anywhere near the awstats configuration, this error message drove me batty. After googling for an exceptionally long period of time, I updated both /var/lib/awstats/ and /usr/local/awstats/ so that the Apache user owned them.

chown apache.apache /var/lib/awstats
chown apache.apache /usr/local/awstats

Finally, unless you want everyone to see your stats (and allow anyone to access the AWStats.pl script), you’ll want to lock down the cgi-bin directory so that only valid users can access your system. Create a .htaccess and a .htpasswd file in your /usr/local/awstats/wwwroot/cgi-bin/ directory (making to sure to set AllowOverride All in your directive inside the VirtualHost in apache). Configure your .htaccess to only allow access to valid-user:

AuthName "reporting"
AuthType Basic
AuthUserFile /usr/local/awstats/wwwroot/cgi-bin/.htpasswd
Require valid-user

You can create the .htpasswd file by running this command:

htpasswd -c $filename $username

and then add users:

htpasswd $filename $username

and make sure to modify your AWStats configuration file to allow access to __REMOTE_USER__ or the specific user that you want to grant access to that file.

You should now have a working, secure and fully automated AWStats reporting system! Congratulations!

Paper Review: Two Case Studies of Open Source Software

The term paper for the class I’m taking (Paradigmatic Software Development) required that we write a review of a article that has appeared in a software engineering journal. I spent about 45 minutes trying to poke around the journal locator that UMass Dartmouth outsourced before giving up (they don’t let you search by topic, how lame is that?) and then remembering that Google has an academic focused search. Scholar.google.com found a bunch of articles about ‘open source’ and ‘software engineering’, I chose one entitled “Two Case Studies of Open Source Software
Development: Apache and Mozilla
“. My review of the paper is over 7 pages long, you can download the entire thing in MS Word format or read the extended entry below.
Continue reading Paper Review: Two Case Studies of Open Source Software

Source Control with Subversion on Windows

We’re getting around to hiring an additional me at work so I needed to get a source control solution in place before he/she arrives and starts hacking away at the codebases. At the last place I worked we used Visual SourceSafe for a long time (which was pretty worthless) and then started using CVS, although not on the projects I worked on. A couple weeks ago I asked a bunch of questions about automating Java web application deployment and Erki suggested that I buy and read the Pragmatic Project Automation book, which I did and which I loved. I read it from start to finish (which isn’t an easy thing to do with a technical book) and took a ton of notes. Anyway, I’m pretty sure the book didn’t mention Subversion even once, but it did drill home the importance of setting up your code in source control and then doing your manual, scheduled, automated (using CruiseControl) or production builds directly from the source control system using Ant or Groovy, which led to alot of ideas about how you could automate builds to production Tomcat systems. I’ve heard alot about Subversion; it’s described as CVS with none of the warts and all of the features, so I tried it out. Following are the notes I kept during the installation process. Hopefully this helps out someone somewhere down the road:

a) Download Subversion (obviously) I downloaded version 1.1.1 ( svn-1.1.1-setup-2.exe)

b) Run the install. I chose to install to c:\subversion\.

c) Test the Subversion install by going to the install directory and then going to the /bin directory. Run the svnadmin from the command line:
CD c:\subversion\bin\
svnadmin

If you get an error that says that says “svnadmin.exe – Unable To Locate DLL”, you’ll need to either a) install the Visual C++ 6.0 VCREDIST.exe, which is available here (but only if you’re using Windows 98 or Windows ME) or b) find someone or some system that has msvcp60.dll and copy that file to c:\$windows\system32\ (where $windows is the directory that contains your Windows installation).

d) Create a repository on the machine that is going to host your source code. From the command line on the server machine:
svnadmin create d:\$projects\$projectname

where $projects is the name of a folder that will contain all your Subversion source code repositories and $projectname is the name of the specific project you’re going to create a repository for.

e) Start the server either from the command line:
svnserve -d -r d:\$projects

or use the SVN Service Wrapper (http://dark.clansoft.dk/~mbn/svnservice/), note that the r switch limits directory browsing by specifying the root of your repositories.

f) Create a users database (which is nothing more than a text file, I created a file called users.conf) and save it in the d:\$projects\$projectname\conf directory and then add your usernames & passwords. An example user database might look like this:[users]
aaron = mypassword

g) Within the d:\$projects\$projectname\directory, open the conf/svnserve.conf file using your text editor of choice and add the following to allow reading and writing to the users in the database we just created:
[general]
anon-access = none
auth-access = write
password-db = users.conf
realm = $projectname

where $projectname is the name of the project you’re setting up and users.conf is the name of the file you created in step f.

h) Finally, restart the server and you’re ready to start using the client and import your project.

i) To import an existing project, invoke the svn executable from the command line on the client:
svn import c:\temp\myproject svn://myserver/myproject -m "initial import" aaron mypassword

where c:\temp\myproject is the directory that contains the existing project you want to import (and should have 3 folders: trunk, tags and branches, your source code should be in the trunk folder; more on repository layout here), myserver is the name of the server that contains the Subversion repository you created in step d, myproject is the name of the project (ie: $projectname from step d) you’re importing into, the m switch signifies the message you want to tag this import with, followed by the username and password you setup in the users database in step f.

j) Finally, checkout the project from the server (invoking the svn executable from the directory that you want the trunk checked out from…, ie: c:\projects\myproject)
svn co svn://myserver/myproject/trunk . aaron mypassword

k) To add, move, delete, or edit and then commit any changes back to subversion:
Add: svn add src/myfile.java -m "adding a file"
Move: svn move src/myfile.java src/mynewfile.java -m "moved myfile.java to mynewfile.java"
Delete: svn delete src/myfile.java -m "removing a file"
Commit: svn commit src/myfile.java -m "the message"

That’s it… there’s a alot more to read. One of the coolest things about Subversion is that the documentation is all packaged up into a free PDF published by Oreilly, which you can download here. I printed out the manual and breezed through it in a couple hours. You should too.

I did try to get the Subclipse Eclipse Subversion plugin working, but for the life of me… I couldn’t. I think it ended up for the better anyhow because while the command line is a bit more tedious, I understand more about what’s going on in the system, which is helpful. I’d recommend that you get familiar with the command line client before attempting to use a GUI plugin.

After you’re all set with Subversion and you’re ready to get started with CruiseControl, note that while Cruisecontrol contains an <svn> tag, you still need to drop down into command line mode in Ant to checkout / update your local repository when doing an automated build. For example:
<exec executable="svn">
  <arg line="update"/>
</exec>

Conflicting mindsets of C# vs. Java: Part II

You all read the the ‘Conflicting mindsets of C# vs. Java‘ weblog post right? And you all noticed that the guys running the Lucene.NET project on sourceforge closed up shop, took all their toys and went on home right? I’m gonna go out on a limb and say that they’re related.

The way I see it, *in general* the .NET community conversation is dominated by talk about the latest and greatest that microsoft is putting out; there’s talk about MapPoint Location Server, SQL Server, Longhorn, ASP.NET 2.0 and Visual Studio; all products of Redmond. The same group of Java developers are talking about JBoss, Hibernate, Struts and Eclipse: none of which came out of the Silicon Valley.

Malcolm’s mindset #1 says that .NET developers accept the tools and services that are provided them by Microsoft and I think for the most part this is true. You don’t see .NET developers spending their cycles on persistence layers, web application frameworks or caching solutions probably because Microsoft has provided Microsoft solutions for all these problems. But if it’s just providing tools, then why aren’t JSF, JDO and NetBeans dominating the javablogs conversations? Seriously, take a look at ASP.NET and JSF. They aren’t that different and yet ASP.NET is widely used in conjunction with Visual Studio while JSF is rarely lauded and more often derided. I think he’s right, it’s really a mindset.

Which brings me back to the Lucene.NET guys. Why would they close up shop? Why not continue to donate their time and energy to an excellent cause? Maybe the Microsoft mindset has something to do with it. How about this: a search on google for ‘lucene’ within the weblogs.asp.net domain yields exactly 17 results. The same search on jroller.com yields 2570 results. Admittedly, Lucene has been around longer, but maybe one of the reasons that the Lucene.NET guys packed it up (and are now trying to sell their work) is that no one paid any attention to them because they were all too busy working with SQL Server full-text indexing, a tool given them by Microsoft (but one that costs thousands of dollars per processor). Another reason that a project like Lucene or Struts or Tomcat flourishes is because there is a certain amount of prestige working on a big open source project. If you work on open source projects for the prestige and you’re not getting the attention you think you deserve, you find another motivation. In their case money was a motivation, so they closed up project on sourceforge and they’re selling a personal edition and a business edition. They might make a couple bucks, but I bet in 1 year there won’t be many people writing about searchblackbox.com.

So what’s my point? That all .NET developers are greedy and don’t care about the community? Not really. I think it’s that the two communities have different bus drivers: .NET developers look to Microsoft to provide the tools they need to do their jobs… and if they look elsewhere or copy something else, Microsoft will eventually come in and make a product of their own that does the job, thereby negating any work the developers do in the meantime. Microsoft drives the bus. Java developers look at the products and specs that Sun puts out and then go and build their own tools or frameworks or applications to do the job. Sun will eventually put out something through the JCP that does the job…. but the developers in the Java community will only use it if they want too, witness the continued popularity of Struts and the lack of interest in JSF. In the Java camp, the developers drive the bus.