I’ve been keeping a list of interesting links in a text file for the last couple years and only recently thought better of it and started using del.icio.us to store links. And I wanted to show the links as part of this blog, but I wasn’t satisfied with just including their RSS feed of my links in blog (what happens when / if del.icio.us disappears?) so I whipped up a some code using the delicious-java library and the Apache XML-RPC library that extracts my delicious links from the previous day and then posts them directly my Movable Type blog using the XML-RPC API that comes standard with Movable Type. After putting it up on my server, I wrote a simple shell script that invokes the Java class and then popped that into /etc/cron.daily/ so that it will get called a nightly basis, voila!
#!/bin/bash
cd /usr/apps/delicious/
java -cp commons-codec-1.3.jar:commons-httpclient-3.0-rc2.jar:
commons-logging-1.0.4.jar:delicious-1.6.jar:xmlrpc-1.2-b1.jar:bin net.cephas.blog.DeliciousPoster config.properties
If you’d like to run it your own server, feel free to download the binaries (which include the source code), edit config.properties to match your Movable Type and de.licio.us profiles and scheduled it to run on a nightly basis.
Category Archives: XML
Using Betwixt, Struts and REST
My latest project at work launched a couple weeks ago, unfortunately there isn’t much to look at, but it’s not like I’m a Flash developer, so unless I start writing Windows apps, no one will ever be able to see my work anyway. Alas, a major portion of the project involved consolidating some business logic behind a SOAP API (I’ll go into that some other time). We were supposed to be moving our front end to ASP.NET but at the last minute, the ASP.NET migration didn’t happen so we ended up needing something that ASP could use as well, which meant REST. With the business logic wrapped up behind business delegates, all that was needed as a Servlet Filter to map requests for /myapp/xml to a Struts Action class, which invoked the business delegates and then returned XML to the client. Now on the SOAP end of things (Axis embedded in Tomcat), I didn’t have to deal with XML (Axis handles all the XML serialization and deserialization). But with REST you’re in conundrum: do you create toXML() methods on all of your JavaBeans? do you put the JavaBean or collection of beans in the Request scope and then generate the XML using JSP? Those options all seemed error prone and harder to test. I looked at JAXB (makes more sense when you have XML that you want to turn into a JavaBean) and Castor (same thing) but finally decided on using Betwixt, a Jakarta Commons library that provide a simple way of turning a JavaBean into XML. The end result looked something like this:
public class RESTAction extends DispatchAction {
public ActionForward doSomething(...) {
MyBean bean = BusinessLogic.getBean();
response.setContentType("text/xml");
OutputStream sos = response.getOutputStream();
BeanWriter beanWriter = new BeanWriter(sos);
BindingConfiguration config = beanWriter.getBindingConfiguration();
config.setMapIDs(false);
beanWriter.enablePrettyPrint();
String declr = "<?xml version='1.0' encoding='UTF-8' ?>";
beanWriter.writeXmlDeclaration(declr);
beanWriter.write("mybean", bean);
beanWriter.flush();
beanWriter.close();
return null;
}
}
which when combined with this action mapping:
<action
path="/xml"
type="com.mycompany.web.actions.RESTAction"
name="xmlForm"
scope="request"
parameter="method"
validate="false">
<forward name="error" path="/WEB-INF/views/xml/error.jsp" />
</action>
is invoked like so:
http://myserver/myapp/xml?method=doSomething
I’m not 100% sold on this as the “way”, but I do like how simple Betwixt is. If you’ve got a different idea, comment away.
Retrieving an RSS feed protected by Basic Authentication using ROME
Today I worked on a feature in a Struts web application where I needed to retrieve an RSS feed that is protected by Basic Authentication and then display the the results of the feed in a web page. I’ve heard alot about ROME in the past couple weeks, so I decided to try it out. It quickly passed the 10 minute test (downloaded rome-0.5.jar, downloaded jdom.jar, used the sample code from the tutorial on the wiki); I was able to retrieve, parse and display the results of my own feed in no time:
String feed = "http://cephas.net/blog/index.rdf";
URL feedUrl = new URL(feed);
SyndFeedInput input = new SyndFeedInput();
SyndFeed feed = input.build(new XmlReader(feedUrl));
System.out.println(feed);
Easy. But that wasn’t my problem. I needed to be able to set the Basic Authentication header which is usually done like this:
String feed = "http://yoursite.com/index.rdf";
URL feedUrl = new URL(feed)
HttpURLConnection httpcon = (HttpURLConnection)feedUrl.openConnection();
String encoding = new sun.misc.BASE64Encoder().
encode("username:password".getBytes());
httpcon.setRequestProperty ("Authorization", "Basic " + encoding);
httpcon.connect();
.. // do stuff
httpcon.disconnect();
Turns out that the designers of the ROME library were pretty smart. In addition to including the XmlReader(URL url) constructor, they also included a XmlReader(URLConnection connection) constructor, which allows you to combine the two blocks of code I wrote above to make this:
String feed = "http://yoursite.com/index.rdf";
URL feedUrl = new URL(feed)
HttpURLConnection httpcon = (HttpURLConnection)feedUrl.openConnection();
String encoding = new sun.misc.BASE64Encoder().
encode("username:password".getBytes());
httpcon.setRequestProperty ("Authorization", "Basic " + encoding);
SyndFeedInput input = new SyndFeedInput();
SyndFeed feed = input.build(new XmlReader(httpcon));
Add this code to your Struts action, put the resulting SyndFeed in the request scope (request.setAttribute("feed", feed);) and then in the JSP:
<c:forEach var="entry" items="${feed.entries}">
<strong>${entry.title}</strong><br />
${entry.description.value}<br />
<fmt:formatDate value="${entry.publishedDate}" type="both" pattern="MMMM dd, yyyy" />
by ${entry.author} | <a href="${entry.link}">link</a>
</c:forEach>
So there you have it. I hope that makes it easier for someone else!
ebay web services talk by Jeffrey McManus
A couple weeks ago I attended the Boston .NET User Group to hear Jeffrey McManus give a talk on how ebay is using web services. I took a bunch of notes during his presentation, which I’m putting up here semi-edited. The first part of the talk was mainly about ebay the company; how much business they do per quarter, how many users they have,etc.. The second part was about the ebay API, which I was interested in because the company I’m working for now is exploring a similar program, so you’ll see more detailed notes there:
45000 product categories
collectibles was the first product category on ebay, it is category id = 1
ebay did approximately $5.6 billion in cars last year105 million registered users as of 3/1/2004
rate of growth is accelerating quarter over quarter
28 countries with a physical presence and localized website
20 million items for sale at any one time…
1 billion items were listed last year
7 gigabits per second outgoing traffic
10 billion api hits in 2004in december 2003, 1 of every 3 people on the internet came to ebay
1 in 3 used golf clubs bought in the US are purchased on ebay
valueguide.pga.com uses ebay data to help determine the price of used golf equipmentEbay isn’t just for knick knacks: more than 125,000 keyword searches for Louis Vuitton, Coach and Prada on ebay every day
superpawn.com
— they build a proprietary POS with ebay and significantly reduced costs
— used to cost $23 per item to list & describe each item on ebay (person costs), now approx $.25highline auctions
— custom system integrator with a business around ebay integration, specifically in cars
— reduced the time it takes to create a professional listing from 30 minutes to under 3 minutesauctionworks
— solution provider on ebayuse cases
— integration w/ business tools (quickbooks, excel, sap, etc..) (ie: a donation can be estimated for taxes using a data licensing application)
— integration w/ back ends for medium & large retailers, manufacturers‘we make inefficient markets efficient’
scarce –> in-season reatail –> end of life–> used/vintage
— end of life scenario for retailers
— products at the end of life can be put on ebay programmatically saving companies the most of having to get rid of excess producttop level meta categories
— sports, tickets, etc…
— ‘browseability’ is a one of the challenges
— most likely use cases are sellers listing items programaticallydata licensing services
— you can purchase data feeds from ebay so that you can find out what the given price for a given car in a given location is
— andale.com: provides ebay pricing and selling recommendationsAPI
UBL 1.0
Last week Tim Bray mentioned the May 1st release of UBL 1.0, which he defines as “… a set of general-purpose XML-encoded business documents: orders, acknowledgments, packing slips, invoices, receipts.” He goes on to compare UBL to HTML, saying that because it (UBL) is a generic format rather than a format made for a particular industry (just like HTML was a generic, simpler subset of SGML), it has a chance to become the HTML of the business document world (read: explosive growth, eventual ubitquity). Tim quotes an email from Jon Bosak on some of the other reasons for the creation of UBL:
· Developing and maintaining multiple versions of common business documents like purchase orders and invoices is a major duplication of effort.
· Creating and maintaining multiple adapters to enable trading relationships across domain boundaries is an even greater effort.
· The existence of multiple XML formats makes it much harder to integrate XML business messages with back-office systems.
· The need to support an arbitrary number of XML formats makes tools more expensive and trained workers harder to find.
My current project, which should be released soon, utilizes software from many different companies: tax software, credit card software, shipping rate software, custom software written by the company that manages the distribution of product, etc.. Obviously having a single format to work with would decrease the time I spend a) digging through each companies documentation trying to understand their format and b) wiring up the custom documents for each format, so I’m definitely looking forward to the day when I can use UBL.
For anyone interested, it looks like there is a smattering of support for UBL out there in the Java world: http://softml.net/jedi/ubl/sw/java/, https://jwsdp.dev.java.net/ubl/, http://www.sys-con.com/story/?storyid=37553&DE=1. For further information regarding UBL, see the OASIS UBL TC web page at:
http://www.oasis-open.org/committees/tc_home.php?wg_abbrev=ubl
Amazon Web Services Newletter cool links
The February Amazon Web Services Newsletter had some pretty cool applications:
Blogfuel: “Use blogfuel to create a link to an individual item at amazon (using ASIN Search) to illustrate your book review. Or place the code in the sidebar of your blog as a “What I am currently reading feature”. You have control over the width, colour, number of columns, image and text size.”
PocketBooks – Amazon: “Check prices and availability of books at Amazon.com and Amazon.co.uk from your Java enabled phone”
Books of note, from Sybex: Mining Amazon Web Services: Building Applications with the Amazon API and from Dimensions: Google, Amazon, and Beyond: Creating and Consuming Web Services
Creating RSS using Java
I wanted to create RSS feeds for karensrecipes.com using Java. I did my ‘research‘, came to this page: Ben Hammersley.com: Java RSS libraries and then used the RSS4j library to create a servlet that serves up dynamic RSS feeds of the 10 most recently created recipes per category (samples: Breakfast, Soup, Barbeque..).
They syntax is pretty simple, you get an RssDocument and set which version you want to use (RSS 1.0, .9 or .91):
RssDocument doc = new RssDocument();
doc.setVersion(RssDocument.VERSION_10);
and then create a RssChannel object and add that to the RssDocument:
RssChannel channel = new RssChannel();
channel.setChannelTitle("Karens Recipes | Most Recent");
channel.setChannelLink("http://www.karensrecipes.com/3/Soup/default.jsp");
channel.setChannelDescription("The 10 most recently added recipes in the soup category.");
channel.setChannelUri("http://www.karensrecipes.com/rss/?categoryid=3");
doc.addChannel(channel);
Next, you’ll retrieve the items using a database, the file system, etc… and add each item as a RssChannelItem:
// connect to the datasource
// iterate over something (db? vector?...)
RssChannelItem item = new RssChannelItem();
item.setItemTitle(label);
item.setItemLink(link);
item.setItemDescription(description);
channel.addItem(item);
and then finally, using the RssGenerator class, call the generateRss() method, in this case I’m sending the output to a Servlet PrintWriter:
PrintWriter out = response.getWriter();
RssGenerator.generateRss(doc, out);
You could just as easily write it to a file:
File file = new File("/opt/data/rss.xml");
try{
RssGenerator.generateRss(doc, file);
System.out.println("RSS file written.");
}
catch(RssGenerationException e){
e.printStackTrace();
}
Simple. Easy to use.
J2ME Recipe Viewer update & code samples
Hey, it’s working! 🙂 I ironed out the problems I was having with my J2ME Recipe Viewer application. Couple things I did to make it purr:
First, I mentioned that I was getting an IO error when trying to make a wireless connection using the phone, it became apparent that I needed way better error handling in the application once it moved from the WTK emulator phase (where I could System.out.println) to the actual phone. So I created a Form that took a String message as an argument to the constructor and then appended the String to the Form. Then in the two Thread classes that I used, if I got an error, I’d simply add the error message to the Form and use the DisplayManager (from the Core J2ME book, you can download the code) to pop that form to displayable:
midlet.displayManager.pushDisplayable(new FormError(midlet, error.toString()));
Second, I noticed that the getType() method of the HttpConnection object wasn’t returning “text/xml” like it was on the emulator, instead it was returning
“text/xml;Charset=us-ascii”
which means that the AT&T proxy is messing with the response header that I’m sending from my webserver. Because my logic looked like this:
if (code == HttpConnection.HTTP_OK && type.equals(“text/xml”)) {
after attempting to connect to my web server, I was never able to get to the XML parsing and subsequent handoff to the next screen. Anyway, I removed the check on the type (why did I have it in the first place? I don’t know…) and then everything worked!
After I got it working on the phone, I wanted to fine tune some of the other things I thought were kludgy about the app. Both the ListRecipes and ListCategory classes (which both extend the MIDP List class) presented a list of choices to the user and then on select would do two things: a) they would present an Alert to the user and then b) would start a Thread responsible for making a wireless HTTP connection, which itself would then redraw the screen with the result of the HTTP connection. The thinking behind this is probably common sense to someone who’s been developing non web-apps for some time, but it was something that I learned about while sitting in the back row of TS-2037: “Advanced MIDP Programming: Developing High Performance, Highly Responsive Wireless Applications” [download the pdf, it’s got 91 very useful slides on MIDP UI]. The problem with this approach is that the Alert dismisses itself after a set period of time (in this case 3 seconds) which resulted in a weird user interface. You’d select “Breakfast”, see an alert for 3 seconds, see the recipe categories again, and then a couple seconds later would see the breakfast recipes. So instead of an Alert, I created another Form that took a String as an argument to its constructor and then displayed that Form… which would not dismiss itself and then let the Thread take care of redrawing the screen.
Finally, I updated the search form that it too starts up a Thread, which connects to the webserver, which returns XML and then displays a list of matching recipes.
You can download the updated J2ME app and source:
J2ME Recipe Browser
As an exercise, I wrote up a J2ME client for Karensrecipes.com. As I mentioned a couple days ago, it sends HTTP requests to a servlet to receive a list of recipes as XML, and then (given a specific recipe), sends another HTTP request for the recipe details (again returned as XML). It works great on the emulator, not so good on the 3650. It looks like the phone can’t establish a connection to my server, because as I tail the Apache logs, I don’t see the request come through (I do see the request when using the emulator). I’m guessing it has something to do with the default connection for Java apps on the phone. Anyone have any experience with making wireless connections on the 3650 through J2ME?
If you’re interested, you can download the application and the source:
Alternative Uses for RSS
At work we use a homemade bug tracking system I built a couple years ago during a slow month. Although the code isn’t exactly stellar, it works pretty well and most importantly, people actually use it(!). My biggest beef with it (and all our internal web-based applications) is that every time I want to use the application, you have to login. It might be ok if I only used it once a day, but it’s something that I want to see immediately. Multiple times throughout the day someone will say “hey, what’s the status on issue #14044?” so after falling in love with SharpReader this week, I created an RSS feed for each project. Now I can keep SharpReader open in the system tray, it’ll automatically refresh the data every n minutes, and I don’t have to login each time to view the bugs.
I mention it because I think it’s one of those examples of an application being used for something that wasn’t originally intended by their creators. Luke Hutteman, who created SharpReader, probably never imagined that his 3 paned aggregator would be used to view tickets in a bug tracking system. What other ways are you using RSS?