All posts by ajohnson

Interface Design, J2EE

Transparent PNG Charts with JFreeChart

20 Comments

If you’re one of the 3 people in the world that need to create a transparent PNG image using JFreeChart, you’ve come to the right place:

JFreeChart chart = ChartFactory.createXYBarChart(...);
chart.setBackgroundPaint(new Color(255,255,255,0));
...
KeypointPNGEncoderAdapter encoder = new KeypointPNGEncoderAdapter();
encoder.setEncodingAlpha(true);
encoder.encode(chart.createBufferedImage(width, height, BufferedImage.BITMASK, null));

The key is that you must use the KeyPointPNGEncoderAdapter, which is slower for big charts, but is the only one of the two PNG encoders that JFree ships with that has the ability to do alpha transparency. On systems running JDK 1.4 and above, the ImageEncoderFactory will return the SunPNGEncoderAdapter, which does not support alpha transparency.

Also, if you’re planning on using alpha transparency in a web application that needs to support IE, you’ll want to check out this JavaScript fix by Bob Osola.

J2EE, Open Source, Software Development

Using Apache James and JavaMail to implement Variable Envelope Return Paths

7 Comments

I submitted a skeleton of the article that follows to JDJ, was told they would like to ‘commission’ it and then finally submitted it only to never hear anything back from them. So instead you get to read it here. Enjoy!

—————————–

Anyone who has spent any time working a application that sends emails has come across more than their fair share of bounced emails. If you actually read the bounced emails, you probably noticed that many of them either came from the ISP’s error handler (mailer-daemon@isp.com) or from an email address that wasn’t on your mailing list. The bounced message may not even have contained a copy of the original message. All of above scenarios make it very hard to figure out who the original message was sent to. Enter Daniel Bernstein, also known as djb, who in 1997, in response to this problem of matching email bounce messages to subscription addresses, wrote a paper describing a technique he called Variable Envelope Return Paths or VERP for short. In the paper, he describes process as:

…each recipient of the message sees a different envelope sender address. When a message to the djb-sos@silverton.berkeley.edu mailing list is sent to God@heaven.af.mil, for example, it has the following envelope sender:

djb-sos-owner-God=heaven.af.mil@silverton.berkeley.edu

If the message bounces, the bounce message will be sent back to djb-sos-owner-God=heaven.af.mil@silverton.berkeley.edu.

If God is forwarding His mail, the bounce message will still go to djb-sos-owner-God=heaven.af.mil@silverton.berkeley.edu. No matter how uninformative the bounce message is, it will display God’s subscription address in its envelope.

But you probably noticed that this article isn’t only about VERP: Apache James is a full featured SMTP, POP3 and NNTP server built using 100% Java and more importantly it has been designed from the ground up to be a mail application platform. The James mail application platform makes it a perfect candidate for handling bounced messages using VERP. Similarly, the JavaMail API is a framework for building mail and messaging applications using SMTP and POP3. JavaMail makes it easy to customize the envelope sender address, which means Java developers can utilize JavaMail on the client and James on the server to build email applications that enables VERP.

This article will describe an example VERP implementation, show how JavaMail can be used to modify the envelope sender address and will then illustrate how James can be used to recognize and process bounced email messages. It is not intended to be a in-depth look at either the Apache James mail server or the JavaMail API. If you’re interested in learning more about Apache James, a product review is available on the Sys-Con.com website (http://java.sys-con.com/read/38667.htm) and an extensive introduction to Apache James on IBM developerWorks (http://www-128.ibm.com/developerworks/library/j-james1.html). The JavaMail API is also reviewed on the Sys-Con.com website: http://java.sys-con.com/read/36545.htm.

VERP and JavaMail
Let’s start by looking at the email newsletter that a fictional store called �Javazon’ is sending to its’ customers. The developers at Javazon have been using the JavaMail API to successfully send the newsletter through their mail server using code similar to the example below.

String senderemail = "deals@javazon.com";
String toemail = "ajohnson@cephas.net";
Properties props = new Properties();
props.put("mail.smtp.host", mailserver);
Session session = Session.getInstance(props, null);
javax.mail.Message m = new MimeMessage(session);
m.setFrom(new InternetAddress(senderemail));
m.setSubject("New Deals at Javazon!");
m.setRecipient(javax.mail.Message.RecipientType.TO,
new InternetAddress(toemail));
m.setContent(content, "text/plain");
Transport.send(m);

The above code will produce an email message with headers that look like this:

Date: Wed, 26 Apr 2006 21:00:21 -0000
From: deals@javazon.com
To: ajohnson@cephas.net
Subject: New Deals at Javazon!

Because they want to good email citizens, the developers at Javazon use the POP3 functionality in JavaMail to retrieve the emails that bounce back to the address specified as �senderemail’ in the example above. Unfortunately, many of the bounce emails come from daemon accounts (instead of the recipient email address) which makes it difficult to figure out what email address the original message was sent to.

As mentioned at the start of this article, the only way to address the bounces that come from daemon accounts is to use VERP, which is a two part process. The first is relatively simple. An email message, according to the SMTP RFC-821 Section 2, is composed of two parts: an envelope which contains the SMTP source and destination addresses and the message, which consists of the headers and message body. To create a VERP capable email message, you need only modify the envelope, which is easily accomplished using the instance of java.util.Properties associated with the javax.mail.Session. Modifying the first example, the developers would end up with this:

String senderemail = "deals@javazon.com";
String toemail = "ajohnson@cephas.net";
String verpFrom = "deals-" + toemail.replaceAll("@", "=") + "@javazon.com";
Properties prop = new Properties();
props.put("mail.smtp.from", verpFrom);
props.put("mail.smtp.host", mailserver);
Session session = Session.getInstance(props, null);
  javax.mail.Message m = new MimeMessage(session);
m.setFrom(new InternetAddress(senderemail));
m.setSubject("New Deals at Javazon!");
m.setRecipient(javax.mail.Message.RecipientType.TO,
new InternetAddress(toemail));
m.setContent(content, "text/plain");
Transport.send(m);

When excecuted, the code above would create an email message with headers that look like this:

Return-Path: deals-ajohnson=cephas.net@javazon.com
Date: Wed, 26 Apr 2006 21:00:21 -0000
From: deals@javazon.com
To: ajohnson@cephas.net
Subject: New Deals at Javazon!

Notice the different “Return Path:” header from the first email? If the email message bounces back to Javazon, it will go to the email address associated with the ‘Return Path’ header: “deals-ajohnson=cephas.net@javazon.com” rather than “deals@javazon.com”. This is where Apache James comes into the picture.

VERP and James
James can be configured and used like any other email server, but it’s real power comes from the ability it gives Java developers to plug right into the mail processing pipeline. James enables you to process email messages in a same way you might process HTTP requests that come into servlet container like Tomcat, but in a more flexible manner. If you want to preprocess (or postprocess) HTTP requests in Tomcat, you first create a class that implements the javax.servlet.Filter interface and then you create an entry in your web.xml that matches certain requests to that class. Your configuration might look something like this:

<filter>
  <filter-name>myfilter</filter-name>
  <filter-class>com.javazon.web.filters.GZipFilter</filter-class>
</filter>
<filter-mapping>
  <filter-name>myfilter</filter-name>
  <url-pattern>*.jsp</url-pattern>
</filter-mapping>

The servlet container limits how you match requests to a filter: you are limited to pattern matching on the URL. Instead of a <filter> and <filter-mapping>, James gives you a <mailet>, which is made up of two parts: Matchers and Mailets. They are described on the James wiki:
“Matchers are configurable filters which filter mail from a processor pipeline into Mailets based upon fixed or dynamic criteria.

Mailets are classes which define an action to be performed. This can cover actions as diverse as local delivery, client side mail filtering, switch mail to a different processor pipeline, aliasing, archival, list serving, or gateways into external messaging systems.”

James ships with a number of Mailets and Matchers that you can use without writing a line of code, but the developers at Javazon will need to write their own Matcher and Mailet to handle the bounces generated from their email campaigns.

So the first thing the developers at Javazon are going to need to do is create a class that intercepts the bounces emails. A matcher class can be created in one of two ways: a) create a class that implements the org.apache.mailet.Matcher interface, or b) create a class that extends the org.apache.mailet.GenericMatcher class. Because GenericMatcher already implements both Matcher and MatcherConfig and because it provides simple version of the lifecycle methods, the path of least resistance is to extend GenericMatcher. The NewsletterMatcher class is going to ‘match’ only the recipients where the address of the recipient starts with the string “deals-“:

public class NewsletterMatcher extends GenericMatcher {
  public Collection match(Mail mail) throws MessagingException {
    Collection matches = new ArrayList();
    Collection recipients = mail.getRecipients();
    for (Iterator i=recipients.iterator(); i.hasNext();) {
      String recipient = (String)i.next();
      if (recipient.startsWith("deals-")) {
       matches.add(recipient);
      }
    }
    return matches;
  }
}

The NewsletterMatcher class, as you can see, returns a Collection of String objects, each presumably an email that has bounced. To do something with these matches, the developers will need to write a class that either implements the org.apache.mailet.Mailet interface or a class that extends the org.apache.mailet.GenericMailet class. Again, it will be simpler to extend the GenericMailet class:

public class NewsletterMailet extends GenericMailet {
  private static CustomerManager mgr = CustomerManager.getInstance();
  public void service(Mail mail) throws MessagingException {
    Collection recipients = mail.getRecipients();
    for (Iterator i=recipients.iterator(); i.hasNext();) {
        String recipient = (String)i.next();
        if (recipient.startsWith("deals-")) {
        int atIndex = recipient.indexOf("@");
        String rec = recipient.substring(0,atIndex)
        .replaceAll("=", "@")
        .replaceAll("deals-", "");
        mgr.recordBounce(rec);
        mail.setState(Mail.GHOST);
      }
    }
  }
}

In the above example, the NewsletterMailet class overrides the service() method in the GenericMailet class, loops over the list of recipients in the given email message and then checks to see if the recipient email address starts with the string “deals-“. If the recipient email address does start with “deals-“, then the class decodes the original recipient address by retrieving what is generally the username part of the email address, replacing the equals sign (=) with an @ sign and then replacing the “deals-” prefix. Then the Newsletter mailet class uses CustomerManager (a class that the Javazon developers use to manage customer information) to record the bounced email. If you were to step through the process, you’d see the recipient email address start as something like this:

deals-ajohnson=cephas.net@javazon.com

and then change to this:

deals-ajohnson=cephas.net

and finally to this:

ajohnson@cephas.net

The last step is to wire the mailet and matcher classes together in the Apache James configuration file, which is usually located here:

$JAMES/apps/james/SAR-INF/config.xml

You’ll need to make a number of entries. First, you’ll need to let James know where it should look for the mailet and matcher classes you’ve created by creating <mailetpackage> and <matcherpackage> entries inside the <mailetpackages> and <matcherpackages> elements:

<mailetpackages>
  ...
  <mailetpackage>com.javazon.mailets</mailetpackage>
</mailetpackages>
<matcherpackages>
  ...
  <matcherpackage>com.javazon.matchers</matcherpackage>
</matcherpackages>

Then add references to the matcher and the mailet using a <mailet> element like this:

<mailet match="NewsletterMatcher" class="NewsletterMailet">
  <processor>transport</processor>
</mailet>

The match attribute of the mailet element specifies the name of the matcher class that should be instantiated when the matcher is invoked by the spool processor and the class attribute specifies the name of the mailet that you want invoked should the matcher class return any hits.

After adding these configuration entries and adding the compiled classes to the $JAMES/apps/james/SAR-INF/lib/ directory, restart the James process.

Testing
In order to test the configuration / application, you’ll need to have a James server configured and available via the internet via port 25 with a valid DNS name and a corresponding MX record. As an example. the system administrator at Javazon would configure a machine with James, make it available to the internet on port 25 and assign it a domain name like bounces.javazon.com. The developers could then send an invalid email using JavaMail to:

bounceme@javazon.com

(an account which probably doesn’t exist on the main javazon.com mail server) with a return path of :

deals-bounceme=javazon.com@bounces.javazon.com.

The bounce email will be sent to the server associated with the MX record for the domain name bounces.javazon.com, which should be the server the system administrator set up above. The NewsletterMatcher class should ‘match’ on the “deals-” prefix and then pass it to the NewsletterMailet, which should record the bounce using the CustomerManager instance.

Conclusion
After reading this article, you should hurry on over to the Apache James website, download the latest distribution and read the documentation. There are a number of other interesting ways you can improve your email processing by extending James using mailets and matchers.

References
————————————————————
JavaMail
· http://java.sun.com/products/javamail/
· http://jdj.sys-con.com/read/36545.htm
· http://www.ibm.com/developerworks/java/edu/j-dw-javamail-i.html

VERP
· http://cr.yp.to/proto/verp.txt

Apache James
· http://james.apache.org/
· http://jdj.sys-con.com/read/38667.htm
· http://www.ibm.com/developerworks/java/library/j-james1.html
· http://www-128.ibm.com/developerworks/java/library/j-jamess2.html
· http://james.apache.org/spoolmanager_configuration_2_1.html

JavaScript, Systems Administration

Setting up multiple versions of IE plus cookies

Leave a comment

The web has got the whole ‘multiple versions of IE’ fairly well documented already, but you have to scroll way down to the bottom of the second article to see that if you want cookies to work, you’ll have to download wininet.dll from Microsoft.

So the full directions:

  1. Download IE 5.0 or IE 5.5
  2. Use a zip utility to unpack either / both of the above zip files.
  3. Download wininet.dll from microsoft.com.
  4. Use a zip utility to unpack the .exe file that you downloaded in step 3.
  5. Copy the resulting Wininet.dll from step 4 to the location that you unzipped the IE files in step 2.
  6. Start up IE and go!
JavaScript, Software Development

Prototype, tinyMCE, ‘too much recursion’

1 Comment

If you happen to be using TinyMCE and the Prototype library together and you’re getting a ‘too much recursion’ error in FireFox, make sure to upgrade to the latest version of Prototype, which at the time of this post is version 1.5.0. Near as I can tell, the only way to get version 1.5.0 is to either a) check it out of the subversion repository or b) download script.aculo.us, which contains version 1.5.0 in the lib directory of the distribution.

I think the ‘too much recursion’ Prototype / TinyMCE problem happens because Prototype stores a copy of the Array reverse function in a property called _reverse:

Array.prototype._reverse = Array.prototype.reverse;

and then redefines the reverse function, adding a argument ‘inline’:

reverse: function(inline) {
  return (inline !== false ? this : this.toArray())._reverse();
}

Somehow (and I’m not sure how this would happen) the copy happens again, which means that _reverse() would point to the redefined reverse() method function, which of course points to _reverse(), which leads to a infinite loop. Hence, ‘too much recursion’.

Regardless, the changelog from latest version of Prototype has a pointer to issue #3951 (but it doesn’t appear that the issues are public): in short, version 1.5.0 of Prototype does a check to make sure that _reverse() hasn’t been defined:

if (!Array.prototype._reverse)
  Array.prototype._reverse = Array.prototype.reverse;

In related news, if you’re using script.aculo.us with TinyMCE, make sure to embed the script.aculo.us js after the TinyMCE js.

JavaScript

Can’t add hyperlink with onclick to IE using DOM

18 Comments

Last week I was attempting to dynamically add hyperlinks that contained an onclick event to a document. The hyperlink, when written to the browser, would look something like this:

<a href="#" onclick="fireEvent(); return false;">fire away!</a>

and I was using code that looked something like this to produce the links (and yes, I know about real links):

<script>
function createLink(id) {
  var node = document.getElementById(id);
  var aNode = document.createElement("a");
  node.appendChild(aNode);
  aNode.setAttribute("href","#");
  aNode.setAttribute("onclick", "fireEvent(); return false;");
  aNode.appendChild(document.createTextNode("fire away!"));
}
function fireEvent() {
  alert('fire!');
}
</script>
<a href="#" onclick="createLink('mytable'); return false;">create link</a>
<table cellpadding="0" cellspacing="0" border="0" width="100%">
<tbody>
<tr>
  <td id="mytable"></td>
</tr>
</tbody>
</table>

Running the above code in Firefox worked fine, the fireEvent() method was invoked when you clicked the ‘fire away!’ link. The same exact code in IE didn’t work. Instead, it appeared that IE completely ignored the onclick event and just went with the value of the href attribute. The simple workaround, in this case at least, was to use the innerHTML property of the cell I was adding the link to, so the createLink() function became:

function createLink(id) {
  var node = document.getElementById(id);
  node.innerHTML = '<a href="#" onclick="fireEvent(); return false;">fire away!</a>';
}

Am I wrong in thinking that this is a bug?

Related:

Early event handlers
Javascript: createElement, innerHTML and IE security zones
can’t put a row into a table in IE using DOM methods


Updated 7/3/2006: My boss Bill sent a note mentioned the behaviors library which uses AOP techniques to reduce what the author calls ‘scattering’ and ‘tangling’ and would may also be a solution to the problem mentioned above.

OSX, Systems Administration

Using a HOSTS file on OS X

Leave a comment

In the Windows world it’s a reasonably straightforward process to add pseudo domain names using the HOSTS file usually located in c:\windows\system32\drivers\etc\. Open the file, add a name / IP address entry, save the changes and you’re off and running. It’s slightly more involved on OS X as this site details. OS X uses lookupd to do name resolution and by default (at least on my machine) it is configured to look to a DNS server before referencing the local HOSTS file, which is not how Windows works. Turns out it’s reasonably easy to change (as detailed on the macwrite.com link mentioned above):

cd /etc
sudo mkdir lookupd
cd lookupd
ls
cp hosts hosts.original
sudo echo LookupOrder Cache FF NI DNS DS > hosts
sudo kill -HUP 'cat /var/run/lookupd.pid'

After that’s done, you can add / delete / modify domain name to IP address mappings using the NetInfo Manager tool.