All posts by ajohnson

Clearspace, J2EE, Open Source, Software Development, WebWork, work

Creating a Firefox Sidebar for Clearspace: Part I

1 Comment

It’s been embarassingly quiet on this blog of late, I apologize for all the delicious links, although a case could be made that blogs were originally nothing more than sharing links so maybe I shouldn’t be apologizing, but that’s a different blog post. Today I want to talk about the thing I’ve been working on at night, my non-day job if you will. A couple months ago I was reading all the hype about how JavaScript is going to take over the world and I’d been doing a lot of JavaScript during the day but I needed a project to get me through the night. Right about that time was when Twitter started taking off and I came across twitbin, which is a cool Firefox sidebar that shows you all of your friends tweets in a Firefox sidebar (the same sidebar that livehttpheaders and selenium IDE show up in), updated in real time. Like any good hacker, I wondered “how’d they do that” and started poking around the xpi file that you download to install Firefox extensions. Lo and behold, it’s JavaScript and HTML behind the scenes. Since Clearspace is one of those addictive, constantly updating, can’t get enough of it kind of applications (unlike Twitter you can actually use more than 140 characters! amazing!), I thought it would be both useful and potentially easy to create a sidebar for Clearspace, which brings me to this blog post.

I’m not sure where I started, but I’m pretty sure the first step wasn’t to create a plugin in Clearspace, I messed around for awhile with the technologies that go into creating a Firefox Extension: JavaScript, XUL (pronounced ‘zool’), install manifests, etc. I got a sample Firefox sidebar up and running that and spent way more time than I should have installing, viewing, uninstalling and restarting Firefox than I should have. I spent a lot of time digesting these sites:

I’ll discuss some of the things I ran into on the Firefox / JavaScript side in a second blog post: right now I want to talk about the Clearspace side of the plugin.

Eventually, I got to a place where I was comfortable enough with the Firefox side of things to creating the Clearspace part of the plugin. The content that is displayed in the sidebar is nothing more than plain HTML and CSS. If you install the Clearspace plugin you can actually view the content in any browser: go to http://example.com/clearspace/cf-view.jspa (replacing example.com with the host name of your installation). You’ll see that the content looks surprisingly similiar to the content that shows up on the the homepage of Clearspace, and in fact, it is the same content. The one difference between this page and other pages in Clearspace is that it never does a page reload. All the views (the login page, the settings page, the view page, etc.) are included in the resulting HTML, but hidden using CSS so that you only see one view at a time. When you click a button at the top of the page, two things usually happen: a) the current view is hidden using CSS and the new view is displayed using CSS and b) an AJAX request is sent out to a WebWork action that performs some action: logging you out, getting the updated content since your last time you viewed the page, saving your Clearfox settings, etc. This process might seem a little backwards: why not just work the same way a regular web page browsing session works where you click a link and your browser loads another page? It’s actually a limitation imposed by the Firefox extension model, so I won’t go into it here, but if you’re curious, you can do a search for ‘DOMContentLoaded firefox extension’.

So now that you (hopefully) understand how the client works, I’m going to assume that you won’t have any problems copying the ‘example’ plugin that Clearspace ships with and dive right into the pieces that are distinctive about the Clearspace part of the Clearfox plugin.

First, I needed to define the WebWork actions that Clearfox was going to use, which means I needed to create the xwork-plugin.xml file. The descriptor for the view that I mentioned earlier looks like this:

<action name="cf-view"
   class="com.jivesoftware.clearspace.plugin.clearfoxplugin.ViewAction">
   <result name="success">/plugins/clearfox/resources/view.ftl</result>
   <result name="update">/plugins/clearfox/resources/update.ftl</result>
</action>

That action is pretty standard: there are two possible results: the ‘success’ result shows the list of the 25 most recently updated pieces of content, the update result is used by an AJAX request to update the existing page with the n most recently updated pieces of content since the last refresh (which by default is invoked every 5 minute). The ViewAction class extends com.jivesoftware.community.action.MainAction, which is the WebWork action that handles the display of the homepage of Clearspace, so ViewAction simply invokes 

super.execute();

to get the same content you’d get if you viewed the homepage. When the Clearfox plugin refreshes every 5 or so minutes it invokes the 

public String doUpdate()

method, passing in the time (in milliseconds) that the last piece of content it has a record of was updated. It gets back a (presumably) shorter list of content that it then updates the view with.

The login, logout and settings actions are all using in combination with AJAX and since none of them need to provide any data, they all return HTTP status code headers:

<action name="cf-login"
   class="com.jivesoftware.clearspace.plugin.clearfoxplugin.LoginAction">
   <result name="success" type="httpheader">
      <param name="status">200</param>
   </result>
   <result name="unauth" type="httpheader">
      <param name="status">401</param>
   </result>
</action>
<action name="cf-logout"
   class="com.jivesoftware.clearspace.plugin.clearfoxplugin.LogoutAction">
   <result name="success" type="httpheader">
   <param name="status">200</param>
   </result>
</action>
<action name="cf-settings"
   class="com.jivesoftware.clearspace.plugin.clearfoxplugin.SettingsAction">
   <result name="success" type="httpheader">
      <param name="status">200</param>
   </result>
</action>

The last action is used by the Firefox Extension framework to determine if the plugin (on the Firefox side) needs to be updated. The action descriptor looks like this:

<action name="cf-updater"
   class="com.jivesoftware.clearspace.plugin.clearfoxplugin.UpdaterAction">
   <result name="success">
      <param name="location">/plugins/clearfox/resources/updater.ftl</param>
      <param name="contentType">text/rdf</param>
   </result>
</action>

The one trick about this one is that the result sets an HTTP contentType header, which isn’t remarkable except that Clearspace uses Sitemesh, which attempts to decorate everything it can parse with a header and footer. The contentType header should be a hint to Sitemesh that you don’t want the result to be decorated / wrapped with a header and footer, but apparently the hint is to subtle for Sitemesh because it attempts to wrap the result of this action anyway, which leads to the second distinctive part of this plugin.

The way we bypass Sitemesh decoration in the core product is by modifying a file called templates.xml, which gives us the ability tell Sitemesh to exclude certain paths from being decorated. In Clearspace 1.7, which will be out in a couple weeks, plugins (which can’t modify templates.xml) will have the ability to tell Sitemesh about paths that they don’t want decorated. Hence the following entry in the plugin descriptor (always located at the root of plugin and named plugin.xml):

<sitemesh>
   <excludes>
      <pattern>/cf-updater.jsp*</pattern>
      </excludes>
</sitemesh>

Third, Another thing you may have noticed if you were following along with the source code (which is available from clearspace.jivesoftware.com) is that two of the action classes (ViewAction and LoginAction if you must know) are marked with the class annotation ‘AlwaysAllowAnonymous’. This annotation tells the Clearspace security interceptor that the anonymous users should be allowed to invoke the action without being logged in. This is probably a good place to remind you of one of the differences between Clearspace and ClearspaceX. Clearspace, by default, is configured to *require* users to login before they can see any content while ClearspaceX uses the opposite default: you only need to login (usually) if you want to post content. So back to the AlwaysAllowAnonymous annotation: it’s important mostly for the Clearspace (not ClearspaceX) implementations because you want to give people the ability to invoke the action and then the action itself handles the display of the login page in it’s own specific way. The RSS related actions in Clearspace work exactly the same way: they are all annotated with the AlwaysAllowAnonymous marker and then handle security via HTTP Basic Auth (Clearspace) or simply allow anonymous usage (ClearspaceX) because feed readers

Fourth, because the Firefox part of the plugin needs to be specific to your installation, all the Firefox plugin related files need to be zipped up to create the XPI file that your users will install into Firefox. The plugin framework gives you the ability to define a plugin class:

<class>com.jivesoftware.clearspace.plugin.clearfoxplugin.ClearFoxPlugin</class>

which implements com.jivesoftware.base.plugin.Plugin. The interface specifies a method:

public void initializePlugin(PluginManager manager, PluginMetaData pluginData);

which means that your plugin will get a chance to initialize itself. The Clearfox plugin uses this initialization hook to zip up the Firefox related files. The plugin is told where it lives:

public void initializePlugin(PluginManager manager, PluginMetaData metaData) {
   File pluginDir = metaData.getPluginDirectory();
   ...

and then goes on to zip up the files located in the xpi directory of the plugin source.

So that’s that… if you got this far you probably don’t care, but I actually did a screencast of the whole thing that lives over here or you can check it out on blip.tv.

J2EE, Open Source, Systems Administration

Using a custom socket factory with HttpClient

Leave a comment

The documentation on how to use a custom socket factory in HttpClient is actually pretty good, but there’s one thing they don’t make very clear. If you want to specify a per-host socket factory like this:

Protocol p = new Protocol("https", new MySSLSocketFactory(), 443);
HttpClient httpclient = new HttpClient();
httpclient.getHostConfiguration().setHost("example.com", 443, p);
GetMethod httpget = new GetMethod("/mypath");

you must specify a path and not the full URL in the GetMethod constructor. If you don’t, in other words, if you use a constructor like this:

GetMethod httpget = new GetMethod("https://example.com/mypath");

the custom socket factory you specified in the host configuration will not be used.

J2EE, Software Development, work

Fun with Supporting Multiple Databases

2 Comments

In my day job over at Jive Software I get to work on the crazy cool Clearspace product and unless you’ve been reading the system requirements lately, you probably didn’t notice that we support six different database platforms: MySQL, Oracle, Postgres, DB2, SQL Server and HSQLDB. Clearspace borrowed a number of classes from Jive Forums so a lot of the database specific code has already been flushed out, but I ran into a couple interesting things this past week that I thought needed to be blogged.

The first thing I ran into was a varchar case sensitivity issue: DB2, Oracle, Postgres and HSQLDB are all sensitive about case with respect to the values they store. So if you do this:

INSERT INTO appuser(username,password) VALUES('Administrator','password');

and then try to retrieve the user:

SELECT username,password FROM appuser WHERE username = 'administrator'

you’ll get different results than you will with MySQL and SQL Server (which are both case insensitive by default). Bottom line: if you plan supporting an application on a variety of database servers, make sure you toLowerCase() or toUpperCase() the string values you store if you plan on doing look ups against those values. If you’re into this kind of thing, page 87 of the SQL 92 standard appears to discuss the case sensitivity issues, but I can’t make heads of tails of it. Any good specification readers out there?

The second thing involved Oracle, which has this wonderful feature where you can’t insert an empty string into a varchar column. Example:

INSERT INTO appuser(username, password) VALUES('administrator', '');

Bennett McElwee blogged about this feature in more detail on his blog, but the jist of this nugget of goodness is that if you attempt to insert an empty space into an Oracle varchar, your empty space will get converted to 

null

which is not helpful in the least bit. Consider yourself warned.

Finally, and again Oracle. Assume you have the sample table I used above with two columns: username and password. Assume further that you want to modify the username column to be 100 chars instead of 50. You write an ALTER statement that looks something like this for SQL Server:

ALTER TABLE appuser ALTER COLUMN username nvarchar(100) NOT NULL

on MySQL you’d have this:

ALTER TABLE appuser MODIFY COLUMN username nvarchar(100) NOT NULL

on DB2:

ALTER TABLE appuser ALTER COLUMN username varchar(100) NOT NULL

All very similar. But Oracle, no, if you tried this:

ALTER TABLE appuser MODIFY(COLUMN username varchar(100) NOT NULL)

on Oracle you’d get this error:

ORA-01451: column to be modified to NULL cannot be modified to NULL

See Oracle, for whatever reason, decided that if you are going to modify the column that you can ONLY specify the nullability of a column IF the nullability itself is changing. Which is ridiculous nice.

And yes, nullability is a word.

Software Development, Syndication, work

Using ROME to get the body / summary of an item

Leave a comment

I’ve been using ROME for a couple years now and I’m still learning new things. Today I was working on an issue in Clearspace where we give users the ability to show RSS / Atom feeds in a widget, optionally giving them the choice to show the full content of each item in the feed or just a summary of each item in the feed. The existing logic / pseudo-code looked something like this:

for (SyndEntry entry : feed.getEntries()) {
  if (showFullContent) {
    write(entry.getContents()[0].value);
  } else {
    write(entry.getDescription().value);
  }
}

The assumption was that description would return a summary and contents would return the full content. The problem is that Atom and RSS are spec’ed umm.. differently. RSS 2.0 says that ‘description’ is a synopsis of the item but then goes on in an example to show how the description can be much more than just a short plain text description. So then you’re left with descriptions that aren’t really a synopsis, it’s the full content… or it is sometimes and sometimes not. Then Atom came along with well defined atom:summary and atom:content elements which means ROME had to figure out a way to map description and content-encoded elements in RSS to atom:summary and atom:content. Dave Johnson summarized the mappings nicely in a blog post discussing the release of ROME 0.9, in short the mapping looks like this:

RSS <description> <--> SyndEntry.description <--> Atom <summary>
RSS <content:encoded> <--> SyndEntry.contents[0] <--> Atom <content>

Anyway, all this is to say that if you’re doing any work with SyndEntry, you’ll need to check both description and contents. Generally, if you’re looking for the full content, check the value of contents first. If that’s null, check the value of description. If you’re looking for a summary, check the value of description first BUT don’t assume that you’ll actually get a short summary. Use something like StringUtils.abbreviate(…) to make certain that you’ll get a short summary back and not the entire content.

Systems Administration

Finding the number of files of a certain file type that a process has open

Leave a comment

Used this week when trying to debug a problem with an application that was resulting in errors that said “too many files open”:

/usr/sbin/lsof -p $processID | grep .$ext | wc -l

where $processID is the ID of the process that is opening the files and $ext is the (most likely) three letter file extension of the file that is being open.

Also, if you run:

ulimit

and you get ‘unlimited’ as the result, you’d be wrong to assume that you have an unlimited number of file handles available to you:

ulimit -n

will give you the total number of file handles you’re allowed.

And that’s one to grow on.

More here.