All posts by ajohnson

Daily Links

Links: 7-7-2005

Leave a comment
J2EE, SOAP, Software Development

Fun with Axis / SOAP, Java and Calendar

5 Comments

The fun with dates didn’t stop at the last post. This one took longer to track down but I learned alot more from it. I’m working on both sides of an Apache Axis SOAP implementation where the server side system persists events and client applications can query to find events and CRUD events against this server side application. So the server presents an event:

...
public class EventRemote {
private String name;
private String address;
private Calendar eventdate;
...
// getters & setters omitted
}

and methods that allow client applications to search for events and create / retrieve / update and delete them:

...
public class EventManagerRemote {
public void updateEvent(EventRemote event) {
...
}
public void createEvent(EventRemote event) {
...
}
public EventRemote findEventByID(long eventid) {
...
}
// other methods omitted
}

For the last couple weeks the clients and servers had no problems, create, retrieve, update, delete and search all worked as expected. But recently a user requested a change to the UI of the client system which resulted in a change to the way the client application needed to format the eventdate (a Calendar object). In the past the client would retrieve the event and then format the eventdate property using a SimpleDateFormat:

EventRemote event = manager.findEventByID(12);
Calendar eventdate = event.getEventdate();
SimpleDateFormat formatter = new SimpleDateFormat("M/d/yyyy h:m aa");
String strdate = formatter.format(eventdate.getTime());

So if my event was today (June 30, 2005 at 7:30pm), the above snippet would print:

6/30/2005 7:30 PM

The change to the client I mentioned above meant that I needed to access each element of the Calendar object separately, so instead of having one big datetime string, I need to know that the month was June, the date ’30’, the year ‘2005’ and so on. Since the getMonth()/Day()/Year() methods of the Date class have been deprecated since v1.1, I had to use the Calendar class get(int field) methods. Not a problem, there’s some different things you have to work through in the Calendar class (ie: the month field has values that range from 0 to 11 instead of 1 to 12), but it’s doable:

EventRemote event = manager.findEventByID(12);
Calendar eventdate = event.getEventdate();
int day = eventdate.get(Calendar.DATE);
int year = eventdate.get(Calendar.YEAR);
...
int hour = eventdate.get(Calendar.HOUR);
int minute = eventdate.get(Calendar.MINUTE);
int am_pm = eventdate.get(Calendar.AM_PM);

I printed out the results (again assuming a date of June 30,2005 at 7:30pm) and instead of getting that same date I got this:

6/30/2005 11:30 PM

which was exactly 4 hours ahead of the time I wanted. I dug into Axis source code and saw that the CalendarDeserializer class explicity sets the TimeZone of the returned Calendar object to GMT (to make it UTC right?). Understandably, this causes the HOUR property of the Calendar object to jump four hours ahead (I’m in EST, GMT/UTC is currently 4 hours ahead of EST, it’ll be 5 hours during the winter months, explanation). The workaround (although I’m not sure it’s a good idea), was to set the TimeZone of the returned Calendar object back to EST. This probably doesn’t make alot of sense without seeing some sample code:

Calendar calendar = new GregorianCalendar(2005, 5, 30, 19, 30);
String utcformat = "yyyy-MM-dd'T'HH:mm:ss.SSS'Z'";
SimpleDateFormat zulu = new SimpleDateFormat(utcformat);
String serializeddate = zulu.format(calendar.getTime());
// string gets sent along the wire via soap
// CalendarDeserializer parses date string
Calendar deserial = Calendar.getInstance();
Date deserialdate = zulu.parse(serializeddate);
deserial.setTime(deserialdate);
// CalendarDeserializer explicitly sets TZ to GMT
deserial.setTimeZone(TimeZone.getTimeZone("GMT"));
SimpleDateFormat formatter = new SimpleDateFormat(utcformat);
// will print '2005-06-30T19:30:00.000Z'
System.out.println(formatter.format(deserial.getTime()));
// will print '11', should print '7'
System.out.println(deserial.get(Calendar.HOUR));

So I understand that the date gets sent along the wire in UTC format, which strips out timezone information the date was originally encoded with, my question is this: why does the HOUR property of the Calendar object reflect UTC time (ie: 11:30pm) while the formatted time reflects EST time (7:30pm)?

J2EE, Software Development

Fun With Java dates

1 Comment

Sorry for not posting anything other than links recently, the kid is wearing me out almost as much as working with Java dates. I haven’t had to work with java.util.Date and java.util.Calendar all that much, most of the time it’s been pretty simple stuff but for whatever reason this past week has been spent waist deep in the Calendar and Date API’s trying to fix a bug in a deployed application. The code I was using looked something like this:

// date entered by user through a combination
// of drop down lists
String strdate = "12/6/1975 5:30 PM"
SimpleDateFormat formatter = new SimpleDateFormat("M/d/yyyy H:m a");
Date eventdate = formatter.parse(strdate);
System.out.println(formatter.format(eventdate));

and out of that I expected to see a date of 12/6/1975 at 5:30pm, it should in fact have printed back the exact same string I entered right? Instead I got:

12/6/1975 5:30 AM

I spent awhile trying different things (am I entering the date wrong? is there something wrong with the timezone? etc..) but eventually noticed that ‘h’ is not the same thing as ‘H’ in the date format string. Using an upper case ‘H’ signifies to the SimpleDateFormat parser that you want it to look for a 24 hour time format (ie: instead of 5:30 I should have used 17:30). In hindsight that’s somewhat obvious, but the tricky thing is that I included the PM designation, so in theory, shouldn’t the SimpleDateFormat have picked up the fact that I wanted a time in PM? I think it should have at least have thrown an exception that says you’ve entered a time in the morning and an AM PM designation of evening. True Date class experts would say at this point that I should have set the SimpleDateFormat to be strict in it’s parsing:

formatter.setLenient(true)

which I did, but it still didn’t throw an exception. Should it have?

Daily Links

Links: 6-27-2005

Leave a comment
Daily Links

Links: 6-17-2005

Leave a comment
Open Source, Systems Administration

AWStats Installation Notes

6 Comments

I tried installing AWStats a couple months ago on my server, got frustrated after an hour or two of reading the documentation and trying to figure out permissions problems, gave up and then tried again about two weeks ago, this time with more patience. Maybe after reading my notes (on RedHat Linux) someone else won’t have to exercise as much patience.

First step: download AWStats, unzip (I unpacked to /usr/local/awstats/) and then execute the following script to start the setup process:

perl awstats_configure.pl

After you complete that step, you’ll have a couple AWStats directives in your Apache configuration. Since some of the sites I’ve written use servlet filters to do URL rewriting and because I wanted to minimize the number of AWStats entry points (AWStats was recently hacked), I setup a virtual host in Apache as a reporting domain. I then moved the AWStats directives inside this reporting virtual domain.

Next, using the sample awstats.mysite.com.conf, you’ll need to create the a domain configuration file for every site that you want to report on and place each one in /etc/awstats/ (read more about the domain configuration files here, scroll down to step #4 and #5). So I ended up with something like:

# ls /etc/awstats
awstats.cephas.net.conf
awstats.blackberryblog.com.conf
...

After you’ve got all your configuration files setup, you’ll want to run awstats against all your old log files (assuming that you’ve got old log files that you want to process). There’s no easy way of getting all your old log files processed, I had to modify the configuration file to point to the zipped up log file:

LogFile="gzip -d </usr/hosts/yoursite.com/logs/200205.zip | "

run awstats:

perl awstats.pl -config=yoursite.com -update

and then modify the configuration file to point to the next month. If I was better at shell scripting I’m sure I could have worked out some way of automating this, but I didn’t have that many log files laying around so it wasn’t a big deal. After you’ve processed all your old log files, you’ll want to point AWStats to look at your current log files. I configured my machine so that it always processes the log file based on yesterday’s date, so my configuration looked like this:

LogFile="/usr/hosts/yoursite.com/logs/%YYYY-24%MM-24.log"

and then I setup a cron job to execute AWStats every night by saving this snippet of code as a script and placing it in your /etc/cron.daily/ folder (cron.daily runs based on the settings in /etc/crontab)

#!/bin/bash
cd /usr/local/awstats/tools
perl awstats_updateall.pl now

So now you’ve got your configuration files, you’ve got a script that processes your logs automatically every night and you want to view the reports from a web browser. So you point your web browser to:

http://reporting.yoursite.com/awstats/awstats.pl?config=yoursite.com

and if you’re like me at this point you’ll run into an error message that makes no sense at all:

[Fri Jun 03 15:52:42 2005] [crit] [client xxx.xxx.xxx.xxx] (13)Permission denied: /usr/local/awstats/.htaccess pcfg_openfile: unable to check htaccess file, ensure it is readable

The funny thing was that I didn’t have an htaccess file anywhere near the awstats configuration, this error message drove me batty. After googling for an exceptionally long period of time, I updated both /var/lib/awstats/ and /usr/local/awstats/ so that the Apache user owned them.

chown apache.apache /var/lib/awstats
chown apache.apache /usr/local/awstats

Finally, unless you want everyone to see your stats (and allow anyone to access the AWStats.pl script), you’ll want to lock down the cgi-bin directory so that only valid users can access your system. Create a .htaccess and a .htpasswd file in your /usr/local/awstats/wwwroot/cgi-bin/ directory (making to sure to set AllowOverride All in your directive inside the VirtualHost in apache). Configure your .htaccess to only allow access to valid-user:

AuthName "reporting"
AuthType Basic
AuthUserFile /usr/local/awstats/wwwroot/cgi-bin/.htpasswd
Require valid-user

You can create the .htpasswd file by running this command:

htpasswd -c $filename $username

and then add users:

htpasswd $filename $username

and make sure to modify your AWStats configuration file to allow access to __REMOTE_USER__ or the specific user that you want to grant access to that file.

You should now have a working, secure and fully automated AWStats reporting system! Congratulations!