This last week someone sent me a CD with fifty 60 second tracks (it was a presentation that someone inexplicably saved as fifty separate tracks instead of one contiguous fifty minute block) that they wanted ripped into a single MP3. Ripping is easy with CDex and my initial google search left with with Audacity, which looked promising but I couldn’t figure out how to merge multiple MP3 files into a single file (using Audacity that is). Finally, I came across this little gem on binarybonsai.com, where a commenter mentioned that you can use the DOS copy command to append one files (or multiple files) to one another, creating one big file from many. Syntax is as follows:
> copy /b file1.mp3 file2.mp3 file3.mp3 end_result.mp3
which combines file1.mp3, file2.mp3 and file3.mp3 into end_result.mp3. The /b switch indicates that you’re copying binary files.
Category Archives: Systems Administration
Top 15 Ant Best Practices
Good stuff over at OnJava.com, the article “Top 15 Ant Best Practices” gives 15 recommendations on how you should be using Ant to structure and complete your builds. We’re using most of these already at work; I’d be curious to know the author’s take on the deployment system we’ve developed using Ant. I’ve got Subversion handling source control, so on the production machines, I have a build file called build-production.xml that I invoke from the command line using a batch file:
ant -buildfile build-production.xml
which a) checks out the latest build from Subversion / source control, b) executes the default build file (which is in source control) which compiles the project and then wraps up the result in a war file, c) stops the Tomcat Windows service, d) deletes the existing application from the $tomcat/webapps directory, e) unpacks the application from the .war creates in step b, and then finally f) starts the Tomcat Windows service.
It’s worked out pretty well so far in that it decouples the build file for the project from having to know anything about the environment in which it’s executing so developers can write their own build-development.xml which maybe does something similar to the build-production.xml, but deploys to a different location, or even a different application server.
The wrapper build file (be it build-production.xml, build-local.xml, etc..) can also override properties set in the project build.xml. In our projects, I’ve got the build.xml file conditionally setting properties in log4j.properties using the <propertyfile> tag. There are two such targets: one for production that sets the appender to SMTPAPPENDER and the logging level to ERROR and one for development that sets the appender to STDOUT and the logging level to INFO. Said targets use the if and unless attributes of the <target> tag like so:
<target name="build.resources.debug" if="project.debug">
...
and
<target name="build.resources.production" unless="project.debug">
...
This way, unless I set the project.debug property, the system will always be in ERROR / SMTPAPPENDER mode but developers can override that property at compile time using the wrapper build-local.xml to set the project.debug property, ie:
<ant antfile="build.xml">
<property name="project.debug" value="true"/>
</ant>
Do you do it better? Can you one up me? I’ve love to hear about it.
Tomcat 5.0.x and ClientAbortException
I’ve got an application deployed on Tomcat 5.0.28 that, when in production, uses log4j with the SMTP appender to send relevant debug / error messages via email. Soon after deploying the application I began receiving messages from the application which contained this stack trace:
java.io.IOException: ClientAbortException: java.net.SocketException: Connection reset by peer: socket write error at org.apache.coyote.tomcat5.OutputBuffer.doFlush(OutputBuffer.java:331) at org.apache.coyote.tomcat5.OutputBuffer.flush(OutputBuffer.java:297) at org.apache.coyote.tomcat5.CoyoteOutputStream.flush(CoyoteOutputStream.java:85) at sun.nio.cs.StreamEncoder$CharsetSE.implFlush(Unknown Source) at sun.nio.cs.StreamEncoder.flush(Unknown Source) at java.io.OutputStreamWriter.flush(Unknown Source) at java.io.BufferedWriter.flush(Unknown Source)....
The stack trace isn’t anything special: you’ll get a ClientAbortException whenever the HTTP client on the other end closes the connection prematurely (ie: before Tomcat has a chance to complete the flush of information to the output stream). The problem is that it pollutes your log file, or in my case, my inbox because I was receiving an email every time one of these errors occurred. My log4j configuration file looked something like this:
log4j.rootCategory=ERROR, SMTPAPPENDER
log4j.appender.SMTPAPPENDER.To=me@mydomain.com
log4j.appender.SMTPAPPENDER.layout=org.apache.log4j.PatternLayout
log4j.appender.SMTPAPPENDER=org.apache.log4j.net.SMTPAppender
log4j.appender.SMTPAPPENDER.Subject=[my app] ERROR
log4j.appender.SMTPAPPENDER.BufferSize=512
log4j.appender.SMTPAPPENDER.From=errors@mydomain.com
log4j.appender.SMTPAPPENDER.layout.ConversionPattern=%-4r [%t] %-5p %c %x - %m%n
log4j.appender.SMTPAPPENDER.SMTPHost=localhost
which, in log4j parlance, means that I want any messages that are of type ERROR or FATAL sent to me via email. Instead I should have started it off with this:
log4j.rootCategory=FATAL, SMTPAPPENDER
log4j.category.com.mycompany=ERROR, SMTPAPPENDER
...
which translated means that I want any FATAL errors in any package sent to me via the SMTPAPPENDER but any messages of type ERROR or FATAL in the package com.mycompany sent to me as well. This effectively cuts out the unnecessary sending of ClientAbortException messages, while still generating the messages that I want to receive.
(Note: if your problem isn’t with log4j, but instead with your Tomcat log files, check out the debug settings in Server, Service, Engine and Host elements of your Tomcat conf\server.xml.)
Older MySQL JDBC Driver issues with INSERT, UPDATE, DELETE
I haven’t worked on my wife’s recipe site in ages. Originally written as an experiment with JSP and Java, I think my mom and Karen are the only people that use it (which is fine with me). In any case, I recently moved it to a new host and saw a weird issue pop up in catalina.out where any INSERT, UPDATE or DELETE statements were erroring out with the message:
java.sql.SQLException: Can not issue data manipulation statements with executeQuery()
Turns out that earlier versions of the MySQL JDBC driver allowed you to run INSERT / UPDATE / DELETE statements using the executeQuery() method of the PreparedStatement class, which violated the JDBC specification, but worked nonetheless. The newer versions are JDBC compliant and don’t allow that behavior, which causes the above mentioned SQL exception. More from the MySQL forums.
Modifying Memory Allocation on Tomcat 5.x on Windows
A search for ‘tomcat 5 outofmemory windows‘ returns about 2000 hits, not one of them shows you how can increase the amount of memory allocated to Tomcat via the Apache Tomcat Properties applet when running on Windows (some point to this Tomcat configuration document which seems to be a bit out of date). Start –> Programs –> Apache Tomcat 5.0 –> Configure Tomcat –> Java. The “Initial memory pool” corresponds to the JVM -Xms option (~ initial Java heap size), the “Maximum memory pool” corresponds to the JVM -Xmx option (~ maximum Java heap size) and the “Thread stack size” corresponds to the JVM -Xss option. The picture book follows:
You can read more about the non-standard options on java.sun.com.
Configuring Apache James to send email via a smart host
Unfortunately we’re still using Windows on our servers at work, the last application I deployed didn’t have IIS installed on it (yay!), which means no local SMTP service. I needed a local SMTP server so I installed Apache James (I know, it’s probably overkill, but we might use some of it’s other features later). Anyway, buried deep in the SMTP Virtual Server properties applet in IIS is the ability to configure the server to send all email through another server; in IIS parlance this is called a ‘smart host‘. James (love the name btw) doesn’t have a ‘smart host’ option property sheet, but you can get the same functionality by using the <gateway> and <gatewayPort> elements of config.xml (buried in $james_install/apps/james/SAR-INF/config.xml). Read up on the options under RemoteDelivery.
Tomcat 5.0.x bug with dollar sign (and non ASCII characters)
Hoping to bring this to the top of the queue for anyone else that runs into this: if you put configuration information into your Tomcat conf\server.xml file and said configuration contains a dollar sign ‘$’, according to bugzilla [1,2], the dollar sign is interpreted and thus doesn’t show up unless you double up on it. So if you have a environment entry like this:
<Environment name="ftp.password" type="java.lang.String" value="amsdk$k23"/>
you’ll need to change it to this:
<Environment name="ftp.password" type="java.lang.String" value="amsdk$$k23"/>
to make it work with Tomcat 5.0.x (same thing applies to JNDI entries in server.xml). According to this comment, this issue probably will not be fixed because Tomcat 5.0.x is in maintenance and not in active development. Supposedly this behavior will be fixed in version 5.5.7.
An observation: if you’re running a business selling software, make the bug list open the public. There’s nothing better than being able to find a solution without having to spend an hour on the phone with a technical support person who is doing nothing more than searching the private bug list.
Tabbing Through Tabs
My boss asked me today if it was possible to tab through the tabs of a Firefox window (the only drawback to tabbed browsing is that you lose ALT-TAB). Turns out you can use CTRL-PAGEDOWN and CTRL-PAGEUP to tab through multiple tabs in a Firefox window, Which is nice. Some of the other ones I use without even realizing they’ve become part of my vocabulary are:
CTRL-T — open a new tab
CTRL-D — bring focus to the URL
CTRL-R — reload the current page
CTRL-F12 — bring focus to the google toolbar search box
There’s a nice list of keyboard shortcuts here.
I think you should have to memorize at least 12 keyboard shortcuts before you’re even allowed to use a computer. It kills me when someone uses a mouse to do something you can do in less than a second with the keyboard. Maybe someone can write a Pragmatic Keyboarding book or a Keyboarding Hacks books.
ProFTPD and jailing users
I moved all my sites to vpscenter.com a couple months back on Joe’s recommendation and I’m hosting sites for a couple buddies on it right now. One of the problems with opening your server up to your buddies is that, trustworthy as they may be, you don’t want them mucking around with the system by CD’ing up to places they shouldn’t be hanging out. The guys at VPS Center have provided a nice web-based app for the creation of FTP users, but you can’t lock down the users to a specific directory via the web. I did some research today and found out that you can edit the ProFTPD configuration file so that users are “jailed” (a ProFTPD term, not mine) into their websites and can’t muck around outside their own sandbox.
The configuration file lives on my system at /etc/proftpd.conf, open that puppy up and add:
DefaultRoot /usr/hosts/yoursite.com groupname
where /usr/hosts/yoursite.com is the directory containing the site that you want to lock down and groupname is the name of the group (usually the same as the username) that you want to restrict. Restart the server (/etc/rc.d/init.d/proftpd restart) and the next time the user in question logs in, they’ll be automatically redirected to the ‘jailed’ directory and they won’t be be able to browse directories above the one you’ve established as the base.
On a related note, if it seems like your FTP connections take forever to establish, add:
UseReverseDNS off
IdentLookups off
to the ProFTPD configuration file as well (restart required). You’ll see a dramatic decrease in the amount of time it takes to make an FTP connection.
Automated Application Deployment with Tomcat: Part II with Subversion And Ant
Got around to writing the deployment scripts that go with the Subversion work I did last week. As part of the scripts, I also worked on the problem I wrote about a couple weeks ago (ie: needing different log4j properties files per environment). Once you have a source control solution and a master build file, it’s pretty easy. I stripped out the explicit log4j category and appender directives and instead used the Ant <propertyfile> tag to write out the appropriate category / appender directives based on the presence (or lack thereof) of the project.debug property in my build file. When I want to create a build that outputs logging information to the console, this block of code is fired:
<target name="build.resources.debug" if="project.debug">
<propertyfile
file="WEB-INF/classes/log4j.properties">
<entry key="log4j.rootCategory" value="INFO, STDOUT"/>
</propertyfile>
</target>
Otherwise, this block is run:
<target name="build.resources.production" unless="project.debug">
<propertyfile
file="WEB-INF/classes/log4j.properties">
<entry key="log4j.rootCategory" value="ERROR, SMTPAPPENDER"/>
<entry key="log4j.category.com.mycompany" value="ERROR, SMTPAPPENDER"/>
</propertyfile>
</target>
In Eclipse, you can easily create the project.debug property using the Arguments text box provided in Run –> External Tools –> $your ant build –> Main Tab –> Arguments: -Dproject.debug=true. By default then, log4j will only log error conditions and will send them to the SMTP appender.
The second thing I did was to create a deployment Ant script that wrapped my main build.xml. It checks out the source code from Subversion:
<exec executable="svn">
<arg line="co svn://sourcecodeserver/myproject/branches/v1.0 . --username username --password password"/>
</exec>
<ant antfile="build.xml" />
<exec executable="net">
<arg line='stop "Apache Tomcat"'/>
</exec>
<delete dir="${tomcat.install}myapp"/>
<mkdir dir="${tomcat.install}myapp"/>
<unzip src="myapp.war" dest="${tomcat.install}myapp"/>
<exec executable="net">
<arg line='start "Apache Tomcat"'/>
</exec>
We’ll see how well it works once this goes into production. If nothing else, at least I’ll have a couple more hours in my day.