Tuesday, September 30, 2008

Making web servers accepting chunked uploads

I've finally grown sick and tired of the fact that most web servers seems to be quite unable to accept chunked upload. This is a serious problem since Java ME insists on using chunked upload and I'm working on Java ME software ;)

So we know that Apache/PHP seems to work. Tomcat possibly works in some situation. Jetty doesn't work.

I wish to use Groovy on Grails to develop nice services for mobile phones, but since Groovy by default uses Jetty I need to fix Jetty. My test case ws to use a WebAlbum application I found on the net, and modify it to use the Object tags introduced in widger to allow image content uploads from forms.


I check out the svn repository , but Maven won't build :-/ By asking the very nice people at #jetty @irc.codehaus.org I discovered that this was due to a bug in maven 2.0.9, and that 2.0.8 or 2.1 would probably work. Being an adventurous soul I chose Maven 2.1.0-M1, and sure enough now Maven worked. Unvortunately it was not a clean build, since an unit test failed. The kind people at #jetty had a look at the test and indicated that the failure was probably nothing to worry about, so I disabled it (by renaming LifeCycleListenerTest.java to LifeCycleListenerTest.java-not) and sure enough everything else seemed to build and test quite nicely.

I then installed the newly built binaries into grails's lib directory and verified that chunked upload does indeed not work with a freshly built jetty. I then set out to work. Instrumenting the GrailsParameterMap.java file in Grails, indicated that the MultipartHttpServletRequest parser in Spring for some reason return nothing, instead of something when parsing the input. Instrumenting Jetty's HTTP parser indicated that it continued to read HTTP input after Grails had already thrown an null pointer exception for trying t0 work with nonexistant input. I then spent the better part of a day to install an instrumented version of spring (spring is -h-u-g-e-!) and making it work with my already installed grails. Well, after too many hours I got a combination of consistent spring and flow to compile, install and combine with grails. I then set a trap inside spring, to capture the setting of the result of a multipart parse, and discovered that the culprit probably (but so far not for sure) is in Apache's file upload package. When faced with a multipart and chunked input from the web server (Jetty in this case) it returns an empty map of parameters. It turns out that the fix is simple: Replace the commons-fileupload-1.1.1.jar file with commons-fileupload-1.2.1.jar, and things just work. Whee. Now I can get back to my real task, which is to make intersting web based applications for the brand new widger web client ;)