Tuesday, September 30, 2008

Making web servers accepting chunked uploads

I've finally grown sick and tired of the fact that most web servers seems to be quite unable to accept chunked upload. This is a serious problem since Java ME insists on using chunked upload and I'm working on Java ME software ;)

So we know that Apache/PHP seems to work. Tomcat possibly works in some situation. Jetty doesn't work.

I wish to use Groovy on Grails to develop nice services for mobile phones, but since Groovy by default uses Jetty I need to fix Jetty. My test case ws to use a WebAlbum application I found on the net, and modify it to use the Object tags introduced in widger to allow image content uploads from forms.


I check out the svn repository , but Maven won't build :-/ By asking the very nice people at #jetty @irc.codehaus.org I discovered that this was due to a bug in maven 2.0.9, and that 2.0.8 or 2.1 would probably work. Being an adventurous soul I chose Maven 2.1.0-M1, and sure enough now Maven worked. Unvortunately it was not a clean build, since an unit test failed. The kind people at #jetty had a look at the test and indicated that the failure was probably nothing to worry about, so I disabled it (by renaming LifeCycleListenerTest.java to LifeCycleListenerTest.java-not) and sure enough everything else seemed to build and test quite nicely.

I then installed the newly built binaries into grails's lib directory and verified that chunked upload does indeed not work with a freshly built jetty. I then set out to work. Instrumenting the GrailsParameterMap.java file in Grails, indicated that the MultipartHttpServletRequest parser in Spring for some reason return nothing, instead of something when parsing the input. Instrumenting Jetty's HTTP parser indicated that it continued to read HTTP input after Grails had already thrown an null pointer exception for trying t0 work with nonexistant input. I then spent the better part of a day to install an instrumented version of spring (spring is -h-u-g-e-!) and making it work with my already installed grails. Well, after too many hours I got a combination of consistent spring and flow to compile, install and combine with grails. I then set a trap inside spring, to capture the setting of the result of a multipart parse, and discovered that the culprit probably (but so far not for sure) is in Apache's file upload package. When faced with a multipart and chunked input from the web server (Jetty in this case) it returns an empty map of parameters. It turns out that the fix is simple: Replace the commons-fileupload-1.1.1.jar file with commons-fileupload-1.2.1.jar, and things just work. Whee. Now I can get back to my real task, which is to make intersting web based applications for the brand new widger web client ;)

Tuesday, June 24, 2008

Opening ceremony

This is a blog from the Open Nordic conference in Skien Norway 19-20 june 2008.



It is very very painful to listen to public officials who cannot speak English trying to do so from a podium. Nuff said.

Implementing ODF in Belgium (Bart Hanssens)

This is a blog from the Open Nordic conference in Skien Norway 19-20 june 2008.



Belgium. 10.6 million people, dense population, three languages (Dutch, French, German). Very little layering of public services. All the various parts of the government has a high degree of autonomy.

odf-converter.sourceforge.net: A converter using .Net 2.0 and 3 separate .MSIs. Can convert stuff here and there. Integration with MS-office OK, pretty good .doc conversion, but not quite there yet. A bunch of rough edges but mostly useful.

He mentioned a lot of other stuff as well. It seems to be sort of useful, but still a whole bunch of rough edges. If you want to ODF is probably useful for prime time today.

His presentation was not the most inspiring one, so I spent spent most of my available attention on reading mail and news ;-)

The Neo FreRunner (alias Neo1973 GTA 02)

This is a blog from the Open Nordic conference in Skien Norway 19-20 june 2008.



General stuff

It is a way nice hanset, uses standard USB cables for charging. Wifi, will have two 3 axis axelerometers. Can not feel rotation along top/down axis. Will hav GSM 2.5, no edge no umts, no 3G. About 100kbps. Don´t know why these limitations will be in the handset.

Backing from the supplier

FIC - First Internation Computer. The OpenMoko folks tried to approach Motorola first but they were not interested. FIC works with the community. Intend to get higher profits by selling direct to consumer. FIC produce about 20% of all laptops in the world. Not under their own name (Toshiba is one of the names they sell by).

Stuff

On the wiki there is a ton of information, including information on how to take the phone apart and put it together ;)


On a pc, there are various kinds of firmware drivers that are not available. On the freerunner, one uses Hayes AT commands to talk to the GSM chip, and NMEA with the GPS chip. For everything else there is a driver available as open source code. There is a free wifi driver available. The sourcecode was made available through the community (the community found a supplier and a(n open source) deal was struck).

There are people hired to write contacts, calendar, application managers, and a lot of other boring stuff. Basic smartphone applications will be available out of the box. Everything else (including updates to the basic applications) will be available as Debian packages.

How will networks exclude phones to run on their networks (Terrence Barr asks). Ole didn´t have a good answer for that, but didn´t think it would be a problem either.

Qtopia was ported on the Neo1973, but will run without changes on the FreeRunner. Seems like Qt will be used for most of the OpenMoko stuff. Can run GTK as well.

Android: rumors. Virtual machine with hooks into the operating system, will be open source, will be free. Facts. Not available yet. If rumors are true, then OpenMoko will run Android.

Developers will run as root initially. Later, sandboxing will probably be introduced.

Some ideas he wish to have implemented:
  1. Location based calendar. Wish to have a calendar that at a certain place triggers an alarm that will remind him to buy toohpaste when he is close to the shop. Perhaps including opening hours.
  2. Get off at a bus stop: Point to a map where you want to go off, but get a reminder a bit before.
  3. GPS friends. Choose who your friends are, get an alarm when they are close.
  4. Find the closest wifi. Find the closest open access point. Put it on a map, give some indication when it was located.
  5. Find the nearest whatever, get the gas prices for the closest gas station.
  6. Detect congestion, speed limit, one way street. Use openstreetmap.org. Guess what is one way streets, etc.
  7. Profiles (automatic or semi-automatic profile detection)
  8. Firewalling. timebased, locationbased, whitelisting. We will probably see spam on phones in a few years, so we need something to stop that.
  9. Dasher input: There is a motion sensor, can be used to input text, tilting in the right direction. Really, really cool (I agree with that statement, if the demo was realistic ;)
  10. Cheap data transfer: Different subscbtions give different ideas. Fre voice,d ata sms, call etc. Advice to telcos: Go flat rate.
Some ideas from people in the room
  • Location based profiles (when at work, use the work profile, when at home use the home profile).
  • Dating service :)
To develop, you go to the wiki, install something (the whole environment), and within a few minutes you will be up and running.

Software patents:
Some ideas are patented. Patents are harmful mostly to small developers. Big ones can exchange patents. Suggests joining www.eff.org, www.ffii.org, www.fsfs.org

For the end user:

It will be like a distribution. Can subscribe to various types of distributions (stable, expeimental, etc).

How much and when:
  • 12 juli 2007 : First wave.
  • Summer 2008 ( production started). USD 400
  • wiki.openmoko.org/wiki/GroupSales -> Lower price, but you have to buy ten at a time (USD 370).
Comments

Terrence (Barr) thinks that at once you have an open phone, the carriers will have to use a lot more resources to become a good pipe, because locking customers into a particular (and sometimes quit bad) deck of applications will no lenger be a viable option.

Espen Eskedal Trolltech XUI future GUI on mobile phones.

This is a blog from the Open Nordic conference in Skien Norway 19-20 june 2008.

Espen has been working Qt for Nokia series 60. Yesterday he became a Nokia employee. Has been working based in Berlin. Will talk about what the typical probmes are when making applications for mobile phones.
  • Tools: It is in general hard to get the toolchain up and working. Process tools, memory debuggers. ON the desktop (Desktop bad->annoying) (Mobile nonexisting->annoying). Best of the pack today is probably Microsoft CE.
  • APIs: In general too limited or too difficult.
  • Platforms: Very bad fragmentation. You need a lot of binaries, a lot of toolchains.
Mobile gui problems:
  • Resolutions are different.
  • Orientations are different. Some devices even have animations when orientations shift.
  • Custom styles: Styles can change at runtime. Also you may wish th have applications that looks different from others.
  • Input devices: Huuuge variation. Softkeys, touch screens, multitouch (iphone). You need to take into consideration that the user might have a bunch of different devices.
His thesis:
  • QT solves all of these things, at least part of the way.

Showing demo:
Showing qt running on windows mobile. Different parts of Qt running on an HTC thingy under windows mobile. It´s pretty nice :) Most people see no difference between a Qt application and a really native application. Same application running on OpenMoko (XXX Get link to this demo). Same code, but recompiled and deployed. Different resolutions, but looks nice. Uses layouts to handle tings like orientation changes. Senses dpi resolutions etc.

Qt have both GPL2 and GPL3 and commercial licenses. He is now working on Qt for series 60.

His advice to operators: Flat rate. Available interfaces.

Qt 4.4 has webkit in it, has scriptsupport. There is also a crosspatform multimedia toolkit.

Demo of various stuff to build an application.
He explains the signals/slots thingy that Qt is based on.
He demonstrates that Qt has a stylesheet (qss) file that uses the cascading stylesheet syntax, but does layouts on the qt thingy. Doesn´t need recompliation, but is read at runtime. They have a qstyle class than be hacked

his email address espenr (at) trolltech.com epen.riskedal (at) nokia.com

Goal to make phone input/output as well as video input/output. It should be possible to make a video editor . They only support playback for sound now. There is some kind of support now, they wish to do more, many people wish to go in that direction right now, but it´s not there yet.

On licensing: It´s hard. Terrence Barr says that Sun has the same type of issues that Trolltech has. If you are doing something GPL-like, you may have to talk to some GPL-aware lawyers, because there may be differences in the different companies interpretations of the GPL terms.

OSS work at NTNU

This is a blog from the Open Nordic conference in Skien Norway 19-20 june 2008.


Letizia Jaccieri and Reidar Conradi

History about OSS software work at NTNU:

Looks at various roles (provider, participant, integrator, user, customer). Mentions some skills: Technical comercial, organizational, creative.
More history. Reuse is now very different than before.
They have done s bunch of surveys on COTS. A patte

Forthcoming article IEEE software 2008 (ten facts about open source development)
Finding: In reviews of sprints in SCRUM the customer is very very seldom present. That is probably a mistake.

Metod for learing about OTS:
1. Ask someone on the office. 1b. search the net, download 3-6 canidates. No formal selection method. Quality is not a problem (?? why). Problems: Effort estinatiln, integration, debugging, customer non-participation, cliencing, comany policies, how to classify components. Ontologies are not stable at all.

It is methododically difficult to study open source projects. How do you analyze a project with 300 developers in 30 countries?

ITEA COASI PROJECT

Industry and academia. Shared internal development is a goal.
About 50% reuse and integrate OSS components.

Main findings:

Motivations for adoptions of OSS:
  • Simplify developer´s jobs: High availability of information, pcomponents and their source code. Simple procurement.
  • Reusable software assets: Increase productivity increase quality
  • Reduced costs: Reduced development effort, no license fees.
Selection process:
  • Evaluating util something "good enough" is found
  • Tested further throug test integration and prototyping
  • informal knowledge driven rpocess based on previous experience, recommendations, informal searches, surveilance of the OSS community.

Eskil Sundt: Case open standards (ARM)
  • A masters project, single person one year. Development of a multip.latform graphics application Used ARM platform for embeded graphics.
  • Used OpenVG (Open vector graphics for embeded systems. Target: Map intraces, scalable userinterfaces, e-book reaers,g ames and SVG viewers). SVG (We all know this ;) He was looking at an SVG viewer. OpenVG is built on the SVG model, but there are differences: SVG format, openvg is an API.
  • He wrote a 20KLOC program calledSVG Tiny.
  • Results: It workt nicely ;))

Letizia: Open soure art project.

Cases (all on sourceforge)
  • Open Digital Canvas project
  • Soonic Onyx Project (mobile phones)
  • The Flyndre project
An art competition: http://mediawiki.idt.ntnu.no/wiki/sart

Resources:
  • http://www.idi.ntnu.no/grupper/su
  • http://research.idi.ntnu.no/oss
  • http://www.itea-cosi.org/
What Letizia wish for: She wish to have a master project about open source, perhaps a national research scool on open source software.

Terrence Barr: Competitive landscape in mobile:

This is a blog from the Open Nordic conference in Skien Norway 19-20 june 2008.


Inovator´s dilemma:
  • Carrier networks v.s. voip. Is ubiqutious, open standards, no control point, no barriers to entry, cpractically free to use. Even if not perfect it marganiilzes tradition voice model
  • Walled gardens vs. open internet: Open technologies and economies of scale outside the wall. Harnessing these revenue opportunities requires adopting the rules outside the wall. Scarcity value begins to evaporate (AOL v.s. internet).
Openness turning the model on its head.
  • Openness increases market opportunity, but decrease scarcity value.

The wireless industry at a crossroads:
  • Today: predominatly proprietary and closed
  • Huge investmens and rapidly commoditizing services
  • Massive friction in the ecosystems. Content creators must deal with each island separately making service creation very inefficient.
  • Existing revenue models not suited for new services. Monthly subscribtion fee is not granular enough. Customers are wary of random ads but also of data mining.
  • Customer satisfaction low and turnover high
  • Bottom line: Industry needs new revenue models and needs to renvient itself through innovation and openness.
Comment: Many carriers around the world is using security models to clob developers into submission. Carriers are saying that the developer requires a cut from the usage of every application. This is because there are features on the phones that requires signing of applications.

Open source and open standars: Problems or solution
  • Technology diffusion is a fact of capitalist economies.
  • Must face this evolution, since it is inevitable. Nust reinvent find new sources of revenue fbefore margins disappear. Must open up and deal with loss of control. Must deal with technology evilution
  • Open soure and open standards are simply accelerating this rnd

What is "open" really about?
  • Empowerment
  • Choice
  • Local decision making
  • Control of your destiny
  • People no longer accept to be tied into a proprietary model. People are waking up to the fact that tis is not a good ple to be in.
Contrasting approaches:
hierarchical v.s. flatness. Resisting change v.s. dealing with change. Scarcity vs. value. Us and them v.s. we

Do customers really care?
  • Governenments (Brazil and EU does).
  • Most -people- probably doesn´t care. However, the DRM model example shows that people -do- care in some cases (DRM doesn´give any value to users, users kept circumventing until the industry just gave up). So, consumers -do- care at the end of the day because it gives them choice.
Lead users and open source
  • Lead users crave openness. They represent cutting edge of the user base. Notcontetn with stqus quo. Pus the limits and aren ´t afraid to go through some pain. Primary adopters of open soruce and open standars tchnologies.
  • lead users create market shifts: If they succeed the word spredáds, the herd of customers following, creating a market shift. If you havent anticipated it you´ll be left behind fast.
The wireless industry is different
  • Mobile is the new desktop but. Complex and rapidly evolving technolgy stack. Many players. Security paramount. Enormous scale. Consumer focus (price, functionality) unique deployment model Entrenched distribution and business models. On the desktop everyone have to be a semi expert, but on phones that is just not the case.
  • Mismatches. Open source properties v.s. wireless requirements and established models.


Inhibitors
  • Client-side technology stack. You can´t really build an open blown phone on open source, yet. Bazaar causes fragmentation, at least initially. P nd integration is a headache.
  • Innovation and deployment model: Open source works best on continous beta style. Mobile idustry runs on complete functionality, solid testing and long term stability.
  • Choice: Can there be too much?
  • Open access and data connectivity. Requires open and reasonably fast data pipe. Mobile access predominatnly limited, controlled, slow and expensive. Always on cheal fast mobile eonternet still years avways for most.
My own comment: OpenMoko is really, really important for the near term future.

Terrence mentions a rumour: Venture captialists in the bay area says: If your business model requires a business deal requires on a carrier, the VC won´t fund you.


The shift that is occuring:
  • Customers are increasingly driving technolgy adoptio: Bacic needs are met. Choice important. Empowered by standards etc.
  • Customers vote with their feet.
Positioning for opportunities: Where do you want to be?
You can
  • control Not successful in the long run.
  • lead You need to be the leader in something.
  • follow No value added.

The opportunities:
  • Expect ideas from unexpeted places.
  • Open yoursel to outside innovation
  • Redefine your value-add. "Which business are you -really- in
  • Engage your new market research an customer outreach departmet." The community is your best friend. Lead users an untapped source of value
  • Monetize different: Commodities are not a bad thing. Many small profits are also very good. Selling intangibles are cool. Build side-channels for monetization .
  • Money follow trust ;)


Conclusions
  • Open soure her to stay Customers won´t have anything else
  • Wireless industry being hitt by perferct storm. Commoditzation, infrastructure, shifting expectation.
  • etc.