Antenna 1.2.0 with support for Java ME SDK 3.0 released

Yesterday I made the new antenna version with support for Java ME SDK 3.0 available for download. The testers have been positive so far, so it is ready for more general use.

The changes are documented in here. You will also find the matrix with toolkits ad optional libraries there, it seems like Blogger don't really like my html table...

The download is available here.

Java Me toolkit's and support for optional JSR's

As a part of the new Antenna release I have reviewed the toolkits supported and which JSR's they support. The result is the matrix below.

As you see, many JSR's are widely supported, while others are supported only by a few manufacturers. If you need Bluetooth, PDA, Wireless Messaging, SVG or Mobile Media, they are available in most emulators. But if you need Internationalization, or Open GL your choice is more limited.

Many manufacturers have their own additional APIs. Most used is Nokia UI, which is actually also supported by Sony Ericsson.

JSR API Support	Antenna property	Sun WTK 2.5	Java ME SDK 3.0	MOTODEV Studio for Java ME	Nokia S60 3rd Edition SDK	Nokia N97 SDK	Nokia S40 5thed	Sony Ericsson SDK 2.5	Sprint WTK 3.3.2	LG SDK 1.3 for Java ME
CLDC version		1.0,1.1	1.0,1.1	1.1	1.1	1.1	1.0, 1.1	1.0, 1.1	1.0, 1.1	1.0, 1.1
MIDP version		1.0,2.0,2.1	1.0,2.0,2.1	2.1(2.0)	2.0,2.1	2.0,2.1	1.0,2.0,2.1	1.0,2.0,2.1	1.0,2.0,2.1	1.0,2.0,2.1
JSR 75 PDA	optionalpda	x	x	x	x	x	x	x	x	x
JSR 82 Bluetooth	Bluetooth	x	x	x	x	x	x	x	x	x
JSR 135 Mobile Media 1.2	mmapi	x	x	x	x	x	x	x	x	x
JSR 172 Web Services	j2mews	x	x	x	x	x	x	x	x	x
JSR 177 Security and Trust Services	satsa	x	x	x	x	x	x	x	x	x
JSR 179 Location	locationservices	x	x	x	x	x		x	x	x
JSR 180 SIP	sipapi	x	x	x	x			x	x	x
JSR 184 Mobile 3D Graphics	java3d	x	x	x	x	x	x	x	x	x
JSR 120 WMA 1.0	wma	x	x	x	x	x	x	x	x	x
JSR 205 Wireless Messaging 2.0	wma2	x	x	x	x	x	x	x	x	x
JSR 209 AGUI	agui		x
JSR 211 Content Handler	contenthandler	x	x	x			x	x	x	x
JSR 226 Scalable 2D Vector Graphics	s2dvgapi	x	x	x	x	x	x	x	x	x
JSR 229 Payment	papi	x	x					x	x	x
JSR 234 Advanced Multimedia Supplements	ams	x	x	x	x	x	x	x	x	x
JSR 238 Mobile Internationalization	miapi	x	x	x				x	x	x
JSR 239 Java Binding for OpenGL ES	opengl	x	x	x				x		x
JSR 256 Mobile Sensor	mobilesensor		x			x				x
JSR 257 Contactless Communication	rfid									x
JSR 280 XML	xml		x
JSR 300 DRM	drm									x
G24 MOTO2MOTO	G24			x
Motorola APIs	motorola			x
Nokia UI API	nokiaui					x	x	x
eSWT API	eswt					x
IAP Info	iapinfo					x

Antenna: Support for LG SDK 1.3 for Java ME and Nokia N97 SDK

Today I added support for LG SDK 1.3 for Java ME and Nokia N97 SDK to antenna. This is available now in the repository and hopefully soon as a binary download.

Antenna Java ME SDK 3.0 version now available

My patch for antenna is now comitted into the repository and is available for download. Still no build available, you have to get the sourcecode yourself and build it, or send me a mail and I will send you a jar file. Hopefully the pre-built version will be available soon. Documentation is also not updated yet, but you will find some in this blog.

Antenna support for Java ME SDK 3.0

For building our Java ME application we use Antenna. It is a great tool, that makes automated builds possible and also works well as your regular build tool. But it lacks support for Java ME SDK 3.0 and other newer toolkits. Since we need that, I have made a contribution to the Antenna project, added as a patch yesterday.
If you want to try the modified verions, send me a mail and I will send you the jar.

How it works

A key property of your Antenna setup is the wtk.home property. It should point to the base directory of your WTK/Java ME SDK installation. Antenna uses this to detect which toolkit you are using. In the older version toolkits are hardcoded into Antenna sourcecode. In the new version textfiles are used to set up available toolkits and their properties. Antenna goes through a list of toolkits, described in the text file autodetect.txt, and checks if the unique files for that specific toolkit are available. If they are, the right toolkit have been found, and properties are loaded from a properties file.

This makes it easy to add a new toolkit or new properties for the existing one.

Supported toolkits

I have tested the new version with the following toolkits:
- SUN Java ME SDK 3.0
- Sun WTK 2.5.2
- MOTODEV SDK for Java ME v2
- Nokia S60_3rd_FP2_SDK_v1.1 (Symbian)
- SPRINT WTK 332
- Samsung_SDK_11 (autodetected as WTK 2.5.2)
- SonyEricsson WTK 2.5.2
- Nokia S40 5th Edition SDK Feature

Pack 1

I have also made some tests with Mpowerplayer, but the stub for CLDC 1.1 provided with Mpowerplayer seems not to match the specification. If I replace it with the one from Suns WTK 2.5.2 it seems OK. I have not tested Mpowerplayers preverifier, since it is Mac based and I use a Windows PC.

Antenna setup

Antenna properties are described here. They work as before, with the new feature that additional libraries can be defined dynamically, in the toolkit properties file, without any need for changes in Antenna source code. If I set up a property for a toolkit like this:

jsr999=lib/jsr999.jar

I can then turn it on in my ant file by setting the property wtk.jsr199.enabled to true. I can also turn on all defined additional libraries by setting property wtk.all.enabled to true. During development I used this, since Antenna checks that all jar files are available.

Adding a new Toolkit
You can add a new toolkit to antenna just by editing a text file and adding another. Do like this:
First you add a new line to the file autodetext.txt (in the res directory). It should look something like this:

#sony ericsson
sonyericwtk2;lib/semc_ext_jp8.jar;lib/cldcapi11.jar

Lines starting with # are comments. The lines defining toolkits should start with the toolkit property filename and then contain a list of files that are unique to this toolkit. File names should be relative to wtk.home.

Then you add a properties file, in this case sonyericwtk2.properties. It looks something like this:

name="Sony Ericsson WTK2"
include=wtk25
nokiaui=lib/nokiaext.jar
semc=lib/semc_ext_jp8.jar
vodafone=lib/vscl21.jar


The properties you could use are:

• name: a descriptive name for the toolkit. Use this always.
• preverifyversion: 1 means WTK 1 style, support only CLDC 1.0, 2 means WTK 2 style, include Target parameter in preverify command, 3 means WTK 3 style, only add parameter cldc1.0 if cldc 1.0 is used
• emulator: exe or jar file for emulator
• cldc10,cldc11: libraries for cldc versions
• midp10,midp20,midp21: libraries for midp versions
• include: include another toolkit definition. Useful if the toolkit is an add-on to another toolkit, like WTK 2.5 or Java ME SDK 3.0

All other properties will be taken for add-on libraries that will be added to classpath if wtk.[propertiy].enabled is true, or if wtk.all.enabled is true.

Please let me know if you have any problems or improvement suggestions.

Catch System.out from BlackBerry simulator

Just a little trick I just learned. Sometimes I want a quick and easy logging, just to see what's happening in my program. The easiest approach, specially early in the project is simply using System.out. But when you run your program in the emulator you don't see system out anywhere. To see it you need the command line parameter /app-param=JvmDebugFile.

Just run the emulator as follows:

fledge /handheld=9000 "/app-param=JvmDebugFile:bblog.txt"


In the bblog.txt you will get a lot of info, and your System.out output.

Java ME automated build

So far I have built my Java ME application using Sun Java ME SDK 3.0 and before the SDK was available with Sun Wireless Toolkit. But now we need to streamline the process and introduce automated build.

The tool we use for this is ant, with an add-on for Java ME called antenna. Antenna is an open source project, which provides support for operations like obfuscating, updating manifest and JAD files and signing.

Setup
To use antenna you need to setup the antenna tasks, using taskdef, and define some properties:

<taskdef resource="antenna.properties" classpath="${basedir}\tools\antenna.jar" />
<property name="wtk.home" location="C:\WTK2.5.2"/>
<property name="wtk.cldc.version" value="1.1"/>
<property name="wtk.midp.version" value="2.0"/>
<property name="wtk.proguard.home" value="${basedir}\tools\proguard4.3"/>

As you can see, we have a directory called tools, where we keep the antenna jar and also a proguard installation, which we need for obfuscation.
Antenna also needs to know where your wireless toolkit is installed and which versions of CLDC and MIDP you use. That's it!

Package
Antennas main function is packaging your application into jar and jad files, preverified, obfuscated and with correct manifest and jad files. You can do this step-by-step, with different tasks for each step, but you can also use the wtkpackage task to make them all.This is the approach we have taken.

<target name="package" depends="compile">
<mkdir dir="${dist.dir}"/>
<copy file="${basedir}/myapp.jad" overwrite="true" failonerror="true" tofile="${dist.dir}/myapp.jad"/>

<wtkpackage jarfile="${dist.dir}/myapp.jar"
jadfile="${dist.dir}/myapp.jad"
obfuscate="true"
preverify="true">

<fileset dir="${class.dir}"/>
<fileset dir="${res.dir}"/>

<exclude_from_manifest name="myapp-dir"/>
<exclude_from_manifest name="myapp-host"/>

</wtkpackage>
<copy file="${dist.dir}/myapp.jad" overwrite="true" failonerror="true" tofile="${dist.dir}/myappu.jad"/>

</target>

Comments:

1. compilation is separate, our package task is made after the compilation
2. we keep a jad template in the project basedir, which we copy to the dist dir
3. wtkpackage will create the jar file and update the jad file. We also want it to preverify and obfuscate using proguard
   
4. in ${class.dir} we have all classes that should be included
5. all used resources are in ${res.dir}
6. we have two parameters in our JAD template that should be only in the JAD file and not in the Manifest file. This allows for changing them without resigning.
7. as a last step we copy the jad file. This is to have an unsigned copy for use in emulators and phones that don't support the Verisign certificate we use (Nokia S40..)

Signing the application
Antenna also supports signing using the very straightforward wtksign task:

<target name="sign" depends="package">
<wtksign
keystore="${basedir}/../cert/keystore.ks"
jarfile="${dist.dir}/myapp.jar"
jadfile="${dist.dir}/myapp.jad"
storepass="******"
certpass="******"
certalias="myalias"
/>

</target>


This will add signature information to the JAD file using the certificate in the keystore and the JAD and jar files are redy to use.

Going native on BlackBerry

Our application is currently based on Java ME and uses the lcdui to a 100%, even though there are some fixes to make it work better on BlackBerry. Since BlackBerry is getting more and more popular in the market we have decided to make a version that uses the BlackBerry native UI instead of Java ME standard lcdui.

BlackBerry does not allow combining its native UI with lcdui, which means we will have to convert everything, including the midlet. The majority of the code is however in our own classes, since 90% of tha application is Canvas-based. It seems quite possible to make them work in both environments.

Major changes will be:
- our midlet is replaced by a UIApplication
- the Canvas will be replaced by a MainScreen
- instead of the List we use we will have a KeywordFilterField, which will gives the users better functionality.

Our components will be placed in the MainScreen (instead of a Java ME Canvas). To make this work we will use an interface for painting with two different implementations, that encapsulate Java ME Graphics and BlackBerry Graphics. We will also need to handle the different Font classes and lcdui Image versus BlackBerry Bitmap.

Our immediate gains will be:
- scalable fonts, which is a problem for us now, since the Lcdui fonts on BlackBerry Bold are very large
- some UI improvements, like the KeyWordFilterField instead of the List etc
We will also gain access to possiblities like opening an input field on top of ouyr Canvas, with we really need and have a better looking app.

Java ME Search Engine

Finding information on Java ME issues can be time consuming. A lot of information is available on the Internet, and Google finds lots of it. But most of the time you get Java ME information mixed with a lot of Java SE information.

To solve this I have collected a lot of Java ME resources and created a Google customized Search Engine. This is a first try, based on some links I had collected. There is certainly a lot more resources available out there. If you know one, don't hesitate to send me a mail.

Here is the search box:

Java ME Search Engine

Analyzing Java ME memory heap

We have some memory problems with our Java ME application. On some platforms we get out of memory error after working a while. To some extent this is expected, and the error is handled reasonably well, but nether the less we wanted to get a view of how memory is used.

The Java ME SDK 3.0 includes some tools for this. They are not available in the IDE but has to be started from the command line. So you open two command line windows. In the first you start your midlet:

cd C:\Java_ME_platform_SDK_3.0\bin
C:\Java_ME_platform_SDK_3.0\bin>emulator.exe -Xdevice:DefaultCldcPhone1 -Xdebug -Xrunjdwp:transport=dt_socket,suspend=n,server=y,address=51307 -Xdescriptor:[jadfile]

This should start the emulator with your midlet. I choose the DefaultDldcPhone1 which is configured to have 4 MB of heap. You can change that in the device properties if you want to.

In your second command window you start the memory profiler:

cd C:\Java_ME_platform_SDK_3.0\bin
memory-profiler.exe -host 127.0.0.1 -port 51307

This should open up the memory profiler window. You then press Connect to cennect to your emulator instance.This will take a while but then you get a window something like this:

Now you press resume and run your application and try to recreate your memory problems. After that you press Pause in the memory tool. This will cause the tool to get a memory dump from your runing application. You will get a memory map at the top and at the bottom left a list of classes and primitives. You can select a class in the list and see what instances exist. It is however quite difficult to find information this way. Better is to use the statistics button. This will get you a window something like this:


The data is:

• class name or primitive. a [ means 'array of'
• object number (or number of objects rather)
• size
• average size
• % of heap
• live %
• old generation %

Since you can not search in this table I copy it al and paste into a spereadsheet, where I can find my classes easier.

Java ME SDK and SSL

I have been struggling a few days with getting Https communication to work in Java ME SDK 3.0, but with little success so far. To be able to create a Https connection I need to get the SDK to accept the (self-signed) certificate on my test server. The SDK does not however use the Java SE keystore but has its own (actually it has a lot of keystore's) with its own tool to update it. So far I have not managed to get my certificate into the right keystore. Tried asking on Sun's forum, but they don't seem to be interested in the problem at all.

So instead I used Nokia's S60 emulator, which gives me the option to accept the server certificate even though it can not be verified. Perhaps I should give up on the SDK for now, specially since the support and documentation is not really good enough.

Programming for BlackBerry with Java ME

Research in Motion BlackBerry phones support the Java ME standard, so for a Java ME developer is easy to get started on BlackBerry. But a few things do not work as expected...

Menu
The BlackBerry has a very consistent user interface, with a menu key and a back key. These keys can not be catched by your Java ME canvas, pressing them will not call the keyPressed() method as you might have expected. Instead the menu key will always open the menu. If you have not added Java ME commands to the Canvas, the BlackBerry will create a menu itself, with 'Close' as the only alternative. The back key will trigger the command with the command type EXIT if there is one and do nothing is there is no such command.

This basically means that you should always use Java ME Commands in yor BlackBerry application. There is no other way within the Java ME API to get the BB keys to work as expected.

Other keys
There are no Nokia-style softkeys on a BlackBerry keyboard. But there is a trackball (trackwheel on older models) and a qwerty keyboard.

The trackwheel sends Canvas UP, DOWN, RIGHT and LEFT codes and trackball press can also be handled and expected. Your application should probably use these keys a lot. Key repeat is however not supported, so if the user scrolls the trackball you will get a series of keyPressed events and no keyRepeated.

The qwerty keys will call keyPressed with codes as expected, you can use getKeyName. Just remember that some models have a smaller keyboard, with two characters per key. You will always get the first character.

The return, backspace and space keys will also deliver key events as expected. The shift and alt keys will however not. Instead they affect the keycode delivered to the application. That is ok for the qwerty keys, but also it means that you can not tell if the user is holding down shift and scrolling or pressing the trackball.

Fullscreen mode
You Java ME Canvas will always cover the full screen. The call to setFullScreenMode has no effect. This also means that the sizeChanged method will never be called. So you should make sure that your application does not depend on it beging called.

The conclusion is that the BlackBerry Java ME implementation is a good one, and can be very well be used as a base for your implementation on BlackBerry. But if your midlet is based on the Canvas class and developed for Nokia-style phones you will probably need to make some changes, mainly with menues, softkeys etc. You can them detect at runtime if the midlet is on a BlackBerry or not and adjust to that.

Java ME on BlackBerry

One of the platforms you can support with Java ME is BlackBerry. BlackBerry devices are a bit different from your ordinary cellphone. They are very good at email and includes a handful of applications.

Most of the applications for BlackBerry use the native Java API, but the device also supports Java ME midlet applications. But there are a few points to note...

Create COD files
BlackBerry supports standard jar/jad files to some extent, and installing your application entering a jad URL in the phones browser might work very well. But my experience is that sometimes it doesn't. Converting the jad/jar files to BlackBerry cod file format on your PC might be a better idea.

For this you need BalckBerry Java development tools. I use BlackBerry JDE 4.6.0, but there are new versions. You find them here.

From the command line you run something like:

"C:\Program Files\Research In Motion\BlackBerry JDE Component Package 4.6.0\bin\rapc" import="C:\Program Files\Research In Motion\BlackBerry JDE Component Package 4.6.0\lib\net_rim_api.jar" codename=XXX -midlet jad=yyy.jad yyy.jar

This will take your yyy jar and jad files and create XXX.cod. It will also modify your jad file and add a lot of BlackBerry-specific lines to it. This makes the JAD file much larger, so will probably not want to use it for non-BlackBerry devices, even if the original data is still there (some devices have restrictions on jad file size).

If your midlet is large, it will be split into several cod files. They will all be in the yyy.cod, which might by a zip archive. So you need to check if it is (you can also check the modified jad file to see if there are several cod files in it). If it is you should unpack it and put the individual cod files on your web server together with the modified jad file. Make sure that the cod mimetype is configured on your web server, and your cod file is ready to install.

Portable Java ME development

The last six months or so I have been busy with developing a new application in Java ME environment. I have done some work in this envirnment before, but even though the platform has not changed much, my impression is that the quality of the implementations have improved a lot.

The goal was to support a wide selection of mobile phones:
- Nokia, both System 40 phones and Symbian System 60
- Research in Motion BlackBerry, at least the latest models like Bold and storm
- Sony Ericsson devices, where we started with Symbian UIQ models and now target mostly Xperia X1, based on windows mobile
- Samsung
- LG
- Motorola
etc...

Main focus is on more powerful, smartphone segment, but the application works well also on smaller phones.

Java ME was not the only possible choice. There is always the possibility to use native tools, for Symbian and BlackBerry at least. Our choice was to start developing on Java ME and see if it works and later add Symbian and/or BlackBerry specific code if needed. So far we have not needed it. And what surprises me is that it now works very well and very portable. When we try a new device it almost always works right away. We might need to add some code to make better use of the keyboard or make it look better, but generally it works well.

It is actually more of a problem to make it work in emulator, which surprises me. A special problem is with hasPointerEvents() method, which is supposed to tell us if the device has a touch screen or not. Since the application changes its user interface slightly depending on if there is a touch screen available or not this is a very important method for us. And on the phones it always works, but in the emulator it does not.

Another problem with the emulators is to find a good-looking emulator for demonstration purposes. There are a lot of good devices out ther, like Sony Ericsson Xperia X1 and Nokia S60 phones, but I have not managed to find good emulators for them. For the X1 I have so far not found a Java ME emulator at al, and for S60 phones there is one, but it does not look good enough for demonstration puposes. Guess this is because the development of new devices is so fast that manufacturers don't have the time to fix emulators.