The problem with CustomItem on some SonyEricsson phones

[NewAgeMobile]

There are several places on this forum where people have posted about their SonyEricsson not displaying the coloured text (StringColourItem) and bitmap fonts (BitmapFontCanvas). I am one of them (with a K800i).
Excuse the long post explaining what is behind this problem.

Over the last 3 days, I have spent some time looking into this problem and can now suggest a workaround, while not perfect is lots better. It is quite simple too.

So, What seems to be causing this problem?
Well.. ..With Forms (and I am not all that experienced in using them since I went the way of a custom UI on a canvas with my apps), the SE has a bit of a quirk to it. One that is not present on the Sony SDK emulators.

Normally, what you would expect when running some process is that the device would be in a state of 'lock', or completely unresponsive to user interaction, bar those that will interrupt the KVM (such as the method for forcing program closure). This is how it is with the WTK and SE emulators. The SE however is not really like this, for some strange reason they decided that the stuff that is behind their form implementation will still function. It would appear that you can still scroll the form and interact with the items on it. Items can be updated, and their new contents are shown. Items can be added and will get displayed on the form if say, you added one, then did some un-threaded processing and added another. With this last point, we are getting close to our problem.

Here is an example description of a MIDlet to demonstrate this quirk:

It consists of a Form, StringItem and ChoiceGroup with 3 check boxes.
There are 2 commands, one to 'exit', and one to 'go'.
Initially, the StringItem contains "Ready"

When 'go' is used, we loop from 1 to 10 and for each cycle of that loop:
We set the StringItem to "Count: "+loop cycle value.
We add a TextField containing "TextField "+loop cycle value.
We wait for 3 seconds.

With a 'Normal' phone, when 'go' is used, the phone is in a state of apparent 'lock' for about 30 seconds.
After this, the StringItem contains "Count: 10" and the 10 TextFields are displayed.

With the Sony, we see "Count: 1" ... "Count: 10".
We can change the check boxes.
We see The TextFields appear and can even edit them as they do!

So what has this got to do with the CustomItem problem?

We modify our MIDlet.
This time we append a CustomItem that displays our text instead.

Now what does the SE do?

The same thing DictionaryForMIDs does. Some items are painted, some not, several are the wrong size.

We have broken the Sony's background form processing; it would appear that when it was setting up one CustomItem on the form, another came along and it forgot about the first or something. This first item was (I presume) incompletely set-up on the form and now never gets things like its paint method called, even if you force it to request a repaint.

DictionaryForMIDs can execute the translation in the background, on a thread. Does this help? Yes. Things are fixed again for our test, but not in DictionaryForMIDs if we execute the translation in the background.

We have a lot of simulated overhead in our item appending loop, and very little overhead in our CustomItem compared to, say, BitmapFontCanvas. So its down to timing, I guess.

A Thread.sleep(long delay) will fix things in DictionaryForMIDs. I seem to be getting good success with about 300ms on my K800i and this is our simplest answer.

The delay causes some other problems:

• is it enough for other SE phones?
• is it enough for other dictionaries? (my test one is the English-Chinese one btw)

So this needs testing on other SonyEricsson phones.

Another problem also comes to light. DictionaryForMIDs needs to not call MainForm.refreshAllTranslationResults() from within MainForm.updateMainFormItemsObj() after accept is chosen on the settings form. The new delay on the inserting of items is going to cause a long delay as they are re-inserted. During that delay, the settings form can be 'toyed' with while the SE user waits

EDIT: I forgot; it appears that hiding the form is of no help.

[NewAgeMobile]

Changes to MainForm.

New method for our sleep after inserting a result

Code Select Expand


    public void applySonyEricssonWorkaroundSleep() {
        if (sonyEricssonWorkaroundRequired) {
            long t = System.currentTimeMillis();
            synchronized (this) {
                try {
                    Thread.sleep(300L);
                } catch (Exception e) {
                    Util.getUtil().log("sonyEricssonWorkaround: " + e.toString() + " / " + e.getMessage());
                }
            }
        }
    }


Called from addTranslationItem(Item item)

Code Select Expand


     void addTranslationItem(Item item) {
        ++indexOfLastTranslationItem;
        insert(indexOfLastTranslationItem, item);
        /*
        This SHOULD fix problems with SonyEricsson adding of CustomItems to the form
        It MAY fix problems that some have reported with normal Items too
        (testing is needed)
         */
        applySonyEricssonWorkaroundSleep();
    }


Force execution in background in translateWord(....)

Code Select Expand


    public void translateWord(String word,
            boolean executeInBackground)
            throws DictionaryException {
        if (DictionarySettings.isDictionaryAvailable()) {
            // for JSR75 support this needs to be done in a separate thread
            if (DictionarySettings.isUseFileAccessJSR75()) {
                executeInBackground = true;
            } else
            // force execution in background for SonyEricsson
            if (sonyEricssonWorkaroundRequired) {
                executeInBackground = true;
            }

            ....


(while I'm at it, a wee bug fix to clear the last search results status in the subroutine that is just above that last snippet)

Code Select Expand


    public void translateToBeTranslatedWordTextField(boolean executeInBackground)
            throws DictionaryException {
        removeStartupDisplay();
        //clear last translation results status
        translationResultStatus.setLabel("");
        translateWord(toBeTranslatedWordTextField.getString(),
                executeInBackground);
    }


Reposition the if statement in updateMainFormItemsObj() so that we do not get left waiting on the settings form

Code Select Expand


        ....

        if (!sonyEricssonWorkaroundRequired) {
            // redisplay translation results
            refreshAllTranslationResults();

            //Causes an error on SonyEricsson devices when called before the midlet is started
            display.setCurrentItem(toBeTranslatedWordTextField);
        }
    }


[NewAgeMobile]

We can still get caught out with some incorrectly sized CustomItems on the SE; sometimes they will be set to the phone's screen height, so we need to call invalidate() in our items if sizeChanged is called with a height that is not what our Item is calculated to be during the formatting of the contents.

In StringColourItem

Code Select Expand


    protected void sizeChanged(int w, int h) {
        if (w != width || height != h) {
            setWidth(w);
            if (h != height) {
                invalidate();
            }
        }
    }


*This should be ok, but I think I should point out that the WTK docs do mention that invalidate() can result in a call to sizeChanged(). However I have not experienced a looping effect where invalidate() has not fixed the height and sizeChanged has been re-called.

In BitmapFontCanvas
This is a little more complicated (and I did not study the class much, so what I have done here to allow re-formatting if the width changes (usually when a scrollbar is added) may not be optimal for it.

Code Select Expand


    public BitmapFontCanvas(StringColourItemText input, String fontSize, int maxWidthPixels, boolean colouredMode) {
        super(null);
        this.stringItem = input;
        this.bitmapFontSize = fontSize;
        this.colouredMode = colouredMode;
        setWidth(maxWidthPixels);
    }

    private void setWidth(int newWidth) {
        if (maxWidthPixels != newWidth) {
            try {
                maxWidthPixels = newWidth;
                String fontDir;
                fontDir = "/dictionary/fonts/" + bitmapFontSize + "/";
                font = BitmapFont.getInstance(fontDir, fontDir + "font.bmf");
                font.loadFont();
                font.loadChars();
                viewer = font.getViewer(stringItem, maxWidthPixels, colouredMode);
                lineHeightPixels = font.getLineHeightPixels();
                totalHeightPixels = viewer.getLinesPainted() * lineHeightPixels;
            } catch (Exception e) {
            }
            repaint();
        }
    }
    protected void sizeChanged(int w, int h) {
        if (w != maxWidthPixels || h != totalHeightPixels) {
            setWidth(w);
            if (h != totalHeightPixels) {
                invalidate();
            }
        }
    }


[NewAgeMobile]

The test MIDlet

Here the idea evolved a little.
Instead of needing a sleep between appends, we append our items in a 'lightweight' mode.
Then when we have done them all, we activate their simulated heavy overhead paint and formatting tasks
(their last formatting request invoked by sizeChanged is queued)

Code Select Expand


package TestForm;

import javax.microedition.midlet.*;
import javax.microedition.lcdui.*;

/**
* @author Tom Rowland
* a workaround for the SonyEricsson CustomItem on forms bug
*
* we append several CustomItems to our form that will do little
* until we 'activate' them.
* Once activated, wait(delay) is used to simulate overhead in their
* paint and reformatting methods
*
*/
public class TestMIDlet extends MIDlet implements CommandListener {

    static final int maxHits = 60;
    //
    private static final Command exitCommand = new Command("Exit", Command.EXIT, 0);
    private static final Command goCommand = new Command("GO", Command.OK, 0);
    private Form form;
    private StringItem stringItem;
    private ChoiceGroup checkBoxes;
    private String[] choices = {"Choice 1", "Choice 2", "Choice 3"};

    public TestMIDlet() {
        super();
        form = new Form("Test");
        form.addCommand(exitCommand);
        form.addCommand(goCommand);
        form.setCommandListener(this);
        initForm();
    }

    public void commandAction(Command command, Displayable displayable) {

        if (command == exitCommand) {
            notifyDestroyed();
        } else {
            doHits();
        }

    }

    void initForm() {
        form.deleteAll();
        stringItem = new StringItem(null, "Ready");
        form.append(stringItem);
        checkBoxes = new ChoiceGroup(null, ChoiceGroup.MULTIPLE, choices, null);
        form.append(checkBoxes);
    }

    void doHits() {
        initForm();
        //it appears that we MUST append CustomItems in a thread on
        //the SonyEricsson
        Thread t = new Thread() {

            public void run() {
                for (int count = 1; count <= maxHits; ++count) {
                    stringItem.setText("Count: " + Integer.toString(count));
                    synchronized (this) {
                        try {
                            form.append(new MyCustomItem("TextField " + Integer.toString(count), form.getWidth()));
                        //wait(3000L); no overhead between appends
                        } catch (Exception e) {
                        }
                    }
                }
                activateHits();
            }
        };
        t.start();
    }
    //after all of our CustomItems have been added,
    //turn on their simulated slow formatting and paint
    //routines

    private void activateHits() {
        //
        for (int i = 0; i < form.size(); ++i) {
            MyCustomItem item;
            if (form.get(i) instanceof MyCustomItem) {
                item = (MyCustomItem) form.get(i);
                item.activate();
            }
        }
    }

    public void startApp() {
        if (Display.getDisplay(this).getCurrent() != form) {
            Display.getDisplay(this).setCurrent(form);
        }
    }

    protected void pauseApp() {
    }

    protected void destroyApp(boolean unconditional) {
    }

    //a lightweight that turns heavyweight
    //a lightweight during the append to our form
    //a heavyweight after 'activation' (using waits^^)
    class MyCustomItem extends CustomItem {

        String label;
        int width, height, paints, formats;
        boolean needsFormatting, lightweightMode;

        MyCustomItem(String label, int screenWidth) {
            super(null);
            this.label = label;
            //lightweight mode
            lightweightMode = true;
            //set up our size
            reformat(screenWidth);
        }

        //called when 1st shown
        //called when a scrollbar is added too
        //ques a format if not 'lightweightMode' for our SE
        protected void sizeChanged(int w, int h) {
            if (w != width || h != height) {
                if (lightweightMode) {
                    //remember new width and set-up to
                    //call our reformat routine in the next paint
                    needsFormatting = true;
                    width = w;
                    return;
                }
                if (w != width) {
                    reformat(w);
                }
                /*
                It should be noted that invalidate can result
                in another call to sizeChanged according to the
                J2ME docs.
                We could have a problem if for some unknown reason
                the KVM continually refuses to recognize our desired
                contents height, but it seems to be ok.
                 */
                if (h != height) {
                    invalidate();
                }
            }
        }

        //Simulates a heavy overhead reformat
        private void reformat(int width) {
            ++formats;
            this.width = width;
            height = Font.getDefaultFont().getHeight() << 1;
            synchronized (this) {
                try {
                    //simulated processing is not too
                    //heavy for this test
                    wait(250L);
                } catch (Exception e) {
                }
            }
            needsFormatting = false;
            repaint();
        }

        protected int getMinContentWidth() {
            return width;
        }

        protected int getMinContentHeight() {
            return height;
        }

        protected int getPrefContentWidth(int height) {
            return this.width;
        }

        protected int getPrefContentHeight(int width) {
            return this.height;
        }

        //until we 'activate' our item, the 'heavy' formatting
        //and paint tasks are lightweight
        public void activate() {
            lightweightMode = false;
            repaint();
        }

        //Simulates a (very) heavy paint task
        //but ony draw the background until we are 'lightweightMode'
        //for the SonyEricsson. ie. after all our items have been
        //appended/insertes onto the form.
        protected void paint(Graphics g, int w, int h) {
            //lol, really we cannot do ANYTHING in paint
            //when we append many items quickly
            //well, we may get away with a 1 line box!
            //but often not 2 lines!!
            if (lightweightMode) {
                return;
            }
            if (needsFormatting) {
                reformat(width);
            }
            ++paints;
            synchronized (this) {
                try {
                    //simulated processing is not too
                    //heavy for this test
                    wait(50L);
                } catch (Exception e) {
                }
            }
            g.setColor(0x008080);
            g.fillRect(0, 0, width, height);
            g.setColor(0x0);
            g.setFont(Font.getDefaultFont());
            g.drawString(label, 0, 0, Graphics.TOP | Graphics.LEFT);
            g.drawString("Paints: " + Integer.toString(paints) +
                    "Formats: " + Integer.toString(formats),
                    0, Font.getDefaultFont().getHeight(), Graphics.TOP | Graphics.LEFT);
        }
    }
}


Sometimes while our items are being 'activated' we will have to scroll them off screen and back on so that they refresh.
This may be problematic when you have not used more height than the display, but also, the activation will take up less time, so the SE user may not have the problem.

I'm going back to the Canvas me

[Gert]

That reads like excellent analysis work !!!!!!!!

I'd still like to give you some peace of information on that SE bug from the past. I am not sure if this is related to the investigation that you are doing, anyway, it may be useful for you to know:

Already in the early 2 versions of DfM (before 2.5), when there was not StringColourItem and no BitmapFont (i.e. no CustomItem) there were problems with showing the translation results. On some Java Platforms (maybe also depending on the firmware revision number; I never understood what really was the criteria for the affected devices), the translation results were not shown; this led to the incorporation of the 'strange' workaround that is still in the DfM source now (the workaround was extended at some places since then). Also, the affected Java Platforms sometimes did throw an OutOfMemory error during the translation output, although of course there still was plenty of memory available.

Besides, Sebastian ('Tomcollins') did do some investigations in the past, you may already have read his postings here in the forum.

Hope one day those SE problems will get solved !!!
Thanks !!!
Gert

[Gert]

Already anything new from the SonyEricsson front ... ?

Gert

[Gert]

Tom,

in order to solve the SonyEricsson problem, what code exactly needs be included into the DfM source ?

Please, only well tested code which does work on all devices 

Best regards,
Gert

[NewAgeMobile]

what code exactly needs be included into the DfM source ?

Pretty much all of the MainForm code that I posted seems to be needed:

The sleep subroutine to implement a forced delay between adding the items: It appears that it is a time sensitive problem.
The call of this when new items are added: Having this delay stops the (I suppose) failure of the SE properly registering the message calls (like 'paint') on the form's queue.
The forcing of background (threaded) execution of the dictionary search: Without this, the SE tries to update the form as items are added, but in a very broken way; a bit like a severe windows slowdown.
Moving the redisplay translation results call so that this is not done on the SE when the settings form is closed: Without this, the user can interact with the settings form while refreshAllTranslationResults(); is executing and since the added delay between item adding is in place, the process is much slowed too (depending on the number of results).

The sizeChanged() additions to the two CustomItem types classes are not essential, but probably also needed on a more general level just to make sure that when the display width is changed, the formatting is re-calculated so that eg: the VM adding a scrollbar does not cause the ends of lines to get hidden by the scrollbar.

well tested code

Put it this way, it works on my k800i 90-99%. I doubt there is a 100% fix because it is a problem with the VM on the phone. Without it, the custom items work about 0-2% of the time.
I have done all of the testing I have had the time and patience for; this was only with the one dictionary:- The English -> Chinese one and mostly with the test Midlet, which eventually ended up as the one I posted, though I think the method used in that of 'activating' the 'full functionality' of the CustomItems after they are added is not important to DFM: The 'sleep' works, and I think there would be too much to change (and too much potential for breaking things) to do this in DFM.

which does work on all devices

I can't know this because I only have a K800i to test on, so:
I have no idea if newer SE models have this bug.
I have no idea if older ones have it.
The problem appears to be time sensitive: Things break when an item is added when the previous one is still being processed by what appears to be a VM thread, so the delay may not be sufficient on older (read: maybe slower) SE phones if they have the same problem.

I can only appeal for any people who have other SE models who read this forum to try the fix on them!