Menu

Show posts

This section allows you to view all posts made by this member. Note that you can only see posts made in areas you currently have access to.

Show posts Menu

Topics - Gert

#41
Anyone who knows HTML 5 ?

Currently there is a lot of talk about creating 'applications' with HTML 5. So I was wondering whether some time in the future it might be possible to do an implementation of DictionaryForMIDs with HTML 5. But I do not know if a good application could ever be done with HTML 5. I mean a real good and fast offline dictionary (dictionary stored locally on the device).

For the Web Storage (see http://en.wikipedia.org/wiki/Web_Storage) I read that there are space limitations for browsers; that could be an issue.

Also, I guess only JavaScript will be available on all devices as programming language, true ? Then the DictionaryForMIDs Translation Layer will need to be ported to JavaScript [the current format of the dictionary files (Jar files) will have to be handled in JavaScript]; would JavaScript fast enough to do a quick search in the dictionary files ?

Download of dictionaries may be done with the interface that is used by the Android version.

Well, just thoughts -- I will appreciate if there is someone who knows HTML 5 and can say what are all the obstacles.

Best regards,
Gert


#42
I just came across the following article at the Nokia forum:
http://www.forum.nokia.com/Distribute/Packaging_and_signing.xhtml#article0

Maybe that will be the solution for using the "loadable dictionaries" on the mass-market Nokia S40 devices (the S40 devices support only small Jar file sizes and thus currently only can run small dictionaries).

I will look at that more closely ... when I have more time.

Regards,
Gert
#43
Who will implement DictionaryForMIDs for the iPhone ?

For example by using the Android implementation as a starting point for the iPhone implementation.

If you have questions on DictionaryForMIDs, just post them here in the forum !

Regards,
Gert
#44
We have the testversion of 3.5.0 for Java ME, i.e. for devices other than Android.

For a list of new features see http://dictionarymid.sourceforge.net/history.html#DictionaryForMIDs_3.5.0.

Here is the 'empty' version (without dictionary):
http://prdownloads.sourceforge.net/dictionarymid/DictionaryForMIDs_3.5.0_empty.zip?download

We still will provide a test version with a dictionary.

If you detect any bug with that testversion, please make a posting in this thread (i.e. "Reply" to this posting).

If no bug is reported till June 11 2010 then this will be the new 'official' version for Java ME.

Thanks for testing !
Gert
#45
Colleagues,

several dictionaries out there in the world are maintained in an XML format. Some of these dictionaries use an XML structure that is specific to a tool. Some adhere more or less to a standard such as TEI or XDXF (both TEI and XDXF are XML-based).

In order to ease set-up of such dictionaries in XML format, I am currently establishing an XML representation for the DictionaryForMIDs dictionaries. I call it "DfM_XML_schema".

The main task for setting up a dictionary that is maintained in XML format, is to transform its structure into the DfM_XML_schema.

Dictionaries in the format of DfM_XML_schema can be automatically converted into an 'inputdictionaryfile' (read below). And the inputdictionaryfile is then run through DictionaryGeneration/JarCreator.

This allows to set up quickly a dictionary that is maintained in XML format (well, some experience with XSLT or another XML transformation tool will be useful to get it done quickly).


Here some technical information:
1.
The is the XML schema file: DfM_XML_Schema.xsd. Look at http://www.kugihan.de/dict/download/Preprocessing/DfM_XML_Schema.xsd (not yet documented and still in work)

2.
There is the XSLT transformation script that converts an XML file in the format of DfM_XML_Schema: http://www.kugihan.de/dict/download/Preprocessing/DfM_XML_to_inputdictionaryfile.xsl (still in work)

3.
Here a sample file in the format of DfM_XML_Schema:
<?xml version="1.0" encoding="UTF-8"?>
<dictionary xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.kugihan.de/DfMSchema DfMSchema.xsd ">
  <translationOfDictionary>
    <translationForLanguage includeInIndex="true">
      <translationForLanguagePart>
      <partNonContent>cat</partNonContent>
      </translationForLanguagePart>
    </translationForLanguage>
    <translationForLanguage includeInIndex="true">
      <translationForLanguagePart>
        <partNonContent>chat</partNonContent>
      </translationForLanguagePart>
    </translationForLanguage>
  </translationOfDictionary>
  <translationOfDictionary>
    <translationForLanguage includeInIndex="true">
      <translationForLanguagePart>
        <partNonContent>bird</partNonContent>
      </translationForLanguagePart>
    </translationForLanguage>
    <translationForLanguage includeInIndex="true">
      <translationForLanguagePart>
        <partNonContent>oiseau</partNonContent>
      </translationForLanguagePart>
    </translationForLanguage>
  </translationOfDictionary>
</dictionary>


Run this file through any XSLT conversion tool and the output will be the inputdictionaryfile:
cat    chat
bird   oiseau


Well, that does not yet look sophisticated, I know. But it will help a lot to set up XML dictionaries !

Important note: all of the above is work in progress and is not yet ready to use. I plan to improve the above files during the next weeks.

Ok I keep you updated on the progress !
Gert
#46
Colleagues,

some people who did run DictionaryGeneration experienced that the files which were generated are very big. For example, if the inputdictionaryfile is 2 MB, then the generated files were > 10 MB. In this case very probably the index files contain unnecessary information.

To illustrate the problem, here is an example with a line from the inputdictionaryfile:

sleep  The state of reduced consciousness of a human or animal[tab]Schlaf  Zustand der Ruhe eines Tieres oder Menschen

Note: [tab] is for the tab-separator character.

Here, without additional information, DictionaryGeneration will index all expressions that are included in the explanatory texts (e.g. "The state of reduced consciousness of a human or animal"). This is undesireable.

The solution is to use a DictionaryUpdate-class that avoids including the unnecessary indexes for the explanatory texts. In simple cases you can use the class DictionaryUpdatePartialIndex. If you set DictionaryUpdatePartialIndex as DictionaryUpdateClass, then the text between {{ and }} will not be included in the index.

In the example:
sleep  {{The state of reduced consciousness of a human or animal}}[tab]Schlaf  {{Zustand der Ruhe eines Tieres oder Menschen}}

And put in DictionaryForMIDs.properties these two lines:
language1DictionaryUpdateClassName=de.kugihan.dictionaryformids.dictgen.dictionaryupdate.DictionaryUpdatePartialIndex
language2DictionaryUpdateClassName=de.kugihan.dictionaryformids.dictgen.dictionaryupdate.DictionaryUpdatePartialIndex

Then the size of the generated files will collapse. For an inputdictionaryfile with lines as in the above example, the compressed result will likely be below 2 MB.

For advanced information on DictionaryUpdate-class read here: http://dictionarymid.sourceforge.net/newdictDictionaryUpdate.html (you do not need to read this if you use DictionaryUpdatePartialIndex)

Regards,
Gert


#47
Dear cell phone users,

I'd like to know what you think about downloading DictionaryForMIDs with your cell phone from the internet ?

In the old days it was rare that people downloaded applications with their cell phones from the internet, because often the telecon companies did have ridiculously high prices on data transfer (only a very little percentage of cell phone users had an internet flat rate on their cell phones). So people downloaded the applications on their PC and installed them from there on the PC.

Nowadays data flat rates are becoming more and more common also for normal cell phone users. This allows more convenient download directly from the cell phone.

For downloading DictionaryForMIDs with a cell phone, what would be a convenient way to do this ? For example, would you want that all DictionaryForMIDs dictionaries are available from the cell phone manufacturer's 'app store'. I.e. from the Nokia app store, the Samsung one, the {put your manufacturer there} app store ?

Or, would you like to have a dictionary download page similar as http://dictionarymid.sourceforge.net/dict.html, but better adapted to cell phones ? That means making something like a dictmobile.html where people can directly install the Jad/Jar-files (instead of first downloading a zip file).

Or should all the DictionaryForMIDs dictionaries be downloadable from one application hoster, such as phoload.com or getjar.com.

Or maybe any other idea ?

Just let me know your viewpoint on this !

Regards,
Ger
#48
There is a very important Java SE development task that needs to be done: developing a User Interface for the DictionaryGeneration/JarCreator command line tools. That means: develop one or more windows/dialogues, so that users can enter their data there.

Currently the tools DictionaryGeneration and JarCreator are Java SE command line applications, i.e. these tools are called from a command prompt with a set of command line parameters. (for the existing command line interface read here:http://dictionarymid.sourceforge.net/newdict.html#SetupGeneration).

It turned out, that there are many people who like to set up their dictionary for DfM, but the command line interface of DictionaryGeneration/JarCreator is too complicated for them. So it is important to develop a User Interface for these tools !!

Actually, for this task a 'first shot' was already made, read here: http://dictionarymid.sourceforge.net/forum/index.php?topic=95.msg529#msg529 [unfortunately, it seems that the attachments there are not available in that posting; just contact me if you need them]; for a screenshot see the attachment to this posting.


It will be straightforward to do further development. Other people will benefit very much ! I believe that having a simple Graphical User Interface a _lot_ more people will be able to set up their dictionaries for DfM. This is why it is so important to work on this task.


Here are a few considerations for further development:

  • The tools DictionaryGeneration/JarCreator (and later BitmapFontCreator) may be included into a single new tool; proposed name: "DfM Builder".
  • The latest version of 'DictionaryForMIDs_empty' can be directly included in the "DfM Builder.jar"-file.
  • In an advanced version, the DfM Builder may have a dialogue which allows to enter all data of DictionaryForMIDs.properties !
  • DfM Builder should still support the command line interface. When DfM Builder is started with command line arguments, then the command line arguments shall be used. When DfM Builder is started without command line arguments, then the User Interface shall be displayed.

This Java SE development task is suited for an individual developer, as well as maybe for a college course or similar (is there anyone who knows a college that searches a useful topic for a programming course ?).

Ok, just let me know if there is anyone who is interested and may help here  :)

With best regards,
Gert
#49
Zdenek did set up a dictionary that comes from the Wiktionary. You can download it here:

http://www.kugihan.de//dict/download/test_versions/3.5prerelease1/dfm_wiktionary.zip

Just let us know if it works well and if it is useful.

Regards,
Gert
#50
Achim,
Sabine,

from the International Mother Language Day, what User Interface translations were made for DictionaryForMIDs ?

How is the result available for the Java ME version ? Guess till now no one made the effort to build an updated file DictionaryForMIDs.languages.

In any case, Achim, Sabine, thanks for your support there !!

Regards,
Gert
#51
When there a language is added to the user interface translations, then on Java ME the LanguageUIGeneration tool must be run.

On Java ME there is the file DictionaryForMIDs.languages that contains the user interface translations (see http://dictionarymid.sourceforge.net/LanguageUITranslations.html).

The LanguageUIGeneration tool reads the file DictionaryForMIDs.languages and generates the files UIDisplayTextContents.java and UIDisplayTextItems.java. These two files are part of the DictionaryForMIDs source tree and are compiled like all other DictionaryForMIDs Java files.

A description for using the LanguageUIGeneration tool is found here: http://dictionarymid.sourceforge.net/LanguageUIGeneration.html.

Regards,
Gert
#52
Seems the DictionaryForMIDs entry on Wikipedia does not yet recognize the Android version: http://en.wikipedia.org/wiki/DictionaryForMids

Anyone who would like to update that entry ?
Gert
#53
Colleagues,

we received the "VocTrainVH-dictionary" from Sabine from Vox Humanitatis and from Zdenek. This dictionary contains almost 30 languages.

For testing purposes, I just tried to do a "quick-and-dirty" set-up of this dictionary, but DictionaryGeneration reported an error:

Checking: ..\..\Related Files\Dictionaries\vox\VocTrainVH-eng.txt

Number of separator-characters is not correct in line 359

Number of found separator-characters: 30 / expected: 29


Here are the files that I used: http://www.kugihan.de/dict/download/test_versions/3.4vox/vox dictionary.zip


  • File VocTrainVH-eng.xls:  the file that I received
  • File VocTrainVH-eng.txt:  the xls-file saved as UTF-8 encoded tab-separated txt file. Note than I did put the first 2 columns at the end. I also removed the top line.
  • DictionaryForMIDs.properties: the corresponding properties file.


I did not yet spend time to check what is wrong at line 359; maybe there is a tab within a word, so that in total there are 30 separator characters instead of 29 ?

Maybe Sabine, could you have a look at that ?

Best regards,
Gert
#54
Cause I do not yet have an Android device, I would like to see how DictionaryForMIDs looks on the Android SDK Emulator.

Well, simple question: how do I install DictionaryForMIDs on the Android SDK Emulator ? I understand that DictionayForMIDs can be downloaded from the Android Market. But how do I do this with the Android Emulator ?

I do use the 'platform 2.0' on my SDK.

Thanks ...
Gert
#55
I just read about the release of Android 2.0 and I noted that there are quite a few changes on the Android API. I was wondering whether the old APIs are still supported with Android 2.0.

Well, is the existing DictionaryForMIDs working on Android 2.0 ? Or does there need to be an updated version of DictionaryForMIDs for Android 2.0 ?

Gert
#56
The latest DfM versions (for Java ME) provides some improved support for handling 'loadable dictionaries'. You can download it here:
http://www.kugihan.de/dict/download/testversions/3.5development2/DictionaryForMIDs.jar

[I still need to write down the list of improved features during the next days]

Gert
#57
Gentlemen,

srecently there had been an increasing amount of spam postings here on the forum. As s short-term action all DfM developers are now endowed with the permission to ban users and to delete postings. Thank you to all and specifically to Achim who used their permission to get rid of the latest spam postings !

What is your opinion, how should we continue on this problem ? Just by deleting the spam postings once these show up ?

Or should we change our registration scheme for the forum, so that a person who requests access to the forum has to send a 'personal' email to a 'forum member administrator' who then manually activates the account ? Well, we would need to find a few 'forum member administrators' who are ready to respons to registration requests.

Or any other solution ?

Besides, Stefan (our 'all forum administrator') told me that there are spambots which are able to decode the Captcha pictures during the registration.

Waiting for your opinion,
Gert

#58
A few years ago I spent quite some time to set up the FreeDict dictionaries for DictionaryForMIDs. However by now DfM advanced a lot and it would be nice to set up these dictionaries for the latest version of DictionaryForMIDs.

Jeff may have some time to add these FreeDict dictionaries to his dictionary archive and to set them up for the latest version of DictionaryForMIDs. However I realize that this may be quite some effort, so I just want to describe here what I have ... and maybe there will be more volunteers who like to help ?


1. Some background information

The FreeDict dictionaries are found at http://www.freedict.org/. An overview of their dictionaries is provided at http://freedict.org/en/list.html.

When I was setting up their dictionaries, then I was using their files in the DICT format, together with the DfM application DictdToDictionaryForMIDs, see http://dictionarymid.sourceforge.net/DictdToDictionaryForMIDs.html. DictdToDictionaryForMIDs produces the normal CSV files from the DICT files. The DICT files have .dz and .index as extension.

I do need to mention that generating the CSV files from the DICT files was sometimes troublesome, because the DICT files had subtle differences in the format.

Well, somehow I managed to generate the CSV files, actually I gave them the extension .txt.


2. The content of my FreeDict archive

I uploaded my FreeDict archive here: http://www.kugihan.de//dict/download/dictionaries/freedict/FreeDict_dictionaries.zip

There is the directory 'inputfiles' which contains several subdirectories, one for each freedict dictionary. Within the subdirectories there are the .txt files along with the files DictionaryForMIDs.properties. Usually also the dict-files (extension .dz and .index) are in these subdirectories.

Not for all subdirectories I made the effort to set up the dictionary; for example for cro-eng and several others, there is no generated  .txt file.


3. How to generate the files for DictionaryForMIDs

Well, run DictionaryGeneration on the .txt file with the corresponding DictionaryForMIDs.properties that is in the same directory. See for example eng-deu.

And here is the problem: the current DictionaryGeneration is more strict than the old one was. DictionaryGeneration now reports empty fields in the inputdictionaryfile and exits (I added this feature to DictionaryGeneration because I was receiving error reports that resulted from missing translations in the inputdictionaryfile). I found that his happens for eng-ara.

To solve this problem, one would have to load the .txt file in Excel or similar and remove all lines that contain empty fields.


4. Any more challenges ?

Sure ;)

The FreeDict dictionaries are unidirectional, means for example English to German. Of course a user will like to have also German to English in one dictionary.

Please do read the instructions at http://dictionarymid.sourceforge.net/newdictMultiple.html for generating a combined dictionary such as English to German plus German to English.

Well I mentioned above, it requires some effort ...  ;)


5. Where is support needed ?

Just let me know.

If it helps, then I can upload the files that I generated some years ago. However, because these are very old, most of the generated dictionaries will not run on the current version of DictionaryForMIDs.


Greetings,
Gert
#59
I just did put 3.4.0 on sourceforge as the new official version for Java ME.

Honestly speaking, I did not have time for testing, so this time I fully trusted in the correctness of the build script  :o

Well, if someone would do a test and set up a dictionary with that new 3.4.0 version, and check if it runs fine, I would appreciate that !

But most of all - thank you very much to all people who contributed to this 3.4.0 version and made this progress possible. Special thanks to Joshua whose significant improvements on the 'loadable dictionaries' made it to 3.4.0 (more than one year after Joshua submitted his improvements  :-[ ). And special thanks to Heiko for his Norwegian user interface translation. Hope we will have his Norwegian dictionary on 3.4.0 for public release soon.

Best regards,
Gert
#60
We have the first testversion of 3.4.0 for Java ME.

For a list of new features see http://dictionarymid.sourceforge.net/history.html#DictionaryForMIDs_3.4.0.

A version with a dictionary can be downloaded here:
http://www.kugihan.de/dict/download/test_versions/3.4testversion1/DictionaryForMIDs_EngTagSpa_CAL.jad
http://www.kugihan.de/dict/download/test_versions/3.4testversion1/DictionaryForMIDs_EngTagSpa_CAL.jar

And here is the 'empty' version (without dictionary):
http://www.kugihan.de/dict/download/test_versions/3.4testversion1/DictionaryForMIDs_3.4.0_test version 1_empty.zip

And here the updated JarCreator (if you like to set up a dictionary):
http://www.kugihan.de/dict/download/test_versions/3.4testversion1/DictionaryForMIDs_JarCreator_3.4.0_final.zip

Besides, 3.4.0 does require CLDC-1.1 (previous versions also did run with CLDC-1.0).

If you detect any bug with that testversion, please make a posting in this thread (i.e. "Reply" to this posting).

Thanks for testing !
Gert