Java Specification Wanted

During the weekend, I wasted long hours on searching for a Java specification. The document is question is called Configurations and Profiles Architecture Specification and it is like the Constitution of JavaME. It defines the architecture for the JavaME platform, what are the rules that you must follow when standardizing APIs for JME. What can you do in a JSR and what you mustn't.

However, if you Google this title, you will get only a handful hits, almost each of them are references to this document from other specifications. And a 5-year old entry from a forum where somebody was seeking the same document. I searched for it on java.sun.com, even in the Wayback Machine but couldn't find it. The closest I got to this document is that I talked to someone who saw it but didn't have a copy anymore.
I'm not saying that it cannot be found, but I failed miserably. So if you happen to know the whereabout of this document, or happen to have a copy of it in some dusty corner of you hard drive, I would really appreciate if you could share it with me.

And apart from the fact how lousy I am in web searches, wouldn't you expect such an important a document to be in a very obvious and visible place?

ProSyst publishes OSGi for Android

Today ProSyst released the beta version of mBS for Android. mBS is the leading OSGi implementation in the embedded market. The beta release of the Android version is available from ProSyst Developer Zone. This release is under ProSyst Non-Commercial License, which permits free use for non-commercial trials.

Why to use OSGi on top of Android?

OSGi and Android are both component platforms. However, due to the differences discussed in an earlier post about Android, there are significant benefits in using OSGi on top of Android:

• Lack of class-sharing: Android doesn't support the class sharing between application packages, which leads to class duplication between components of a modularly deployable application. Such class duplication puts high burden on platform maintenance as well if there are shared components.
• Fine-grained components: in Android, component communication involves interprocess communication which leads to coarse-grained components with high abstraction-level interfaces. The efficient communication between OSGi components - i.e. direct method calls - permits a much finer grained decomposition. The fine-grained component model enables plugin architectures like in Eclipse eRCP.
• Device Management: Remote manageability is an essential feature both for enterprises and mobile operator. Android currently doesn't include this feature, but OSGi offers an out of the box solution for this are.

Release Content and Requirements

Content of the release:

• mbs.apk - mBS for Android
• mconsole-proff-6.1.41-19-12-2007.zip - ProSyst mConsole remote management tool
• demo.zip - Bundles demonstrating the use of OSGi services on top of Android
• readme.htm - Detailed installation instruction for the release and the demos
• ProSyst Non-Commercial license.txt

mBS for Android

The mbs.apk package contains the mBS distribution as an Android application package. It can be directly deployed to Android using the adb tool. The product can then be found under the All desktop item.

This mBS distribution contains the full OSGi R4 framework implementation and the following services:

Configuration Admin IO Connector Console Data store (db.jar) Device Manager HTTP Service Servlet API Kit Installation Manager Log Service(log.jar) Metatype Service ProSyst Message Protocol Initial Provisioning Service ProSyst Utility Package Socket Connection Service Telnet Console User Admin Service Remote User Administration support

The mBS port works with Android M5 or later as it depends on some bugfixes introduced in the M5 release.

mConsole

The mconsole-proff-6.1.41-19-12-2007.zip contains ProSyst mConsole, a remote management console that allows the management of OSGi framework. This package can be installed to a PC and it can be used to remotely manage the OSGi framework running on Android. This version of mConsole uses the ProSyst Management Protocol for the management but other management protocols are supported as well. Using the management console allows:

• Installing/updating a bundle to the framework
• Uninstalling bundles
• Starting/stopping bundles

mConsole runs on Windows-based PCs.

Porting Experience

Porting Java applications to Android, if it doesn't involve the porting of UI should be easy. However, during the technical work, ProSyst team had to face some challenges due to some bugs in Android and the lack of documentation for the Dalvik Virtual Machine. The major issues are described here to help the work of other porting projects:

Stack Size

By default the stack size of the Dalvik VM is set to 1 KB. Which was a bit few when starting mBS with a large number of bundles. The stack size can be set using the -Xss command line argument. For example -Xss65536 sets the stack size to 64K. Unfortunately, it's not known how this argument can be specified for a regular Android application. As a workaround for this issue, mBS was modified to explicitly set the stack size when the corresponding Thread objects are created.

DEX converter bugs

There are some bugs in the dx tool provided in Android. ProSyst reported these bugs to Google, but it's good to be aware of these limitations until the problems are fixed.

The dx tool fails to generate code for the following try-catch-finally structure:

try {
// do something but do not return anything.
// If you have "return object" here the
// construction is OK.
} catch (Throwable th) {
// log the error
} finally {
// free the resources
// The DEX file generator fails to parse
// this section and starts eating memory
// and doing nothing.
}


Another failing construct is when Object[] is used to refer to different types of arrays, like in the code snippet below:

Object[] data = null;
if (condition1) {
data = new String[10];
} else if (condition2) {
data = new MyObject[10];
}


Future Plans

ProSyst is working on additional demos and tools that help to leverage this technology. This release was created as a proof of concept to demonstrate the feasibility of using mBS on Android. However, ProSyst can productize this version if there is a significant interest in using this technology.

A quick comparison of MIDP3 and OSGi

A number of times I heard the argument, that JSR-271 MIDP 3.0 provides a solution for everything that OSGi provides, maybe less elegantly, so JSR-232 is not needed. It had a been a while since I last read JSR-271, so now I took the time and read the Public Review version of the specification. So here I provide a brief comparison of the two specifications. I don't want to claim that OSGi is better or worse than MIDP 3. It would be comparing apples to peaches, they are meant for different purposes. I just want to prove, that OSGi does add value, even to MIDP 3, so it makes sense to use them together. And the added value is not only by providing more elegance.

Device Management

I mention Device Management only briefly, as this is clearly the apples to peaches category. A significant part of JSR-232 is dealing with operational management of the components throughout their complete lifecycle, from deployment to removal. I think operational management is a very important added value. It is easy to see that manageability of software assets is essential for enterprise applications, where the financial success of a company can directly depend on the proper operation of an applications. However, it is equally important for mobile operators to reduce their customer care costs and improve customer satisfaction. Just consider the recent release of Sprint Titan platform.

MIDP 3 on the other hand hardly contains anything related to management of the applications. The only exception is the over the air provisioning of MIDlets. So there is a lot of value to add here.

Component Platform

As remote management is so clearly out of the scope of MIDP 3, people were referring to the component model of the two platforms when they said that MIDP3 has everything that OSGi has.

In earlier releases, MIDlets used to be monolithic applications running in isolated silos. There was no interaction between MIDlets and it wasn't possible to share code either, well except by copying the classes from one version to the other. MIDP 3 introduced some new concept to improve in these ares:

• Liblets are class library like packages, which can be used to share code between MIDlet suites
• Inter Midlet Communication using a special ICM protocol
• Autostart MIDlets are MIDlets that the AMS must start when the device starts and restart if they fail.

These are the technologies that now form the "MIDP 3 component platform". Let's see how these realize some common characteristics of such platforms.

Encapsulation

One characteristic of a module system is encapsulation, the ability of a component to hide the specifics of its implementation and selectively export its internals. OSGi bundles can decide on Java package level which classes to make available for other components. Liblets are completely transparent, all resources in a Liblet are available for its clients. Additionally, the Liblet is executed in the namespace of the MIDlet using it - together with all other dependent Liblets as well - and it is the responsibility of the Liblet and MIDlet developers to avoid name collisions between these entities.

Dependency Management

In OSGi, the dependencies are expressed in terms of Java packages and their versions. The dependencies are resolved at runtime, the whole system is so dynamic that the components can be updated even at runtime, without breaking the operation of the platform.

MIDlet suites have to express their dependencies on Liblets in the MIDlet descriptor and manifest. During the installation, these dependencies are recursively resolved, and if the resolution succeeds, all required Liblets and the MIDlet suite will be installed.

OK, MIDP 3 is "less elegant" than OSGi. But it still handles dependencies, doesn't it? Well, there is a "small" difference between OSGi and MIDP 3 dependency model. As I said, OSGi uses package names and versions to express dependencies. MIDP 3 uses Liblet name, vendor name, version and the SHA-1 hash of the Liblet package as the means of expressing dependencies.

What does this mean? All dependencies refer to an exact compilation of a a particular Liblet, as sometime even a recompilation gives slightly different byte code, which results in a completely different hash value. So if you want to update a Liblet on a device, you have to update all Liblets and MIDlets that depend on it. Even if you corrected a small bug or just lost the binary copy of the earlier compilation.

So yes, literally speaking there is dependency handling, but the true purpose of managing dependencies - i.e. the ability to mix and match components while ensuring their proper interoperability - is completely lost, due to the rigid mechanism, which prevents any mixing.

Besides, this solution greatly reduces the value of Liblets in decomposition as well. You cannot change a Liblet individually. If there is a bug in a Liblet, you have to redeploy all dependent applications and Liblets to fix it.

Security Model

In OSGi, each bundle has its own associated permissions, its own protection domain. Bundles coming from a trusted source can obtain high privileges and expose services for less trusted components in a controlled manner. For example the permission to load native code should not be available for all components. But a component from trusted source can expose a native service to all other components by wrapping it with a Java interface. The clients of this interface will not need the permission to load any native code to use this service. This enables the creation of bundles that extend the platform with new services.

MIDP 3 Liblets are executed in the protection domain of their client MIDlets, which means that the code in the Liblet is executed with the exact same privileges as the host MIDlet. Therefore a Liblet cannot do anything that the host MIDlet could not do either.

Component Interactions

OSGi components interact via services. Services can be found via the central service registry, where any component can register its services. A service is then accessed via method simple calls and arbitrary data types can be used as parameters.

Apart from shared files or shared databases, MIDlets can interact via events. MIDP 3 defined an eventing mechanism, which allows MIDlets to publish and listen to events. An event can carry a single scalar or String value and an additional byte array as data.

I guess it's pretty clear which mechanism is more developer-friendly.

Summary

I have no problem with MIDP 3. Honestly! I do respect the work they did in the expert group and I understand how heavy burden the backwards compatibility requirement with MIDP 2.1 and CLDC was. I just wanted to point out that JSR-232 can add value, even to MIDP 3. And this value is not only marginal improvement of some existing features of MIDP 3. The means provided by MIDP 3 to decompose and manage complex applications are not suitable. JSR-232 can complement these weaknesses by its well-established component model and rich features in operational management.

Developing Titan Applications - Tools

Sprint has released their new Titan platform last week. Titan is a next generation Java platform for Sprint Windows Mobile devices. In addition to supporting existing MIDP Java ME applications, it includes the more capable CDC/Foundation Java Virtual Machine and OSGi framework. You can read more about Titan in Mobile OSGi blog. Titan is currently available for Windows Mobile 6 devices as an installable add on. In the future, devices will ship with pre-installed Titan, and Sprint plans to extend Titan support to other phone categories and platforms as well. This is a very important milestone for mobile OSGi and it opens up very exciting new opportunities for mobile Java. I plan to write a number of articles about how to develop applications for this new platform. However, before starting development work, it worth to have a look at the tools available. There is a step-by-step introduction - in the form of videos - on the Titan page at Sprint ADP. I don't want to repeat the information available there, rather complement it with some practical notes.

The Development Tools

From the Titan page, after some registration you can download the development tools for creating applications for Titan. The tools are based on the Eclipse IDE 3.3 Europa release. First you need to set up the environment. Follow the instructions on video for how to install and configure the development tools. The configuration step is very important in order to make sure that the IDE uses the provided CDC 1.1 Java runtime instead of your installed JDK version and the provided eRCP version as the development target instead of RCP provided as part of the Eclipse IDE.

First Project

Now you're ready to create your first project. The project template creates a simple eRCP application with some basic UI elements. We're ready to try it! Unfortunately, there's no built in device emulator in the toolset. The tool supports on-device execution and debugging (see video) only. So what can you do, if you don't have access to a WM6 device, or you find it more convenient to start first with an emulator on PC? Fortunately, Titan Runtime can be installed to Windows Mobile 6 Emulator.

The Emulator

First download and install the emulator to your PC. The emulator is available in a number of different localized versions, pick the one you like. However. make sure that you download one of the Professional images, Titan doesn't work with the Standard ones. Now you need to install Titan Runtime to the Emulator. The easiest way to get TitanDeviceRuntime-v1.0.CAB to the emulator is via a shared folder. After starting the emulator, you can select a shared folder in File > Configure... menu. The content of the shared folder can be accessed from the Emulator using the File Explorer in the "\Storage Card" folder. Just click on the CAB file to begin the installation. Similarly, you need to install the Titan Device Tools, which enable "on-device" debugging of applications or application startup from the IDE.

Network Setup

The desktop and device-side tool components use sockets for communication, so you need to setup a network access.

1. First you need to download and install Virtual PC 2007 because the Emulator uses the virtual network adaper support in VPC.
2. Now you can enable the "NE2000 Network Adapter" in the Network tab of the Configuration window of the Emulator.
3. After enabling it, you should bind the adapter to one of the network interfaces installed to your PC. For this purpose, I used the Microsoft Loopback Adapter.
4. You have to assign a static IP address to the emulator, because you have to be able to configure that IP address in the tools in your IDE. Open Settings in the Start Menu of the Emulator, select the Connections tab and then Network Cards. Select the "NE2000 Compatible Ethernet Driver" and then Use specific IP address. I used 192.168.42.1 as the address of the loopback adapter and assigned 192.168.42.2 to the Emulator. You need to softreset the Emulator (File > Reset > Soft) to ensure that the changes take effect.
5. Finally, you need to configure the development plugins in the IDE. Open Run > Run Dialog..., select OSGi on Device from the list on the left and then fill in the form, entering the IP address in the Device IP field.

Now you can try to run the sample application in the Emulator. Click Run! You should see something like this:

Final Notes

After the installation of Titan Device Tools, a black and blue disc-like icon appears on the task bar. This is a simple application that allows you to manage the lifecycle of the OSGi framework on the device. By default the framework is always running, you can use this application to stop or restart it. Notice that the icon is animating while an operation is in progress! WM6 Emulator emulates an ARM CPU and executes ARM native code, which is quite resource demanding, so don't expect too quick operation. In particular, when you start the emulator, it is useful to wait until the framework completely starts, before trying to run an application.

A Comparision of OSGi and Android

Author's note: this article was originally posted to the ProSyst blog. I just publish it here as well, without changing the content.

Introduction

Google and the Open Handset Alliance announced the Android platform late last year. Shortly after that the Android SDK was published to the developer community and an exciting 10 million dollar developer challenge was announced too, which generated lots of developer interest. Due to this interest and the support of a number of industry players, Android is expected to gain significant market share in the future.

OSGi is a technology that has been developed for 10 years. Originally created to be the core platform of home gateways, slowly it was adapted in a number of verticals, including automotive industry or enterprise applications. In 2006, Release 4 of OSGi was published, which includes support for mobile phones and other handheld devices. Based on this technology, JSR-232 defines a dynamic, manageable component platform for mobile devices. Nokia eRCP platform, which is available for E90 and some other S60 3.1 devices, is based on JSR-232. The same technology is used by the Sprint Titan platform, for which devices will hit the market in 2008.

Both OSGi and Android define a development platform for component-based development and provide an implementation for the service-oriented architecture, within the mobile device. So, are they the same or completely different?

Admittedly, there are a number of similarities as well as some key differences. This article provides a high-level comparison of the two platforms, highlighting these key differences.

Comparison

Runtime Environment

Though somewhat external to the platforms, it worths to have a look at the runtime environments in which the two platforms are used.

OSGi can be used in a number of standard Java environments. It defines the minimum runtime requirements – so called Minimum Execution Environment – and any Java implementation that meets those requirements is sufficient to run OSGi. Minimum EE is a quite small API subset, which ensures that OSGi is usable in a wide variety of Java Runtimes, including JSE 1.1 or JME CDC1.0/FP1.0 as examples. This gives OSGi the required flexibility, which makes it applicable in a wide range of verticals from home automation to mobile devices. Frankly, OSGi extends a Java VM by introducing additional runtime execution environment features like the component model.

Android is an operating system with a new and proprietary application runtime technology. Though the applications are developed using the Java language, they are then compiled to a proprietary byte code format and are executed on the Dalvik Virtual Machine, a special VM created for Android. It is interesting to note that if converted to a Dalvik Executable, OSGi can be run on Android.

A key difference between the two platforms is that while Android applications are executed in separate instances of the DVM, OSGi executes all applications in the same JVM. Executing application in separate DVM instances – i.e. separate OS processes – has the advantage of easier resource management and resource reclamation, when applications need to be stopped forcefully. On the other hand, it requires more memory and communication between components is more complex and more expensive.

Component Model

Though both OSGi and Android are component platforms, there are a number of differences between their approaches. Probably the most important difference is the class-sharing ability of OSGi. Android components are executed in separate processes, which are fully isolated. It is not possible for a component to load a class from another one. This means that when a component uses the services of another component, all the service interfaces and parameter classes must be part of the client component as well, which creates a strong coupling between components and leads to unnecessary code duplication, which makes the maintenance very complicated. OSGi uses a classloader-based isolation model, which allows the components (bundles in OSGi terminology) to selectively export some of their internal packages to other bundles. Bundles can declare their package dependencies and the framework is responsible for matching the required and provided packages and make sure that the system is consistent. These mechanisms ensure that there’s no need to duplicate code among bundles and it helps to reduce the coupling of bundles and simplifies maintenance.

A related difference between the two models is component versioning. In OSGi, everything is versioned, while Android provides no support for versioning at all. Though at first glance, the extensive use of versioning may seem to be annoying in OSGi, the use of version numbers and version ranges enables the platform to very elaborately manage dependencies between components and thereby ensure that the whole system is in a consistent shape. This is extremely important for post-shipment maintenance of the platform.

Service Model

The key difference between the two platform’s service models is the service access. In OSGi, services are lightweight. They are accessed using direct method calls, they are easy and cheap to use. In Android, services are heavyweight. They are accessed using inter-process communication, which makes service access slower and more expensive. This leads to a different style of using services. In OSGi, services are used extensively and they play a very important role in keeping components lightweight and loosely coupled. In Android, services are coarse grained and will likely be used much less frequently. While in OSGi services can be used as plugins to other applications, in Android they are closer to the traditional web service model.

Application Model

Android defines its own application model. Applications can be constructed from components of four types. Activities are components, which interact with the device user. Services are background execution tasks. Content Providers are data stores and Intent Listeners are event handlers for system-wide events.

OSGi is application model agnostic. It defines a component model, where components – bundles – have a lifecycle, but it doesn’t define a concrete application model. Rather, it can host a number of application model. Eclipse Foundation defined (embedded) Rich Client Platform, an application model based on OSGi bundles. JSR-232 defines the Foreign Application Model, which allows OSGi to host any existing application model. This is particularly interesting in the light of the next wave of mobile application fragmentation. The number of open mobile application platforms grows rapidly. A variety of different Linux systems (LiMO, Qt, etc.) and Java platforms (Android, JavaFx, CDC Java) add to the well known “oldies” like MIDP, S60 and Win Mobile. How could a carrier or an enterprise ever support such a fleet? OSGi is one answer – pick the application model you like and run it on any platform!

Manageability

One of the key requirements for mobile devices – especially in the smart-phone segment – is remote manageability (often also referred to as MDM, Mobile Device Management). MDM includes device configuration, application deployment and maintenance, status monitoring, data backup and replication services. While device complexity, use cases and volumes increase, the ability to remotely manage devices becomes vital for mobile operators and enterprises. JSR-232 defines a mobile service platform, which is designed from the ground up to support device management. As a result, a device administrator is in full control of all components running on the device, he can monitor and control all aspects of the device without requiring the application developer to deal with this complexity. Device management is not visible in Android. Presumably, it is left up to OEMs and operators to take care of. Though a platform should make MDM as transparent to developers as possible, still there are some aspects developers are concerned with:

• Versioning – it is a simple feature but it’s the key for manageability. Without versioning support, it is impossible to manage updates and dependencies.
• Management-related APIs – though some basic APIs (configuration, logging) are present, there are a number of APIs missing, like deployment, lifecycle management or monitoring.

How can enterprises update their Android devices with a newer application version?

Standards

An important difference between the two platforms is their grounding on standards. OSGi is a standard that has been developed for ten years and reached a high level of maturity. It has been adopted by a number of JSR in JCP and there are number of independent commercial and open source implementations, optimized for different fields of use. In addition, the OSGi Alliance has very strong backwards compatibility requirements.

Android is an open source initiative, where APIs are defined by the implementation itself, which is currently managed by a single company. Though this approach is a lot more flexible, the lack of a specification standard, the lack of backwards compatibility and the lack of acceptance tests bear significant fragmentation risks to the community. How does Android look like tomorrow?

Does Open Mean Undefined?

The biggest challenge of comparing Android to another platform is the large number of open questions. Android so far published the SDK for application developers, which focuses on the areas that Google thinks are of developers’ concern. As long as you want to develop simple end user applications, you probably have what you need. But if you happen to develop a complex application package, maybe an enterprise application or an application suite, where security, deployment, configuration and other management functions are important, you realize that all these are left open i.e. the platform doesn’t specify them and you have to build your application on assumptions. Cross-implementation compatibility of Android will be an interesting topic to observe.

Conclusion

The two platforms give very different solutions for the same question. Process-based isolation model in Android is an advantage in resource management on the other hand the lack of class-sharing ability seriously limits the sharing of components and makes platform management very difficult. The lack of versioning and lack of management APIs is a further limitation. Presumably, OEMs will provide some solutions for these important requirements; however those will be proprietary and will lead to fragmentation of the platform. The lack of standard and acceptance testing also increase the fragmentation risk. Though Android has some promising features, there seems to be a long way to go to reach maturity.

OSGi Specification as a book

I don't know how you feel, but books are special for me. A book, you can touch it, you can feel it and it smells like.... well, like a book. I know that the use of electronic media is more economic, easier to access and more environment friendly, but I just have to have my favorites in hard copy as well. If you are dealing with OSGi, the ultimate source of information is the Specification. As opposed to many sloppy specifications, OSGi specs are well-written, easy to read, with lots of explanations and pointers to additional material, they are more like a book than a usual spec. And finally, the Release 4 version of OSGi specification is available as a book. I really have to have a copy! The only sad thing is that the book contains the core platform specification only, the R4 mobile specification - a separate book - is not (yet?) available.