Many embedded systems require a UI for interaction with the application. For complex applications, or for maximum ease of use, a graphical windowing system is a natural choice. However, the windowing systems on desktop PCs simply require too much in the way of system resources to be practical in an embedded system where memory and cost are limited.

Drawing upon the successful approach of the QNX® microkernel architecture to achieve a POSIX OS environment for embedded systems, we have followed a similar course in creating the Photon microGUI windowing system.

To successfully implement a microkernel OS, we first had to tune the microkernel so that the kernel calls for IPC were as lean and efficient as possible (since the performance of the whole OS rests on this message-based IPC). Having implemented this low-overhead IPC, we were able to structure a GUI as a graphical "microkernel" process with a team of cooperating processes around it, communicating via that IPC.

While at first glance this might seem similar to building a graphical system around the classic client/server paradigm already used by the X Window System, the Photon architecture differentiates itself by restricting the functionality implemented within the graphical microkernel (or server) itself and distributing the bulk of the GUI functionality among the cooperating processes.

The Photon microkernel runs as a tiny process (45K of code), implementing only a few fundamental primitives that external, optional processes use to construct the higher-level functionality of a windowing system. Ironically, for the Photon microkernel itself, "windows" do not exist. Nor can the Photon microkernel "draw" anything, or manage a pen, mouse, or keyboard.

To manage a GUI environment, Photon creates a 3- dimensional "event space" and confines itself only to handling regions and processing the clipping and steering of various events as they flow through the regions in this event space.

This abstraction is roughly parallel to the concept of a microkernel OS being incapable of filesystem or device I/O, but relying instead on external processes to provide these high-level services. Just as this allows a microkernel OS to scale up or down in size and functionality, so also a microkernel GUI.

The core microkernel "abstraction" implemented by the Photon microkernel is that of an abstract, graphical event space that other processes can populate with regions. Using QNX® IPC to communicate with the Photon microkernel, these other processes manipulate their regions to provide higher-level graphical services or to act as user applications. By removing service-providing processes, Photon can be scaled down for limited-resource systems; by adding service-providing processes, Photon can be scaled up to full desktop functionality.

The "event space" managed by the Photon microkernel can be visualized as an empty void with a "root region" at the back. The end-user can be imagined to be "looking into" this event space. Applications place "regions" into the 3-dimensional space between the root region and the end-user; they use those regions to generate and accept various types of "events" within this space.

Processes that provide device driver services place regions at the front of the event space. In addition to managing the event space and root region, the Photon microkernel maintains an on-screen pointer, projected as draw events towards the user.

Photon uses a series of regions, ranging from the Root region at the back of the Photon event space to the Graphics region at the front. Draw events start at an application's region and move forward to the Graphics region. Input events start at the Pointer/Keyboard region and travel towards the Root region.

We can think of these events that travel through this space as "photons" (from which this windowing system gets its name). Events themselves consist of a list of rectangles with some attached data. As these events flow through the event space, their rectangle lists intersect the "regions" placed there by various processes (applications).

Events traveling away from the root region of the event space are said to be traveling outwards (or towards the user), while events from the user are said to be traveling inwards towards the root region at the back of the event space.

The interaction between events and regions is the basis for the input and output facilities in Photon. Pen, mouse, and keyboard events travel away from the user towards the root plane, with the location of the on-screen pointer associated with them. Draw events originate from regions and travel towards the device plane and the user.

Regions

Regions are managed in a hierarchy associated as a family of rectangles that define their location in the 3-dimensional event space. A region also has attributes that define how it interacts with various classes of events as they intersect the region. The interactions a region can have with events are defined by two bitmasks:

• sensitivity bitmask
• opaque bitmask.

The sensitivity bitmask uses specific event types to define which intersections the process owning the region wishes to be informed of. A bit in the sensitivity bitmask defines whether or not the region is sensitive to each event type. When an event intersects a region for which the bit is set, a copy of that event is enqueued to the application process that owns the region, notifying the application of events traveling through the region. This notification doesn't modify the event in any way.

The opaque bitmask is used to define which events the region is opaque to. For each event type, a bit in the opaque mask defines whether or not the region is opaque or transparent to that event. The optical property of "opaqueness" is implemented by modifying the event itself as it passes through the intersection.

These two bitmasks can be combined to accomplish a variety of effects in the event space. The four possible combinations are:

Events

Like regions, events also come in various classes and have various attributes. An event is defined by:

• an originating region
• a type
• a direction
• an attached list of rectangles
• some event-specific data (optional).

Unlike most windowing systems, Photon classifies both input (pen, mouse, keyboard, etc.) and output (drawing requests) as events. Events can be generated either from the regions that processes have placed in the event space or by the Photon microkernel itself. Event types are defined for:

• keystrokes
• pen and mouse button actions
• pen and mouse motion
• region boundary crossings
• expose and covered events
• draw events
• drag events.

Application processes can either poll for these events, block and wait for them to occur, or be asynchronously notified of a pending event.

The rectangle list attached to the event can describe one or more rectangular regions, or it can be a "point-source" - a single rectangle where the upper-left corner is the same as the lower-right corner.

When an event intersects a region that is opaque to it, that region's rectangle is "clipped out" of the event's list of rectangles such that the list describes only the portion of the event that would ultimately be visible.

The best way to illustrate how this clipping is performed is to examine the changes in the rectangle list of a draw event as it passes through various intersecting regions. When the draw event is first generated, the rectangle list consists of only a single, simple rectangle describing the region that the event originated from.

If the event goes through a region that clips the upper-left corner out of the draw event, the rectangle list is modified to contain only the two rectangles that would define the area remaining to be drawn. These resulting rectangles are called "tiles."

Likewise, every time the draw event intersects a region opaque to draw events, the rectangle list will be modified to represent what will remain visible of the draw event after the opaque region has been "clipped out." Ultimately, when the draw event arrives at a graphics driver ready to be drawn, the rectangle list will precisely define only the portion of the draw event that is to be visible.

Regions opaque to draw events are clipped out, resulting in an area consisting of rectangular tiles.

If the event is entirely clipped by the intersection of a region, the draw event will cease to exist. This mechanism of "opaque" windows modifying the rectangle list of a draw event is how draw events from an underlying region (and its attached process) are properly clipped for display as they travel towards the user.

Graphics drivers are implemented as processes that place a region in front of the event space. Rather than inject pen, mouse, or keyboard events, a graphics driver's region is sensitive to draw events coming out of the event space. As draw events intersect the region, those events are received by the graphics driver process. In effect, the region can be imagined to be coated in "phosphor," which is illuminated by the impact of "photons."

Since the Photon drawing API accumulates draw requests into batches emitted as single draw events, each draw event received by the driver contains a list of individual graphical primitives to be rendered. By the time the draw event intersects the graphics driver region, its rectangle list will also contain a "clip list" describing exactly which portions of the draw list are to be rendered to the display. The driver's job is to transform this clipped draw list into a visual representation on whatever graphics hardware the driver is controlling.

One advantage of delivering a "clip list" within the event passed to the driver is that each draw request then represents a significant "batch" of work. As graphics hardware advances, more and more of this "batch" of work can be pushed directly into the graphics hardware. Many display controller chips already handle a single clip rectangle; some handle multiple clip rectangles.

Although using the QNX® IPC services to pass draw requests from the application to the graphics driver may appear to be an unacceptable overhead, our performance measurements demonstrate that this implementation performs as well as having the application make direct calls into a graphics driver. One reason for such performance is that multiple draw calls are batched with the event mechanism, allowing the already minimal overhead of the QNX® lightweight IPC to be amortized over many draw calls.

Multiple graphics drivers

Since graphics drivers simply put a region into the Photon event space, and since that region describes a rectangle to be intersected by draw events, it naturally follows that multiple graphics drivers can be started for multiple graphics controller cards, all with their draw-event-sensitive regions present in the same event space.

These multiple regions could be placed adjacent to each other, describing an array of "drawable" tiles, or overlapped in various ways. Since Photon inherits QNX®'s network transparency, Photon applications or drivers can run on any network node, allowing additional graphics drivers to extend the graphical space of Photon to the physical displays of many networked computers. By having the graphics driver regions overlap, the draw events can be replicated onto multiple display screens.

Many interesting applications become possible with this capability. For example, a factory operator with a wireless-LAN handheld computer could walk up to a workstation and drag a window from a plant control screen onto the handheld, and then walk out onto the plant floor and interact with the control system.

In other environments, an embedded system without a UI could project a display onto any network-connected computer. This connectivity also enables useful collaborative modes of work for people using computers - a group of people could simultaneously see the same application screen and cooperatively operate the application.

From the application's perspective, this looks like a single unified graphical space. From the user's perspective, this looks like a seamlessly connected set of computers, where windows can be dragged from physical screen to physical screen across network links.

Color model

Colors processed by the graphics drivers are defined by a 24-bit RGB quantity (8 bits for each of red, green, and blue), providing a total range of 16,777,216 colors. Depending on the actual display hardware, the driver will either invoke the 24-bit color directly from the underlying hardware or use various dithering techniques to create the requested color from less-capable hardware.

Since the graphics drivers use a hardware-independent color representation, applications can be displayed without modifications on hardware that has varied color models. This allows applications to be "dragged" from screen to screen, without concern for what the underlying display hardware's color model might be.

In addition to full support of bitmap fonts, Photon also provides scalable fonts. These fonts can be scaled to virtually any point size as well as anti-aliased (16 shades) for crisp, clean output on screen at practically any resolution.

Photon's scalable fonts are achieved by means of a high-speed font server that retrieves highly compressed font data from Portable Font Resource files (*.pfr), and then renders the font character shapes at any point size and resolution. Note that PFRs provide better than 2-to-1 compression over PostScript fonts.

Font sets

Core Latin set

The Photon Core Latin set (latin1.pfr), which comprises the Unicode Basic Latin (U+0000 - U+007F) and the Latin-1 Supplement (U+0080 - U+00FF) character sets, includes the following scalable fonts:

• Dutch
• Dutch Bold
• Dutch Italic
• Dutch Bold Italic
• Swiss
• Swiss Bold
• Swiss Italic
• Swiss Bold Italic
• Courier
• Courier Bold
• Courier Italic
• Courier Bold Italic

Extended Latin set

The Photon Extended Latin set (latinx.pfr) comprises the Unicode Latin Extended-A (U+0100 - U+017F) and Latin Extended-B (U+0180 - U+0217) sets. The Extended Latin set includes these fonts:

• Dutch
• Dutch Bold
• Dutch Italic (algorithmically generated)
• Dutch Bold Italic (algorithmically generated)
• Swiss
• Swiss Bold
• Swiss Italic (algorithmically generated)
• Swiss Bold Italic (algorithmically generated)

Languages supported

Armed with the Photon Core Latin set, the developer can support a host of languages, including:

Danish
Dutch
English
Finnish
Flemish
French
German
Hawaiian
Icelandic
Indonesian
Irish
Italian
Norwegian
Portuguese
Spanish
Swahili
Swedish

By adding the Photon Extended Latin set (latinx.pfr), the developer can support several additional languages, including:

Afrikaans
Basque
Catalan
Croatian
Czech
Esperanto
Estonian
Greenlandic
Hungarian
Latvian
Lithuanian
Maltese
Polish
Romanian
Slovak
Turkish
Welsh

International language supplements

Several language supplement packages are available for Photon:

• Japanese Language Supplement - offers developers full support for Japanese characters.

  Besides a complete set of PFRs (Hon Mincho Kanji set), the package also includes a front-end processor that allows users to provide input using either an English or a Japanese keyboard (via the popular VJE input method used in Japan).

• Chinese Language Supplement - offers developers full support for Chinese characters as well as a corresponding input method.
• Korean Language Supplement - offers developers full support for Korean characters as well as a corresponding input method.
• Cyrillic Language Supplement - offers developers full support for Russian or any language that relies on the Cyrillic alphabet. This package also includes the appropriate keyboard drivers to support English or Cyrillic keyboards.

Photon is designed to handle international characters. Following the Unicode Standard (ISO/IEC 10646), Photon provides developers with the ability to create applications that can easily support the world's major languages and scripts.

Unicode is modeled on the ASCII character set, but uses a 16-bit encoding to support full multilingual text. There's no need for escape sequences or control codes when specifying any character in any language. Note that Unicode encoding conveniently treats all characters - whether alphabetic, ideographs, or symbols - in exactly the same way.

UTF-8 encoding

Formerly known as UTF-2, the UTF-8 (for "8-bit form") transformation format is designed to address the use of Unicode character data in 8-bit UNIX environments.

Here are some of the main features of UTF-8:

• Unicode characters from U+0000 to U+007E (ASCII set) map to UTF-8 bytes 00 to 7E (ASCII values).
• ASCII values don't otherwise occur in a UTF-8 transformation, giving complete compatibility with historical filesystems that parse for ASCII bytes.
• UTF-8 simplifies conversions to and from Unicode text.
• The first byte indicates the number of bytes to follow in a multibyte sequence, allowing for efficient forward parsing.
• Finding the start of a character from an arbitrary location in a byte stream is efficient, because you need to search at most four bytes backwards to find an easily recognizable initial byte. For example:

  isInitialByte = ((byte & 0xC0) != 0x80);

• UTF-8 is reasonably compact in terms of the number of bytes used for encoding.

The OS library includes a convenient set of conversion functions:

In addition to the conversion functions listed above, developers can also rely on Photon's own PxTranslate library functions, which will translate various character set encodings to and from UTF-8.

Photon provides flicker-free animations via a special "double-buffer" container widget (PtDBContainer) that creates a dedicated off-screen memory context for drawing images.

The PtDBContainer widget uses a block of shared memory large enough to hold an image the size of its canvas, and will also dynamically load the render shared library.

Photon provides built-in printing support for a variety of outputs, including:

• bitmap files
• PostScript
• Hewlett-Packard PCL
• Epson ESC/P2
• Canon
• Lexmark

Photon also comes with a print-selection widget/convenience dialog to make printing simpler within developers' own applications.

Adding a window manager to Photon creates a fully functional desktop-class GUI. The window manager is entirely optional and can be omitted for most classes of embedded systems. If present, the window manager allows the user to manipulate application windows by resizing, moving, and iconifying them.

The window manager is built on the concept of filtering events with additional regions placed behind application regions, upon which a title bar, resize handles, and other "gadgets" are drawn and interacted with. Since the replaceable window manager implements the actual "look and feel" of the environment, various UI flavors can be optionally installed.

Photon provides a library of components known as widgets - objects that can manage much of their on-screen behavior automatically, without explicit programming effort. As a result, a complete application can be quickly assembled by combining widgets in various ways and then attaching C code to the appropriate callbacks the widgets provide. The Photon Application Builder (PhAB), which is included as part of the Photon development system, provides an extensive widget palette in its visual development environment.

Photon provides a wide range of widgets:

• basic widgets (e.g. a button)
• container widgets (e.g. a window widget)
• advanced widgets (e.g. an HTML display widget).

Basic widgets

Label widget (PtLabel)

The label widget is mainly used to display textual information. The PtLabel widget is the superclass for all text-based widgets, providing many customizable attributes (e.g. font typeface, pop-up balloons, colors, borders, alignment, margins, etc.), all of which are inherited by all its subclasses.

Push button widget (PtButton)

Push buttons are a necessary component in every windowing system. They have a raised look that changes to depressed when pushed, giving a visual cue to let the user know the button has been selected. In addition to this visual behavior, push buttons automatically invoke an application-defined callback when they're selected.

Text input widgets (PtText, PtMultiText)

Photon provides two text-input widgets:

• a simple single-line input widget (PtText) commonly used in forms
• a powerful wordprocessor-like multi-line widget (PtMultiText) providing full editing capabilities, word wrapping, automatic scrolling, and multi-font line segments.

Toggle button widgets (PtToggleButton, PtOnOffButton)

Toggle buttons are objects that display two states - either on or off. Photon provides two different types of toggle buttons, each with a different visual appearance. Toggle buttons are used to display or request state information related to a command or action about to be performed.

Graphical widgets (PtArc, PtPixel, PtRectangle, PtLine, PtPolygon, PtEllipse, PtBezier, PtGrid)

Photon has no shortage of graphical widgets. There's a widget to accomplish everything from simple lines and rectangles to complex multi-segmented bézier curves. Graphical widgets provide attributes for color, fills, patterns, line thickness, joins, and much more.

Scrollbar widget (PtScrollbar)

A scrollbar widget is used to scroll the display of a viewable area. The scrollbar is combined with other widgets (e.g. PtList, PtScrollArea) to allow scrolling.

Separator widget (PtSeparator)

The separator widget is used to separate two or more different areas, giving a better visual appearance. The separator can be customized for many different styles and looks.

Slider widget (PtSlider)

Sliders are different from scrollbars. A scrollbar defines a range, whereas a slider defines a single value. The slider widget provides a rich list of customizable attributes.

Timer widget (PtTimer)

The timer widget makes using timers a snap. This widget has no visual appearance - it simply defines a callback whenever a timer event is triggered. The application sets the timer value and optional repeat value. When the timer goes off, the application is notified.

Graphic image widgets (PtBitmap, PtLabel, PtButton)

Photon supports every major graphic file standard, so you can import graphics and display them inside widgets. Many Photon widgets directly support displaying graphics - the most common are PtButton for making push-button toolbars and PtLabel for displaying images.

Progress bar widget (RtProgress)

If an application needs to do something that takes a fair amount of time (e.g. loading a file), it can use the progress bar widget to let the user know what's happening and, more importantly, how much longer the process is going to take. The progress bar can be horizontal or vertical and has many attributes for customization.

Message widget (PtMessage)

Pop-up messages and notifications are quite common in a windowing environment. Photon provides a very handy message dialog widget that displays a message and up to 3 user-response buttons. There's also a very useful modal dialog function call (PtAskQuestion()) based on the message widget.

Numeric widgets (PtNumericInteger, PtNumericFloat)

The PtNumericInteger class lets the user specify integer values between given minimum and maximum values. The PtNumericFloat class lets the user enter floating-point values.

A PtUpDown widget is also shown to allow the user to increase or decrease the value by a set amount.

Container widgets

Window and icon widgets (PtWindow, PtIcon)

Windows are the main application containers. The main UI components (menu bars, toolbars, etc.) appear with the window widget. The widget automatically handles all the necessary interactions with the Photon Window Manager (PWM) - all you need to specify is what should and shouldn't be rendered or managed.

Icon widgets are closely associated with windows and are displayed in the Photon Desktop Manager launch folders and PWM taskbar.

Bulletin board widgets (PtPane)

Bulletin board widgets are simple container widgets that are used to hold other widgets. Although these are parent widgets, they don't manage their child widgets in any way. They are quite useful for designing form layouts commonly found in dialog windows.

Group widget (PtGroup)

The group widget is a very powerful widget that manages the geometry of all its child widgets. It can align the widgets horizontally, vertically, or in a matrix. The group widget can be anchored to the side of any other container (like a window) so that it automatically resizes when the window resizes. The group widget also provides attributes that let you specify whether the children should be stretched to fit the group if it's resized larger due to anchoring.

Scrolling area widget (PtScrollArea)

The scrolling area widget is a very powerful widget that provides a viewport into a potentially larger container. You can place any number of widgets inside a scrolling area and it will automatically display a scrollbar if the widgets are contained within the viewable area. Scroll area widgets could be used to implement a text file viewer, wordprocessor, customized list display, and so on.

To scroll child widgets quickly, the scrolling area widget uses a hardware blitter (provided the underlying graphics driver supports it).

Background widget (PtBkgd)

The background widget provides a way to create fancy background displays, from simple color gradations to tiled textures. Just about any background requirement is handled by this widget.

Advanced widgets

Menu related widgets (PtMenu, PtMenuBar, PtMenuButton)

Photon provides for every menu related requirement. There's a widget to simplify the creation of a standard menu bar. The menu widget handles the pop-up display, press-drag-release, point and click, keyboard traversal, and selection of menu items. The menu button widget is used for creating individual menu items.

List widget (PtList)

The list widget is a very powerful widget that manages a list of items. It provides many different selection modes, including single selection, multiple selection and range selection. The list widget also supports multi-columned lists through the use of a divider widget (PtDivider).

Pulldown list widget (PtComboBox)

The pulldown list widget combines the PtText widget (for text input) with a pulldown button for displaying a list widget. When the user selects from the list, the text widget is automatically updated with the current selection. The pulldown list widget is very useful for displaying a list of items using a small space. Dialogs and containers use a lot less screen real-estate, which is important in embedded environments.

Tree widget (PtTree)

The tree widget is similar to the list widget - in fact they both have the same ancestors. The main difference is that the tree widget displays the items in a hierarchical manner. Items, called branches, can be expanded or collapsed; any number of tree branches can be created. Each branch can define its own unique image to display. Trees are useful because they display information in a very logical manner. Photon applications that use the tree widget include: the File Manager (directory display), PhAB (widget layout), vsin (process list), and many others.

Terminal widgets (PtTty, PtTerminal)

Imagine having a text console inside your application. That's exactly what this widget does. It creates and manages an entire text-mode terminal inside a widget. Just drop it into your application and you've created your very own pterm (our terminal application).

The terminal widget doesn't stop there - it also provides complete cut-and-paste functionality and quick-launch help by highlighting any text within the widget.

Divider widget (PtDivider)

This powerful widget manages its children in a unique and useful way. When you place two or more widgets inside a divider widget, it automatically puts little separators in between the child widgets. Using these separators, the user can drag back and forth, causing the child widgets on either side of the separator to be resized. This is very useful for creating resizable column headings for lists. In fact, if you drop a divider widget into a list widget, it will automatically turn your simple list into a resizable multi-column list.

Dividers aren't limited to just labels or buttons. Any widgets can be placed inside to create side-by-side resizable trees, scroll areas, and so on.

Trend graph widget (RtTrend)

Realtime systems often require trend graphs. Photon comes with a trend bar widget that supports the display of multiple trend lines simultaneously. If your graphics hardware supports masked blits, it can even smooth-scroll the trend across grid lines.

Realtime meter widget (RtMeter)

The RtMeter widget is drawn as a half circle with divisional ticks at 1/3, 1/2, and 2/3 of the arc. The needle can be moved with the mouse, the keyboard, or programmatically. A single mouse click moves the meter to the current mouse position; a mouse "drag" causes the needle to follow the mouse through meter values.

Font selection dialog (PtFontSel)

To accommodate the wide selection of fonts available for Photon, a Font Selection widget is provided. This widget can read the standard font-mapping files and display a list of all available fonts. It allows you to choose the typeface, style (bold, italic, etc.) and also indicate whether the font should be anti-aliased.

File selection widget (PtFileSel)

The PtFileSel widget is a tree widget that displays files, directories, links to files or directories, or custom entries. Besides selecting a particular file in response to an application prompt, users can also use this widget to navigate an entire filesystem and choose their own file and directory.

Print selection dialog (PtPrintSel)

The PtPrintSel widget lets a user select a printer or control its properties. The user may also select a range of pages to print as well as the number of copies to submit.

HTML viewer widget (PtHtml)

Using the HTML widget makes it easy to create a customized help viewer. It will format a standard HTML file and even autoload all the images. It handles resizing, scrolling, just about everything you would need to do. This widget is ideal for creating helpviewers for touchscreen environments.

Widget building toolkit

If all the standard Photon widgets aren't enough, you can easily build your own! The Photon Development System comes with complete documentation and sample source code to create your own custom widgets. You can sub-class off of existing widgets to inherit their functionality or create a complete new widget class tree. Your widget possibilities are endless.

Photon represents a new approach to GUI building - using a microkernel and a team of cooperating processes, rather than the monolithic approach typified by other windowing systems. As a result, Photon exhibits a unique set of capabilities:

• Low memory requirements enable Photon to deliver a high level of windowing functionality to environments where only a graphics library might have been allowed within the memory constraints.
• Photon provides a very flexible, user-extensible architecture that allows developers to extend the GUI in directions unique to their applications.
• With flexible cross-platform connectivity, Photon applications can be used from virtually any connected desktop environment.

<< Previous | Index | Next >>