The following suggestions are made for components which take images, process them in some way, and produce images. Recognition components for example may not fit this framework and therefore work better as sink modules which strictly consume images. The current system is designed to work with real time video, therefore space and time are both concerns.

All components which take and produce images need to be subclasses of Filter. Filter type operators should produce images which make the results clear to a human user, but most of those operators are also needed for other programs which need the results in a more machine readable form. Therefore it might be a good idea to implement the operation as a separate class which produces machine readable results. A viewer class can then be developed which takes the results of the operation and converts them into an image for the user. The third class, the subclass of Filter, then would simply use these two classes to process the source images it receives. UML for this design is shown below.

Settings which the user will want to change need to reside in the OperationToFilterAdapter class. If the user needs to be able use mouse events to send information to the operator, then handleMouseEvent needs be used. Settings which are not likely to change during runtime (like color format) can be specified in key/value sets in a ModuleList type XML file. The setProperties method then needs to grab these settings from the properties parameter. A form (needs to subclass IEModuleForm) can be used to allow the user to change some settings while the component is running. This form will be provided with a FilterAdapter object when it is created, which provides access to its Filter.

The images used in this system are saved in BufferedImage objects. Processing an image using getRGB(x,y) for each pixel is independent of the underlying format used but very slow. A faster method is to get at the underlying data array by doing the following:

// Get the data
byte inBuffer[], outBuffer[];
inBuffer = ((DataBufferByte)sourceImage.getRaster().getDataBuffer()).getData();
outBuffer = ((DataBufferByte)rgbImage.getRaster().getDataBuffer()).getData();

// Get RGB from inBuffer[i]
bValue = (int)(inBuffer[i] & 0xFF);
gValue = (int)(inBuffer[i+1] & 0xFF);
rValue = (int)(inBuffer[i+2] & 0xFF);

This provides you with RGB values for each pixel i in the image as a one dimensional array.

Parent classes:
IEModule
FilterAdapter
Filter
IEModuleForm

A good example of what is needed is Segmenter which is a color segmenter which can handle up to 32 colors at a time.
The Segmenter itself: Segmenter
The Viewer class: SegmenterViewer
The Filter: ColorSegmentationOp
The Form: ColorSegmentationForm which is actually too elaborate for its own good but provides an example for interaction between the user and the Filter.