Xaval v0.8.1

• Improve css loading to avoid FOUC
• Add logo and favicon
• Add link to documentation (https://docs.xaval.app)
• Add link to issue tracker (https://github.com/habbes/xaval-issues)
• Minor UI tweaks

Xaval v0.8

Changes:

• Switched editors from Ace to Monaco
• Display Xaval version in the header instead of OpenCV.js status
• Integrated Headway in order to display updates/release notes in app
• Added basic intellisense support and code-completion. This does not yet cover the entire xaval API

Known issues:

• The app js bundles have become huge as a result of integrating Monaco (the main bundle is 1.6MB). I will work on reducing the bundle size in future updates.

Xaval v0.7

Xaval v0.7 Release

Here's an overview of what's new:

• support for importing and working with videos and other file types
• the app has a new domain: https://xaval.app
• other minor changes and bug fixes

Support for more file types

The file library now allows you to import videos and other file types from the local system and new APIs have been introduced to make it easy to work with them. The file library detects and sets the correct type of the file automatically when being imported. The library treats images, videos and other file types differently.

Video

Once you've imported a video file, you can use files.readVideo(name) to read the video. This returns a Video object, which is a wrapper around the underlying video. Like Camera, Video implements the VideoSource interface, which means it provides the same methods for reading frames (video.read) and obtaining a stream video.getStream(opts). But it also provides more methods and properties not available for camera objects:

const video = files.readVideo('scene');
const stream = video.getStream({ fps: 30 });
stream.pipe(imageViewer);
video.looping = true;
video.play();
setTimeout(() => video.pause(), 1000);
setTimeout(() => video.currentTime = 0, 2000);
setTimeout(() => video.play(), 3000);
setTimeout(() => video.stop(), 5000);

Like camera.stop(), video.stop() also stops and releases the attached stream.

Video API

Like Camera, Video has the following methods and properties from VideoSource:

• video.width
• video.height
• video.read()
• video.getStream(opts)

In addition, it implements the following:

Member	Descritpion
video.duration: number	gets the duration of the video in seconds
video.looping: boolean	gets or sets whether the video is looping
video.playing: boolean	gets whether or not video is playing
video.ended: boolean	gets whether or not video has ended
video.currentTime: number	gets or sets the current time of the video in seconds
video.play()	plays the video, if there's a created from the video with autoStart to true, it will be started as well
video.pause()	pauses the video. Also pauses the attached stream, if any, but does not release it. The video (and the stream) can be resumed again by calling video.play()
video.stop()	stops the video and releases the attached stream, if any. To get another stream, you must call video.getStream()

Note: the fps option passed to getStream() determines only the rate at which the stream will read a frame from the video, and not the rate at which the video plays.

Other file types

For files other than videos and images, the file library provides files.getReader(filename), which returns a BinaryFileReader. This is a wrapper around the file blob that allows you to asynchronously read the file contents either as text, array buffer or data url.

const file = files.getReader('somefile');
file.readText().then(text => console.log(text));
file.readBuffer().then(buffer => console.log(buffer));
file.readDataURL().then(dataUrl => console.log(dataUrl));

BinaryFileReader API

Member	Description
url: string	the object url of the file
reader.readText(): Promise<string>	reads the file contents as text
reader.readDataURL(): Promise<string>	reads the file contents as a data url
reader.readBuffer(): Promise<ArrayBuffer>	reads the file contents into an array buffer

Extended FileLibrary API

Here are new methods added to the FileLibrary API

Member	Description
files.addVideo(url: string, filename?: string)	adds a new video to the library from the specified source url. If filename is provided, it will be used as the name of the file in the library, an error will occur if the name already exists.
files.readVideo(filename: string): Video	gets the video from the library. Every time this is called, it returns the same video object, not a new one like how files.readImage() works.
files.addBinary(file: Blob, filename?: string)	adds a new file to the library (non-video and non-image).
files.getReader(filename: string): BinaryFileReader	gets a reader for the specified file

Minor changes

• widgets updates are not triggered if some inputs or params are not set (i.e. are undefined)
• app shows confirmation dialog when closing to prevent user from accidentally closing app and losing work
• the import button has been slightly tweaked
• google analytics integration added to get insights on app usage

Bug fixes

• files.rename() shows error when trying to rename a file that does not exist
• files.addImage() now correctly applies the custom name (if provided) to the file tile
• files.addImage() now checks for collisions and shows error if specified name already exists
• widgets controls are now updated when the param value is programmatically set using widget.setParam()
• fix bug causing slider control to display value 100 when the initial value was set to a number > 100

Xaval v0.6

Xaval v0.6 Release

This release contains a number of new features, improvements and breaking changes.

Here's a brief overview of what has changed:

• This release introduces camera support. You can now work with video streams from connected cameras via a simple API (xaval.io.cameras) and (camera.getStream())
• You can now import multiple files in your workspace and access them in code via the new xaval.io.files API
• Widgets now support 3 new additional parameter controls (checkbox, select list and text field)
• Widget ids are displayed on the widgets' headers
• You can now explore different example code from the samples menu
• Minor UI improvements including a new font family and slightly rounder buttons

Breaking changes

• xaval.io.imageSource has been removed. You should now use xaval.io.files.readImage(name) instead of xaval.io.imageSource.read()
• xaval.widgets.create(tplName) now returns the created widget instead of the widget id

Introduction camera support

One of the major highlights of this release is that Xaval finally allows you to work with cameras. It exposes xaval.io.cameras which is an instance of CameraManager. This allows you request connected cameras from the browser user: cameras.getDefault() returns the default camera with a muted video stream, and cameras.getWithConstraints(constraints) allows you to specify options.

The returned objects are instances of Camera. This is a wrapper that allows you to request permission to use the camera via camera.start(), which returns a promise. You can read the current frame from the camera using camera.read(). But most often you will want to get video a stream from the camera. For that you can use the camera.getStream({ fps }) method which returns a data source that you can pipe to a widget's input or other data sink.

Example:

const camera = xaval.io.cameras.getDefault();
const stream = camera.getStream({ fps: 30 });
stream.pipe(xaval.io.imageViewer);
camera.start();

Here are more detailed descriptions of the new APIs.

CameraManager API (xaval.io.cameras)

Member	Description
cameras.getDefault(): Camera	returns a camera instance representing the default camera from the browser, without audio capture
cameras.getWithConstraints(contraints: MediaStreamContraints): Camera	allows you to specify more fine-grained contraints. Refer to the MediaStreamContraints web API for available options.

Note, access to the camera is not requested until you call camera.start(), so there's not guarantee that the instance returned by the camera manager represents an available or accessible camera.

Camera API

Member	Description
camera.start(): Promise	tries to get the camera with the specified constraints. This also prompts the user for permission. If access is granted, the camera will start the video feed and the returned promise will resolve. From this point on you can call camera.read() to get the current frame.
camera.read(dest?: cv.Mat): cv.Mat	returns the current frame of the camera feed. If dest is provided, the image will be stored in that memory, otherwise the camera will store it elsewhere (in a recycled memory area, so a new one is only created when the current one is deleted)
camera.width: number	width of the video
camera.height: number	height of the video
camera.getStream(opts: {fps: number, autoStart?: boolean}): VideoStream	returns a video stream from the camera feed at a rate specified by the opts.fps option. If opts.autoStart is true (which is the default), the camera will start the stream automatically when the camera feed starts. However, if it's false, you will have to manually call stream.start() after camera.start() has resolved successfully. Furthermore, this method does not create a new stream if one is already connected to the camera, even if you pass different options. If you want a different stream (for example, with a different frame rate), you need to stop the camera (camera.stop()) and then call camera.getStream(), and don't forget to start the camera again.
camera.stop()	This stops the camera feed and disposes of the resources used by the camera. This also stops the stream obtained from getStream() if any, and disconnects it. You can start the camera again using camera.start() but you will need to get a new stream from camera.getStream(opts).

Camera implements the new VideoSource API, which exposes the read(dest?) and getStream(opts) methods, as well as the width and height properties.

VideoStream API

Member	Description
stream.streaming: boolean	gets whether or not its currently streaming data
stream.params: { fps: number, autoStart: boolean }	returns the options used to create the stream
stream.start()	starts streaming data, before this is called, not data is transmitted to observers
stream.stop()	stops streaming data. To start streaming data again call stream.start()

VideoStream implements the DataSource interface and therefore exposes the following members:

• stream.observable
• stream.subscribe(observer: Observer): Subscription
• stream.pipe(dest: DataSink): DataSink

New file library

With the new file library, you can now import multiple images (more file types coming soon). The new file library is located at the same position as the old image source. Now you will see an Import button that allows you to select images to import. You can select and import multiple files at once. Once imported, the thumbnails are displayed on a horizontal list along with their file names. You can edit a file's name by simple clicking it and editing it.

You can also manage and access the files programmatically via the new FileLibrary API, that xaval.io.files exposes.

FileLibrary API (xaval.io.files)

Member	Description
files.addImage(url: string, name?: string)	adds a new image to the library, where url is the image's source. If name is not provided, a new one is generated (file1, then file2, etc.). Warning: if the image is from the web, it will still be added, but you may get errors when working with the image in Xaval due to cross-origin restrictions in the browser. This is a known issue that be addressed in the future.
files.readImage(name: string): cv.Mat	returns the image with the specified name, undefined is returned if the name is not in the library. You will get unexpected results if the file exists but is not an image.
rename(oldName: string, newName: string)	renames a file. You will get an error alert if a file with the target name already exists

Widgets improvements

Widget id displayed in the header

The widget id is now displayed on the right end the widget header, prepended with a # character. For example if #widget1 is displayed, the real id is actually widget1.

WidgetManager#create(name: string) returns the widget

It was counter-intuitive for widgets.create() to return the widget id. This was done because there was no easy way to tell what the id was, but is now no longer necessary.

So instead of:

const id = widgets.create('MyWidget');
const widget = widgets.get(id);

You should now do:

const widget = widgets.create('MyWidget');

New param data types and controls

Up to now only the slider control was available. Now Xaval adds checkboxes, select lists and text fields, as well as support for boolean param data types.

checkbox control

This is the default control for boolean widget params.

text control

This renders a text field. It's the default control for string widget params.

select control

This is a single-select list. It supports multiple data types, namely numbers and strings. It requires an options option in the param config, which is an object mapping possible values to their labels. The labels are what are displayed on the list, but its the values that are used internally. If the param type is 'number', then the values will be automatically converted to numbers. Furthermore, if the an initial option is provided, it should be of the correct type.

Minor changes

• A Samples menu has been added to the main header, allowing the user to check out different examples of what you can do in Xaval
• The app now starts with an empty editor that only contains a few comments and link to OpenCV.js tutorials
• The editor indentation size has been reduced to spaces to allow for more writing width
• The font family has been changed to the system's default ui font
• Buttons were given slightly rounder edges

Xaval v0.5

Xaval v0.5 Release

Xaval library reorganized

The Xaval "standard" library has been reorganised around a root xaval module.

The image source and image viewer are now accessible under the io submodule: xaval.io.imageViewer and xaval.io.imageSource and the widget manager is directly accessible as xaval.widgets.

The entire module looks like this:

Reference	Description
xaval.cv	OpenCv module
xaval.widgets	widget manager
xaval.io.imageViewer	image viewer
xaval.io.imageSource	image source
xaval.core.widget.createWidgetTemplate	widget template factory function

Note: While cv is part of the xaval module, it's also still accessible from the global scope.

Note: This organisation might change in future releases as I'm still thinking of better ways to go about it.

WidgetManager's new API

Changes were made to the WidgetManager to make it more useful in managing widgets on the screen. Here's the new API:

Method	Description
widgets.define(name, opts)	creates and registers a new template. Returns the created template
widgets.setTemplate(name, tpl)	registers a widget template
widgets.getTemplate(name)	gets the template registered with the specified name
widgets.create(name)	creates a new widget from the specified template and adds the template to manager. Returns the widget's id.
widgets.add(widget)	Adds a widget instance to the manager and returns the widget's assigned id. Displays the widget on screen.
widgets.get(id)	gets a widget by its assigned id
widgets.hide(id)	hides widget with the specified id
widgets.hideAll()	hides all widgets in the manager
widgets.show(id)	displays widget with the specified id
widgets.showAll()	shows all widgets in the manager
widgets.remove(id)	remove widget with specified id (also removes from DOM)
widgets.removeAll()	removes all widgets in the manager

ImageViewer is now a DataSink

The ImageViewer implements the DataSink interface, this basically means that it has a next(value) method and it can there for be piped to a DataSource or used as an Observer.
If the value passed to ImageViewer.next(value) is cv.Mat object, it will be deleted to clean up memory. This assumes that the image viewer is used as the last node in a pipeline.

Improved widgets I/O

Multiple outputs

Widgets now support multiple outputs. Ouputs now have to be declared similar to how inputs are declared. And consequently the onUpdate callback must return an object which a key for each output:

widgets.define('Example', {
   // ... other opts
   outputs: ['output1', 'output2'],
   onUpdate (ctx) {
      // do stuff ...
      return {
         output1: something,
         output2: somthingElse
     }
}

Widgets data sources and sinks for pipeline support

Widgets inputs and outputs now expose DataSinks and DataSources respectively. You can pipe some source or observable to a widget's input and pipe a widget's output to some destination data sink or observer.

For example:

someSource.pipe(widget.inputs.image);
widget.outputs.image.pipe(imageViewer);

To achieve this, there's widget.inputs object that map inputs to objects implementing the next(value) method allowing them to be used as data sinks or observers. Similarly widget.outputs maps outputs to data sources, implementing pipe(dest), subscribe(observer) and exposing an observable property.

The following table summarises the changes and additions made to the WidgetModel's API:

Method/Property	Description
widget.observable	now emits the entire outputs object returned from onUpdate(ctx)
widget.pipe(dest)	pipes the outputs object to the destination whenever an update occurs. Returns the dest object to enable chaining: widget.pipe(dest).pipe(otherDest)
widget.subscribe(observer)	shortcut for widget.observable.subscribe(observer)
widget.inputs[name]	DataSink attached to the specified input
widget.inputs[name].next(value)	pushes the value to the specified input (and updates the widget)
widget.outputs[name]	DataSource attached to the specified output
widget.outputs[name].pipe(dest)	pipes the specified output to the specified DataSink. Returns the dest object.
widget.outputs[name].observable	observable attached to the output
widget.outputs[name].subscribe(observer)	shortcut for widget.outputs[name].observable.subscribe(observer)

Xaval v0.4

Introducing Widgets

Support for widgets has finally been implemented, albeit not yet fully-featured. By widgets I am referring to UI nodes that represent some computation and allow the user to tune parameters triggering the computation in real time. The result is that you're able to tweak slider values and see their effect on the output image immediately.

A new object widgets has been added to the Xaval coding environment, it is an instance of WidgetManager, which allows you to define widget templates and add widgets on screen. The first step is to define a WidgetTemplate via the widgets.define(name, opts) method. A WidgetTemplate specifies which parameters are available to tweak and inputs are expected, the actual computation that will be performed by the widget is defined in the onUpdate(ctx) function. The ctx parameter contains data about the widget's current state, like current parameter values and inputs (ctx.params, ctx.inputs etc.).

Widget instances are create using the WidgetTemplate.create() method. An instance represents an actual widget. You can create multiple widgets from the same template, while they would effectively run the same computation, they are actually independent, so you can pass different inputs to different widgets and set their parameters independently.

Finally, to display a widget use the widgets.show(widget) method. You can drag the widget around the screen. The drag-and-drop is a bit clunky though.

Widgets expose an observable (widget.observable) that can be used to get notified whenever its output has been updated, this can be used to sync the widget output with the image viewer in real time.

Limitations

There's still a lot of work to be done as far as widgets are concerned, these will be implemented in future updates:

• No input or parameter data validation
• More controls should be supported (only sliders are supported at the moment)
• There's only one output per widget
• A simpler API for connecting a widget to input sources/output destinations, akin to piping streams
• Add missing functionality to WidgetManager (e.g. retrieving a template by name or widget by id, hiding and removing widgets, etc.)
• A GUI alternative for managing widgets (defining templates, creating widgets, etc.)
• A predefined library of useful widget templates or a way to save templates for future use
• and probably more ...

UI Changes

I made some noticeable UI changes as well. The code editor now has more real estate after the editor header was removed. The "Run" button was moved to the main header. The main header now also has a lighter background. I changed the font from Roboto to Montserrat. And finally, I made minor improvements to the image import to prevent showing an image with a broken link by default. The file input was also removed from view and now it's triggered by clicking on the image.

Xaval v0.3

The most visible change in this release is the shift from the dark ui to a lighter theme.

Most of the work went into refactoring and restructuring the project from a single js file codebase to a modular project based on Typescript and Webpack.

Xaval v0.2.1

This patch fixed the image scaling issue. Now imagesimsource.read() loads images respecting their original size.

Xaval v0.2

This release is the first prototype to give an idea of what Xaval is meant to be.

It provides a code editor (via Ace) environment with access to the OpenCV cv object as well as imsource and imviewer for image I/O. It has basic layout and styling.

The imsource can only load one image at a time at the moment, using imsource.read().

There's a known bug in the image loading and display: the image viewer only shows images in the same size as the thumbnail in the image source (100px height).

Xaval v0.1.1

In this release the 404 error that occurred when attempting to load OpenCV.js on the web page was fixed.

Now the image loaded is displayed by OpenCV in the canvas below the original image.