OpenSeadragon 6.0.0

Data Modification Pipeline

Any plugin that modifies viewer data should use the tile-invalidated event, which ensures data is updated in a consistent manner. Optionally, instead of modifying the data, you can design a custom renderer - often a more powerful, performant, but also more complex solution. In this article, we will explore the easier option—modifying the data per tile.

The usage is simple: you request an invalidation once you reason it is needed to update the data, using

viewer.requestInvalidate()

and provide a handler for the data update

viewer.addHandler('tile-invalidated', async e => {...})

The invalidation event request requestInvalidate(restoreTiles) takes one optional argument restoreTiles (default true), which controls whether tile data will be reset in the event or not. If tile data is not reset, invalidation will continue from the previous state - this way, we can accumulate modifications, computing changes incrementally.

const viewer = OpenSeadragon({
    id:            "example-simple",
    prefixUrl:     "/openseadragon/images/",
    navigatorSizeRatio: 0.25,
    tileSources:   "/example-images/highsmith/highsmith.dzi",
});
viewer.addHandler('tile-invalidated', async e => {
    // Get the canvas and context
    const ctx = await e.getData('context2d');
    const canvas = ctx.canvas;
    const imageData = ctx.getImageData(0, 0, canvas.width, canvas.height);
    const width = canvas.width;
    const height = canvas.height;
    

    function toGrayscale(data) {
        const grayData = new Uint8ClampedArray(data.length / 4);
        for (let i = 0; i < data.length; i += 4) {
            const avg = (data[i] + data[i + 1] + data[i + 2]) / 3;
            grayData[i / 4] = avg;
        }
        return grayData;
    }

    function sobelEdgeDetection(grayData, width, height) {
        const sobelX = [[-1, 0, 1], [-2, 0, 2], [-1, 0, 1]];
        const sobelY = [[-1, -2, -1], [ 0, 0, 0], [ 1, 2, 1]];

        const gradient = new Uint8ClampedArray(grayData.length);

        for (let y = 1; y < height - 1; y++) {
            for (let x = 1; x < width - 1; x++) {
                let gx = 0, gy = 0;

                // Apply Sobel kernels
                for (let ky = -1; ky <= 1; ky++) {
                    for (let kx = -1; kx <= 1; kx++) {
                        const pixelVal = grayData[(y + ky) * width + (x + kx)];
                        gx += pixelVal * sobelX[ky + 1][kx + 1];
                        gy += pixelVal * sobelY[ky + 1][kx + 1];
                    }
                }

                // Calculate gradient magnitude
                const magnitude = Math.sqrt(gx * gx + gy * gy);
                gradient[y * width + x] = magnitude > 255 ? 255 : magnitude;
            }
        }
        return gradient;
    }

    function drawEdges(ctx, gradient, width, height) {
        const outputImage = ctx.createImageData(width, height);
        for (let i = 0; i < gradient.length; i++) {
            const value = gradient[i];
            outputImage.data[i * 4] = value;
            outputImage.data[i * 4 + 1] = value;
            outputImage.data[i * 4 + 2] = value;
            outputImage.data[i * 4 + 3] = 255;
        }
        ctx.putImageData(outputImage, 0, 0);
    }
    

    const grayData = toGrayscale(imageData.data);
    const edges = sobelEdgeDetection(grayData, width, height);
    drawEdges(ctx, edges, width, height);
    await e.setData(ctx, "context2d");
});
    

By default, invalidation events are broadcasted to all viewers in the parent viewer. You can change this behavior by setting a drawer option broadCastTileInvalidation to false. This will then force you to call invalidation on each viewer separately—for example, to the base viewer and also viewer.navigator instances, which are both Viewer subclasses. Another sub-viewers example includes reference strips.

If you develop a plugin, you should make the order of your event handlers configurable, so that users can control when your plugin is applied.

const eventOrderPriority = options.eventOrderPriority;
viewer.addHandler('tile-invalidated', async e => {
    ... do something ...
}, null, eventOrderPriority);

This way, your plugin gets applied based on eventOrderPriority and thus users can control which plugin gets applied first. See handler priority argument.

Your plugin can request whether to start from the original data or whether to update the current viewport state. Using viewer.requestInvalidate(false) makes your code receive the data on the viewport, so that you can stack changes. You can also request a data reset at any point in the event routine using resetData():

viewer.addHandler('tile-invalidated', async e => {
    e.resetData(); //throw away any possible previous changes
    const freshData = await e.getData('image');
    ... etc ...
}, null, eventOrderPriority);

But beware of data consistency; see section below.

It might happen that you request invalidation while a different event is running. You can tell simply by using:

await viewer.requestInvalidate();  // await finishing

So if you have e.g. a range slider for user-controlled input, you might trigger too many updates at once. You can exit outdated (no longer valid) processes by checking e.outdated() flag.

viewer.addHandler('tile-invalidated', async e => {
    ... some expensive computation ...
    if (e.outdated()) return;
    ... etc ...
}, null, eventOrderPriority);

Note though, with requestInvalidate(false) some tiles might finish before you realize the current data is no longer needed, and the viewport might show inconsistent state.

Removal of if (e.outdated()) return; will not help! The viewer also considers output of such events pointless and throws away any results. This scenario is not yet well-supported, and you must ensure no overlapping events are triggered in this specific case.

A good idea is to collect invalidation calls and truly call requestInvalidate() only once, for example when you know the previous update was finished.

With knowledge of what drawers are being used you can optimize your events. getData() might force unnecessary processing, setData() might force system to do additional type conversions. A good idea is to work with the context2d type, which is flexible enough for processing, but also provides image data through a canvas reference, directly usable in most drawers.

The update routine keeps a working cache bound to the event. This cache is not part of the system; it is a copy of data from the tile. Once the data is ready, we atomically swap the data under the renderer, and the new data gets displayed. Therefore, when you finish there is one last step in the event where we ask the underlying drawer what type it requires (getRequiredDataFormats()) and potentially convert the data.

Some drawers might not be compatible with the type directly, but want to keep it nevertheless. The WebGL renderer uses an internal cache that keeps reference to the uploaded texture, but the cache from outside looks like a context2d or image cache (but with a weird type name). This way we keep a reference to the data shown on screen both on CPU and GPU, and support conversion back to, for example, a canvas context 2D object.

While you can add custom data to the tile using cache-related methods (get/set/create), it is probably not a good idea. Cache invalidation runs on every data-unique tile, both in the viewer and navigator. It might not be executed though on every tile.

No. But all data items are—eventually. To speed things up, only tiles that need to be updated are truly updated. The rest is kept in 'not-ready' state until we realize we need them. You can look at it as if we were re-downloading the tile again, but this time, the data waits for us on the localhost. Also, tile equality is decided using getTileCacheKey(). If this item returns an equal value for two tiles, invalidation is executed only on one of the two, and references to the data are later updated on both.

So if you have the same data but want to apply different processing (e.g. vignetting), you must also make sure these tiles have different (original) cache keys. This will force you to download the same data several times. However, this scenario is not expected to be frequent, and thus it is not optionally supported.

If you happen to attempt implementing drawer logic, you will usually provide custom data converters for the target type you require for rendering.

If you need the system to re-convert every data item (e.g. you upload textures as WebGL renderer and parameters have changed such as texture wrap or filter), you can also use this.viewer.requestInvalidate(). Note though, that you should also change your type name, since the system might not do anything if it realizes there is no data to update and types are fitting.

So you should change your type names and update the return value of getRequiredDataFormats(). Furthermore, you should keep for the time of being old types in getSupportedDataFormats(), and optionally remove them once processing done:

    ...
    const newFormats = this.generateNewTypeNamesAndImplementConverterAndDestructorLogic();
    this._supportedFormats.push(...newFormats);  // variable returned by getSupportedDataFormats
    this._requiredFormats = newFormats;          // variable returned by getRequiredDataFormats
    await this.viewer.requestInvalidate();       // forces re-execution since type has changed
    // optionally throw away old type names after invalidation:
    // this._supportedFormats = newFormats;
    ...

• Examples & Features
• Plugins
• Browser Extension
• Download & Install
• API Documentation
• Support
• Development
• License