Fork me on GitHub

OpenSeadragon 4.1.0

Advanced Data Models with Custom TileSource

Since 2022, OpenSeadragon also supports custom tile data format, loading and caching procedures. This example goes through the process. First, please, make sure you are familiar with custom tile sources. The main information needed is right within the first paragraph, summarized here:

A Custom Tile Source can be created via inline configuration by specifying a single function named getTileUrl, along with the required values for height and width ... additionally, any default functions implemented by OpenSeadragon.TileSource can be overridden.

Advanced Data Models with OpenSeadragon

By default, tile data in OpenSeadragon is represented by an Image object. This, along with the process of tile loading and caching, can be further customized by overriding appropriate TileSource member functions.

Before jumping right to the examples, let's first understand how OpenSeadragon handles tile loading. When we mention overriding a default member method, we always say this with respect to TileSource class hierarchy. When a tile data is missing, an ImageLoader instance gets new job assigned with ImageJob instance. This instance receives all necessary data extracted from methods of the TileSource, namely getTileUrl, getTilePostData, getTileAjaxHeaders.

The ImageJob instance then in return executes downloadTileStart method, also a member of TileSource. ImageJob.prototype.finish() (a method that must be called from downloadTileStart in all cases except when aborting the job) notifies the ImageLoader about the job completion and status, and the data (if any) gets pulled into the system.

Last but not least, other functions may be necessary to override, especially if you change the default tile data type (Image). Changing the data type means you have to override all methods that perform tile caching (*TileCache*() like signature). Overriding cache key generation may or might not be necessary.

getTileHashKey should be overridden in cases your URL does not uniquely distinguish between different tile data. In other words, if it is possible for two same tile URLs to contain different data, you should override this method. Usually this can happen when you send the tile coordinates in POST data. It is a good idea to create the cache key based on tile coordinates, e.g. level, x and y. On the other hand, if you know that some tiles always have the same data although being on different positions, you can speed up the application by making the key equal for such tiles.

Due to flexible ways of loading the data, hasTransparency method should be also overridden to specify whether the certain tile source contains transparency. Usually the default implementation should work for you, but with advanced features it might happen you experience artifacts during the viewing, like stacking of several tile images atop each other and then vanishing when the viewer stops moving. In that case, use this method to specify the transparency flag correctly.

(Re)Implementation of Tile Data Fetching

Below is an implementation of custom tile downloading. A new fetch API is used to distinguish between the original implementation and the new approach. The TileSource can furthermore perform custom handling of faulty tiles.

Example: re-defining the tile downloading process

There are two things you can notice. First, tile cache is used extensively for repeating/mirroring the image, and so failed tiles form a periodic pattern. Secondly, we use our freedom to handle failed requests by drawing a custom failure message on the tile. For documentation specifics, please, see the API documentation.

In this example, we override existing tile source implementation, but it does not matter: you can also plug them in within an inline specification.

//see https://stackoverflow.com/questions/41996814/how-to-abort-a-fetch-request
//we need to provide the possibility to abort fetch(...)
function myFetch(input, init) {
    let controller = new AbortController();
    let signal = controller.signal;
    init = Object.assign({signal}, init);
    let promise = fetch(input, init);
    promise.controller = controller;
    return promise;
}

OpenSeadragon.extend(OpenSeadragon.DziTileSource.prototype, {
        getTilePostData: function( level, x, y ) {
            //here we exploit the POST API, a handy shortcut to pass ourselves
            //an instance to the tile object
            //return tile;
            return {width: this.getTileWidth(), height: this.getTileHeight()};
        },
        getTileAjaxHeaders: function( level, x, y ) {
            // to avoid CORS problems
            return {
                'Content-Type': 'application/octet-stream',
                'Access-Control-Allow-Origin': '*'
            };
        },
        downloadTileStart: function(imageJob) {
            // namespace where we attach our properties to avoid
            // collision with API
            let context = imageJob.userData;
            context.image = new Image();

            // in all scenarios, unless abort() is called, make
            // sure imageJob.finish() gets executed!
            context.image.onerror = context.image.onabort = function() {
                imageJob.finish(null, context.promise, "Failed to parse tile data as an Image");
            };
            context.image.onload = function() {
                imageJob.finish(context.image, context.promise);
            };

            function createErrorImageURL(e) {
                let canvas = document.createElement('canvas');
                let context = canvas.getContext('2d');

                //yay, postData = tile instance
                let tile = imageJob.postData;
                canvas.width = tile.width;
                canvas.height = tile.height;
                context.font = "80px Georgia";
                context.fillStyle = "#ff2200";
                context.fillText(e, 5, 120);
                return canvas.toDataURL("image/jpeg");
            }

            // note we ignore some imageJob properties such as
            // 'loadWithAjax'. This means preventing OSD from using
            // ajax will have no effect as we force it to do so.
            // Make sure you implement all the features the official
            // implementation do if you want to keep them.
            context.promise = myFetch(imageJob.src, {
                method: "GET",
                mode: 'cors',
                cache: 'no-cache',
                credentials: 'same-origin',
                headers: imageJob.ajaxHeaders || {},
                body: null
            }).then(data => {
                //to spice up things, emulate faulty source
                if (Math.random() > 0.7) throw "Oh no!";
                return data.blob();
            }).then(blob => {
                context.image.src = URL.createObjectURL(blob);
            }).catch(e => {
                context.image.src = createErrorImageURL(e);
            });

        },
        downloadTileAbort: function(imageJob) {
            //we can abort thanks to myFetch() implementation
            imageJob.userData.promise.controller.abort();
        }
    });

OpenSeadragon({
    id:            "example-custom-tilesource-advanced",
    prefixUrl:     "/openseadragon/images/",
    navigatorSizeRatio: 0.25,
    wrapHorizontal:     true,
    loadTilesWithAjax:  true, // no effect
    tileSources: [
        "/example-images/highsmith/highsmith.dzi"
    ]
});
    

Note that we do not have to re-define any of the methods dealing with transparency and caching. This is because: finish() is called with an Image object; URL uniquely distinguishes between tiles with unique data; and URL contains 'jpg', a string that will make OpenSeadragon realize this data is not transparent (see the default hasTransparency implementation).

Tip: if you experience timeouts, make sure finish() gets called.

Synthesized images

Sometimes, we just want to have the ability to zoom within a tiled space without the need for fetching data. Or, our data can be derived realtime. In both cases, instead of fetching such data from a server it is a better idea to render everything directly.

Mandelbrot Fractal with OpenSeadragon

The naive implementation uses per-pixel canvas processing in JavaScript. Much better performance could be achieved using WebGL.

OpenSeadragon({
    id:                 "fractal",
    prefixUrl:          "/openseadragon/images/",
    showNavigator:      false,
    blendTime:          0,
    wrapHorizontal:     true,
    tileSources:   {
        //please, do not use Infinity, OSD internally builds a cached tile hierarchy
        height: 1024*1024*1024,
        width:  1024*1024*1024,
        tileSize: 256,
        minLevel: 9,
        //fractal parameter
        maxIterations: 100,
        getTileUrl: function(level, x, y) {
            //note that we still have to implement getTileUrl
            //since we do, we use this to construct meaningful tile cache key
            //fractal has different data for different tiles - just distinguish
            //between all tiles
            return `${level}/${x}-${y}`;
        },
        getTilePostData: function(level, x, y) {
            //yup, handy post data
            return {
                dx : x,
                dy: y,
                level: level
            };
        },
        iterateMandelbrot: function(refPoint) {
            var squareAndAddPoint = function(z, point) {
                let a = Math.pow(z.a,2)-Math.pow(z.b, 2) + point.a;
                let b = 2*z.a*z.b + point.b;
                z.a = a;
                z.b = b;
            };

            var length = function(z) {
                return Math.sqrt(Math.pow(z.a, 2) + Math.pow(z.b, 2));
            };

            let z = {a: 0, b: 0};
            for (let i=0;i < this.maxIterations; i++){
                squareAndAddPoint(z, refPoint);
                if(length(z)>2) return i/this.maxIterations;
            }
            return 1.0;
        },
        downloadTileStart: function(context) {
            let size = this.getTileBounds(context.postData.level, context.postData.dx, context.postData.dy, true);
            let bounds = this.getTileBounds(context.postData.level, context.postData.dx, context.postData.dy, false);
            let canvas = document.createElement("canvas");
            let ctx = canvas.getContext('2d');

            size.width = Math.floor(size.width);
            size.height = Math.floor(size.height);

            if (size.width < 1 || size.height < 1) {
                canvas.width = 1;
                canvas.height = 1;
                context.finish(ctx);
                return;
            } else {
                canvas.width = size.width;
                canvas.height = size.height;
            }

            //don't really think about the rescaling, just played with
            // linear transforms until it was centered
            bounds.x = bounds.x*2.5 - 1.5;
            bounds.width = bounds.width * 2.5;
            bounds.y = (bounds.y * 2.5) - 1.2;
             bounds.height = bounds.height * 2.5;

            var imagedata = ctx.createImageData(size.width, size.height);
            for (let x = 0; x < size.width; x++) {

                for (let y = 0; y < size.height; y++) {
                    let index = (y * size.width + x) * 4;
                    imagedata.data[index] = Math.floor(this.iterateMandelbrot({
                        a: bounds.x + bounds.width * ((x + 1) / size.width),
                        b: bounds.y + bounds.height * ((y + 1) / size.height)
                    }) * 255);

                    imagedata.data[index+3] = 255;
                }
            }
            ctx.putImageData(imagedata, 0, 0);
            // note: we output context2D!
            context.finish(ctx);
        },
        downloadTileAbort: function(context) {
            //we could set a flag which would stop the execution,
            // and it would be right to do so, but it's not necessary
            // (could help in debug scenarios though, in case of cycling
            // it could kill it)

            //pass
        },

        createTileCache: function(cache, data) {
            //cache is the cache object meant to attach items to
            //data is context2D, just keep the reference
            cache._data = data;
        },

        destroyTileCache: function(cache) {
            //unset to allow GC collection
            cache._data = null;
        },

        getTileCacheData: function(cache) {
            //just return the raw data as it was given, part of API
            return cache._data;
        },

        getTileCacheDataAsImage: function() {
            // not implementing all the features brings limitations to the
            // system, namely tile.getImage() will not work and also
            // html-based drawing approach will not work
            throw "Lazy to implement";
        },

        getTileCacheDataAsContext2D: function(cache) {
            // our data is already context2D - what a luck!
            return cache._data;
        }
    }
});
    

Unlike in the first example, we work with RenderingContext2D object instead of Image. Also, instead of downloading we just generate it realtime.

In OpenSeadragon, cache is used unless context2D property of Tile instance exists. By default, cached data can be used AS-IS. But when touching the downloading logics and finishing with different data than Image object, you have to implement also *TileCache* methods.

Note that we do not have to re-define getTileCacheKey(): we rely on the default key generation using URL. If we had returned an empty string in getTileUrl (also a correct solution), we would have to override getTileCacheKey() , but since we are forced to define the url getter, we may as well re-use it for the cache key creation.

In order to use the tile cache, we define how the cache is created: we are given the cache context object and the data object we gave to successful job.finish(...) call. For OpenSeadragon to be able to work with such a generic approach, two methods must be implemented, that convert the data to RenderingContext2D and Image objects. It does not mean these objects will be created, only if needed. Plugins might require to access the data as Image objects, and also the default drawing strategies work either with canvas context (canvas strategy) or image object (html div strategy). But, unlike the default behaviour, we can avoid the creation of Image object, since we work by default with canvas and OpenSeadragon is mostly used with canvas drawing strategy anyway.

Performance caveat: getTileCacheData[...] functions are among other things used to access the data when rendering, it is a good idea to execute only optimized code and store results to return them immediately.