# Proposals/Advanced Gradients

*Prepared by Canon for the April 2014 SVG working group F2F meeting.*

## Background

Canon is willing to give myself and another engineer some time to look at further extending advanced gradients in SVG. This is something for after SVG 2, but the reason I bring it up now is that we want to check that the WG think this is work worth doing and something that they will consider in future.

There are two things that we're interested in:

- Diffusion curves are the main one. We’ve been researching diffusion curves for the last few years and published a paper at SIGGRAPH Asia last year (Ilbery, 2013). We are very interested in eventually getting DC in SVG (because we think a scalable vector representation of blends is a good fit to SVG), and we have some ideas about how that might be achieved
- The other is extending mesh gradients in SVG. Coon’s patch mesh gradients are a good first step for SVG 2. But after SVG 2 we think it would be useful to add other more capable formats.

## Option 1: Diffusion curves for SVG

*For a general overview of what diffusion curves are and a history of the research see my 2013 Graphical Web talk.*

**Diffusion curves can create many types of image.**

*Images (C) CiSRA 2012, 2014*

Example images created by other organisations:

- http://hal.inria.fr/hal-00732992
- http://tel.archives-ouvertes.fr/tel-00528871
- http://artis.imag.fr/Publications/2008/OBWBTS08/info.html

In fact, DC can represent any image that mesh gradients can as they have similar building blocks - areas of smoothly transitioning colour between discontinuities.

But DC have advantages over mesh gradients that make them a good fit for SVG.

- a compact representation based on image features.
- a representation that is easily understood by authors. DC were originally envisaged as an image format that closely resembles traditional artist workflows (sketch outlines, then fill with colour).
- a representation well-suited to animation

We believe there is value in exploring the use of diffusion curves as a method for defining gradients in SVG.
We would like to know if the working group consider this a suitable future direction for SVG. If so, Canon will devote more time to the subject, ultimately drafting a proposal for the SVG WG to consider.
We welcome collaborators for this project.

### Our initial ideas for diffusion curves in SVG

We have spent some time internally discussing how the DC model could be applied to SVG.

So far, published diffusion curve images have been composed using only the single primitive - the diffusion curve (or some close variant).
We believe that in SVG, diffusion curves are more likely to be useful as a primitive that generates a gradient fill that can be used as a component in an image, with the image also making use of other fill techniques where appropriate.

**Biharmonic Model**

The biharmonic diffusion curves model has been proposed in various recent papers (Boyé, 2012) (Ilbery, 2013). This model is general purpose, well defined and incorporates the most useful features for artists. In the biharmonic model, colours and a colour gradient in the normal direction may be specified independently on either side of a diffusion curve or specified as a single value at the curve.

*Image (C) CiSRA 2014*

#### A rough proposal

The content of a diffusion curve patch defines a diffusion curve gradient image. A diffusion curve gradient image is a bitmap image with alpha that is used as a fill. A diffusion curve patch contains one or more diffusion curve paths and an optional boundary path.

A diffusion curve path:

- follows the SVG path syntax.
- may have any number of smoothly transitioning colours specified independently on the left and right hand sides of the curve
- may have gradients specified on the left and right hand side that control the colour spread

Control of gradient allows for smooth transitions of colour across diffusion curves.

**Boundary curves**
**Type 1**
A diffusion curve image typically has a boundary curve to set the colour at the edge of the gradient image. Outside this boundary curve the diffusion is not calculated and pixels remain at some default value (e.g. rgba(0,0,0,0)).

We foresee two options for boundary curves.

Boundary curves provided as part of the definition of the gradient image. In this case, the filled graphic element applies a 'clip' to the diffusion curve gradient image. The colour at the edge of the graphic element's shape is dependent on where the 'clip' falls on the diffusion curve gradient image. This type has the advantage that a single diffusion curve gradient image can be re-used to fill many shapes.

* Image (C) CiSRA 2014 *

**Type 2**
Boundary curves inherited from the filled graphic element. In this case, the colour at the boundary of the filled graphic element is guaranteed to be the boundary colour specified by the author because the colour constraints lie on the boundary of the filled graphic element.
In this case, the diffusion curve gradient image will usually need to be re-calculated when used to fill a graphic element with a different shape or with different dimensions.

* Image (C) CiSRA 2014 *

### Performance

Solving diffusion curve images is a complex operation and so is somewhat expensive computationally.

Inria's vectorial solver (video) appears to offer reasonable performance. They expect to release a library (patetelib) that includes a diffusion curves module around mid 2014.

We think it's important to specify the model with respect to an exact and stable mathematical system. However we believe that implementations could approximate this solution with acceptable quality.

Later in the year we may be able to demonstrate an approximation renderer.

## Option 2: Extending Mesh Gradients

SVG 2 will support mesh gradients defined as an array of Coon's patches. This is a good first step for SVG 2. After SVG 2 is released we think extending the supported mesh types to include a format that supports additional author controls would be beneficial and help address known issues with Coon's patch mesh gradients.

This is less of a technical challenge than supporting diffusion curves.

Mesh gradients are more and more seen as a good output format for processed natural images.

- vectorial output from diffusion curves solver (Inria, 2012)
- vectorisation of bitmap images (e.g. http://research.microsoft.com/en-us/um/people/hoppe/proj/vectorization/)

So this format would be designed to be a suitable target for the above uses. This format would also be a more suitable target for output from Adobe Illustrator which uses their proprietary Gradient Mesh format.

The key thing that we think is useful, and that has been requested, is for a mesh gradient format that supports smooth transitions across boundaries. This can be achieved by adding control for the gradient of the colour profile for each channel, at the boundary.

Note that it's possible to simulate the more complex mesh gradient formats with Coon's patch mesh gradients by subdividing the patches to provide more control points. However this is a one way operation. It reduces editability of the mesh gradient and bloats the SVG source.

## Outcomes from this discussion

- We ask the working group if Canon should devote the time to propose additional advanced gradients for SVG. We would also ask how best this work should be organised so as to not impact on SVG 2 progress (community group?).
- Rename meshGradient to something like CoonsMeshGradient to allow for additional mesh gradient types in future.

## References

(Ilbery, 2013) P.Ilbery, L.Kendall, C.Concolato, M.McCosker, Biharmonic diffusion curve images from boundary elements. ACM Trans. Graph (Proc. Of SIGGRAPH Asia 2013).

(Boyé, 2012) S. Boyé, P. Barla, G. Guennebaud, A vectorial solver for free-form vector gradients. ACM Trans. Graph. (Proc. Of SIGGRAPH Asia 2012).

(Andronikos, 2013) N. Andronikos, What's so cool about diffusion curves. Talk from The Graphical Web conference 2013.