third_party/blink/renderer/modules/csspaint/README.md

# CSS Paint API

This directory contains the implementation of the CSS Paint API.

See [CSS Paint API](https://drafts.css-houdini.org/css-paint-api/) for the web exposed APIs this
implements.

See [Explainer](https://github.com/w3c/css-houdini-drafts/blob/master/css-paint-api/EXPLAINER.md)
of this feature, as well as [Samples](https://github.com/GoogleChromeLabs/houdini-samples/tree/master/paint-worklet).

## Workflow

Historically the CSS Paint API (PaintWorklet) implementation ran on the main
thread, but is currently being optimized to run on the compositor thread. We
will use an example to show the workflow of both cases.

Here is a simple example of using PaintWorklet to draw something on the screen.

<style>
#demo {
  background-image: paint(foo);
  width: 200px;
  height: 200px;
}
</style>
<script>
registerPaint('foo', class {
  paint(ctx, size) {
    ctx.fillStyle = 'green';
    ctx.fillRect(0, 0, size.width, size.height);
  }
});
</script>

In our implementation, there is one [PaintWorklet](paint_worklet.h) instance
created from the frame. There are two
[PaintWorkletGlobalScope](paint_worklet_global_scope.h) created to enforce
stateless. The number of global scopes can be arbitrary, and our implementation
chose two.

During `PaintWorkletGlobalScope#registerPaint`, the Javascript inside the paint
function is turned into a V8 paint callback. We randomly choose one of the two
global scopes to execute the callback. The execution of the callback
produces a PaintRecord, which contains a set of skia draw commands. The V8 paint
callback is executed on a shared V8 isolate.

### Main thread workflow

During the main thread paint, the `PaintWorklet::Paint` is called, which
executes the V8 paint callback synchronously. A PaintRecord is produced and
passed to the compositor thread to raster.

When animation is involved, the main thread animation system updates the value
of the animated properties, which are used by the `PaintWorklet::Paint`.

### Off main thread workflow

Let's see how it works without animations.

1. During the main thread paint, a
   [PaintWorkletDeferredImage](../../core/css/cssom/paint_worklet_deferred_image.h)
   is created. This is an image without any color information, it is a
   placeholder to the Blink paint system. The creation of its actual content
   is deferred to CC raster time. It holds input arguments which is
   encapsulated in [CSSPaintWorkletInput](../../core/css/cssom/css_paint_worklet_input.h).
   The input arguments contain necessary information for the CC raster phase.

1. During commit, the `PaintWorkletInput` is passed to CC. Specifically, the
   `PictureLayerImpl` owns `PaintWorkletRecordMap`, which is a map from
   `PaintWorkletInput` to `std::pair<PaintImage::Id, PaintRecord>`. The
   `PaintImage::Id` is used for efficient invalidation. The `PaintRecord` is
   the actual content of the `PaintWorkletDeferredImage`, which will be
   generated at CC raster time. Initially the `PaintRecord` is `nullptr` which
   indicates that it needs to be produced.

1. After commit, we need to update the pending tree. This happens in
   `LayerTreeHostImpl::UpdateSyncTreeAfterCommitOrImplSideInvalidation`.
   There are two steps involved.

   1. The first step is to gather all dirty paint worklets that need to be
     updated, which happens in `LayerTreeHostImpl::GatherDirtyPaintWorklets`. It
     basically goes through each `PictureLayerImpl` whose
     `PaintWorkletRecrodMap` isn't empty, and if there is a `PaintWorkletInput`
     with its associated `PaintRecord` being nullptr, then this worklet needs to
     be updated.

   1. Once we have gathered all the dirty paint worklets, the next step is to
     produce the `PaintRecord` which is the actual contents. The compositor
     thread asynchronously dispatches the paint jobs that produce the
     `PaintRecord` to a worklet thread. Each paint job is basically a V8 paint
     callback, the paint callback is executed on the worklet thread and the
     `PaintRecord` is given back to the compositor thread such that it can be
     rastered. Given that the V8 paint callback contains user defined javascript
     code and can take arbitrary amount of time, the paint job doesn't block the
     tree activation. In other word, the pending tree can be activated even if
     the paint jobs are not finished, it will just use the `PaintRecord` that
     was produced in the previous frame.

Now let's see how it works with animation. Here is an
[example](https://jsbin.com/muwiyux/9/edit?html,css,output) that animates a
custom property '--foo' with paint worklet. Traditionally
[custom properties](https://developer.mozilla.org/en-US/docs/Web/CSS/Using_CSS_custom_properties)
cannot be animated on the compositor thread. With off main thread paint worklet
design, we can animate the custom properties off the main thread and use them in
paint worklet. Note that currently our implementation supports custom property
animations only, not native properties. We do intend to extend to support
native properties in the future.

1. When resolving style, `CompositorKeyframeValue` will be created through
   `CompositorKeyframeValueFactory::Create` function. This basically tells the
   main thread animation system to not animate the custom properties, and
   instead creating a compositor animation for each custom property.

1. After Blink paint, a compositor animation will be created through the
   `CreateCompositorAnimation` function. The compositor animation is passed to
   CC via commit process.

1. CC ticks the compositor animation, which updates the value for the custom
   property. Currently we only support custom properties that represents number
   or color. This is handled by
   `AnimatedPaintWorkletTracker::OnCustomPropertyMutated`. The
   `AnimatedPaintWorkletTracker` class handles custom properties animated by
   paint worklet.

1. By combining custom property name with `ElementId`, we create
   `PaintWorkletInput::PropertyKey` which can be used to identify a
   `PaintWorkletInput`. Then we can use the `PaintWorkletInput` to find its
   associated `PaintRecord` in the `PictureLayerImpl`'s `PaintWorkletRecordMap`,
   invalidate it and update its content when we update the pending tree via
   `LayerTreeHostImpl::UpdateSyncTreeAfterCommitOrImplSideInvalidation`.

## Implementation

### [CSSPaintDefinition](css_paint_definition.h)

Represents a class registered by the author through `PaintWorkletGlobalScope#registerPaint`.
Specifically this class holds onto the javascript constructor and paint functions of the class via
persistent handles. This class keeps these functions alive so they don't get garbage collected.

The `CSSPaintDefinition` also holds onto an instance of the paint class via a persistent handle. This
instance is lazily created upon first use. If the constructor throws for some reason the constructor
is marked as invalid and will always produce invalid images.

The `PaintWorkletGlobalScope` has a map of paint `name` to `CSSPaintDefinition`.

### [CSSPaintImageGenerator][generator] and [CSSPaintImageGeneratorImpl][generator-impl]

`CSSPaintImageGenerator` represents the interface from which the `CSSPaintValue` can generate
`Image`s. This is done via the `CSSPaintImageGenerator#paint` method. Each `CSSPaintValue` owns a
separate instance of `CSSPaintImageGenerator`.

`CSSPaintImageGeneratorImpl` is the implementation which lives in `modules/csspaint`. (We have this
interface / implementation split as `core/` cannot depend on `modules/`).

When created the generator will access its paint worklet and lookup it's corresponding
`CSSPaintDefinition` via `PaintWorkletGlobalScope#findDefinition`.

If the paint worklet does not have a `CSSPaintDefinition` matching the paint `name` the
`CSSPaintImageGeneratorImpl` is placed in a "pending" map. Once a paint class with `name` is
registered the generator is notified so it can invalidate an display the correct image.

[generator]: ../../core/css/css_paint_image_generator.h
[generator-impl]: css_paint_image_generator_impl.h
[paint-value]: ../../core/css/css_paint_value.h

### Generating a [PaintGeneratedImage](../../platform/graphics/paint_generated_image.h)

`PaintGeneratedImage` is a `Image` which just paints a single `PaintRecord`.

A `CSSPaintValue` can generate an image from the method `CSSPaintImageGenerator#paint`. This method
calls through to `CSSPaintDefinition#paint` which actually invokes the javascript paint method.
This method returns the `PaintGeneratedImage`.

### Style Invalidation

The `CSSPaintDefinition` keeps a list of both native and custom properties it will invalidate on.
During style invalidation `ComputedStyle` checks if it has any `CSSPaintValue`s, and if any of their
properties have changed; if so it will invalidate paint for that `ComputedStyle`.

If the `CSSPaintValue` doesn't have a corresponding `CSSPaintDefinition` yet, it doesn't invalidate
paint.

## Testing

Tests live [here](../../../web_tests/http/tests/csspaint/) and
[here](../../../web_tests/external/wpt/css/css-paint-api/).