Selecting the Right Opacity for 2D Point Clouds

This notebook is a response to the question from Benjamin Schmidt’s notebook, Dot-density election maps with Webgl. Benjamin points out in the section Dynamic sizes and opacities,

For this to look good, the opacity and size needs to change with the zoom level. See my notes on the right way to increase point sizes on zoom here.. It seems clear that opacity needs a similar strategy, but I don't yet know what that formula should be.

I’ve run into this so many times, and it’s agonizing every single time! My goal in this notebook is to hopefully just answer the question, what should the point opacity be when you render a lot of points?

Please do note: the best answer does not involve directly drawing millions of points of data at the same time. It’s unlikely you’re actually processing that much information at one time, whether that’s because you’re only looking at a small subset or because you’re looking at aggregate statistics like the density. There are lots of ways to extract meaning from large datasets; this notebook just happens to concern itself with the particular case where you do, for a variety of legitimate reasons, want to directly render many points.

Update: Kari Lavikka has contributed a wonderful SDF fix for varying the point opacity at the edge to accomplish better antialiasing.

First, CSS versus device pixels

First, let’s recall the difference between CSS pixels and device pixels. A single CSS pixel corresponds to a nicely sized single dot on a screen, as seen by a human. Until retina displays became common, there was generally no confusion and a pixel was a pixel. In a retina display with a device pixel ratio of 2, however, each CSS pixel corresponds to a block of device pixels.

Therefore, so that WebGL points have a consistent CSS pixel size for everyone, we must multiply our `gl_PointSize` by the pixel ratio (which we may select slightly differently than the true device pixel ratio in case we want to downsample the canvas resolution just a bit and help things render a bit more quickly).

Selecting a point size

We start with an arbitrary function to select a point size. The function below implements a possible function with the following rules:

We select a point size in y-axis units.
We apply a the minimum and maximum size, specified in CSS pixels

Remember that these rules are arbitrary and the opacity function we’ll come up with below will happily use whatever size we give it.

// Compute the point size, in device pixels
function pointSizeDevicePixels(ctx, props) {
  // Get the y range from the orthographic view matrix
  const yAxisRange = 2.0 / ctx.view[5];

  // The height as a fraction of the current y range, then converted to device pixels
  const heightFraction = props.pointYAxisSize / yAxisRange;
  const deviceSize = heightFraction * ctx.viewportHeight;

  return clamp(
    deviceSize,
    props.pointScreenSize[0] * ctx.pixelRatio,
    props.pointScreenSize[1] * ctx.pixelRatio
  );
}

function clamp(value, min, max) {
  return Math.max(min, Math.min(max, value));
}

Computing the opacity

Now we get to the fun part. If we view drawing points as depositing opacity, then in order to produce a constant look, we desire some consistent average fill of opacity per screen area. Let’s call this fill density and measure it in units of , where corresponds to a single full-opacity pixel.

Consider a single -pixel rasterized point with opacity . We’ll assume for simplicity that all of our pixels have the same size and opacity. Then the total fill of this point is and the total fill of such rasterized points is

If we have a canvas of size device pixels, then average fill is

Solving for , we obtain

As we zoom in or out, consider that we see a smaller or larger fraction of the points. To account for this, we finally scale the density by the zoom factors and where and are the initial axis dimensions while and are the current axis dimensions. Thus our final expression for opacity is

Note: One might think to simplify this using instead of to eliminate aspect ratio dependence. However, when the plot maintains equal scaling in data units per pixel (aspect ratio = 1), resizing the container causes the data domain to adjust. The form correctly accounts for this and keeps opacity stable across resize operations.

Finally, we do some additional bookeeping in the function below to switch over to adjusting the opacity rather than the point size once we reach the minimum point size with respect to device pixels. These adjustments work together with corresponding modifications in the vertex and fragment shaders in the `drawPoints` function below. The additional factors are:

If we render circles rather than squares, adjust the opacity to account for the fact that the unit circle only covers 78% of the unit square.
If the point size is smaller than the minimum permissible size, we multiply opacity by the square of the ratio between desired size and actual size so that we accumulate the correct amount of fill.

function pointOpacity(ctx, props) {
  // We use the above function for the point size. This means we can plug
  // in any function, even nonlinear, as long as we compute the opacity
  // correctly.
  var p = pointSizeDevicePixels(ctx, props);

  // Compute the plot's x and y range from the orthographic view matrix
  const X = 2.0 / ctx.view[0];
  const Y = 2.0 / ctx.view[5];
  const X0 = props.initialAxisDimensions[0];
  const Y0 = props.initialAxisDimensions[1];

  // Viewport size, in device pixels
  const W = ctx.viewportWidth;
  const H = ctx.viewportHeight;

  // Number of points
  const N = props.N;

  let alpha = ((props.rho * W * H) / (N * p * p)) * (X0 / X) * (Y0 / Y);

  // If it's a circle, only (pi r^2) of the unit square is filled so we
  // slightly increase the alpha accordingly.
  alpha *= props.circularPoints ? 1.0 / (0.25 * Math.PI) : 1.0;

  // If the pixels shrink below the minimum permitted size, then we adjust
  // the opacity instead and apply clamping of the point size in the vertex
  // shader. Note that we add 0.5 since we slightly inrease the size of
  // points during rendering to accommodate SDF-style antialiasing.
  const clampedPointDeviceSize =
    Math.max(props.minimumPointDeviceSize, p) + 0.5;

  // We square this since we're concerned with the ratio of *areas*.
  alpha *= Math.pow(p / clampedPointDeviceSize, 2.0);

  // And finally, we clamp to the range [0, 1]. We should really clamp this
  // to 1 / precision on the low end, depending on the data type of the
  // destination so that we never render *nothing*.
  return clamp(alpha, 0.0, 1.0);
}

Additional considerations

Data types

There are a few extra considerations. First, the data type. If we render ten million points, the opacity of each point may be tiny, perhaps just . For RGBA channels of type `uint8`, the smallest representable opacity is . If our opacities are barely representable, we expect heavy quantization of the colors at best and perhaps nothing on the screen at worst. A simple solution is to render to an offscreen `float16` or `float32` framebuffer, then transfer the colors to the screen.

Gamma

Whether or not we render to a framebuffer with higher precision, since we’ve computed our colors in a linear colorspace by simply adding them up, we should convert to sRGB by applying gamma as a separate step. The input color from the color picker is in sRGB, so we convert it to linear for blending, then convert the accumulated result back to sRGB for display. I’ve used the standard sRGB gamma approximation of 2.2.

Antialiasing

Update: I wasn’t thinking! Thanks to Kari Lavikka for varying the opacity to accomplish antialiasing and also clamping the minimum point size to a single pixel and then varying the opacity instead.

Blending

I’ve used additive blending to overlay the points, with one exception. Since the points accumulate toward black from white, I’ve used `reverse subtract` with the color inverted so that we start with 1.0 and accumulate toward 0.0. This is equivalent to rendering the inverted image and then un-inverting at the end. This tool helps me reason through it.

const GLSL_SRGB_GAMMA = `
  const float SRGB_GAMMA = 2.2;
  vec3 linearToSRGB(vec3 rgb) {
    return pow(clamp(rgb, vec3(0), vec3(1)), vec3(1.0 / SRGB_GAMMA));
  }`;

const copyToScreen = regl({
  vert: `
    precision highp float;
    attribute vec2 xy;
    void main () {
      gl_Position = vec4(xy, 0, 1);
    }`,
  frag: `
    precision highp float;
    uniform vec2 framebufferResolution;
    uniform sampler2D src;
    ${GLSL_SRGB_GAMMA}
    void main () {
      vec4 color = texture2D(src, gl_FragCoord.xy / framebufferResolution);
      gl_FragColor = vec4(linearToSRGB(color.rgb), 1);
    }`,
  attributes: { xy: [-4, -4, 4, -4, 0, 4] },
  uniforms: {
    src: regl.prop('src'),
    framebufferResolution: (ctx) => [ctx.framebufferWidth, ctx.framebufferHeight]
  },
  count: 3,
  depth: { enable: false }
});

function createDrawPoints(circularPoints) {
  return regl({
    vert: `
      precision highp float;
      attribute vec2 xy;
      uniform float pointSize, minimumPointDeviceSize;
      uniform mat4 view;
      void main () {
        gl_Position = view * vec4(xy, 0, 1);
        gl_PointSize = max(minimumPointDeviceSize, pointSize) + 0.5;
      }`,
    frag: `
      precision highp float;
      uniform float opacity, pointSize;
      uniform vec3 invertedPointColor;

      float linearstep(float edge0, float edge1, float x) {
        return clamp((x - edge0) / (edge1 - edge0), 0.0, 1.0);
      }

      void main () {
        float alpha = opacity;
        vec2 c = gl_PointCoord * 2.0 - 1.0;
        #if ${circularPoints ? '1' : '0'}
        float sdf = length(c);
        #else
        float sdf = max(abs(c.x), abs(c.y));
        #endif
        alpha *= linearstep(pointSize + 0.5, pointSize - 0.5, sdf * pointSize);
        gl_FragColor = vec4(invertedPointColor, alpha);
      }`,
    attributes: { xy: regl.prop('pointsBuffer') },
    blend: {
      enable: true,
      func: { srcRGB: 'src alpha', dstRGB: 1, srcAlpha: 1, dstAlpha: 1 },
      equation: { rgb: 'reverse subtract', alpha: 'add' }
    },
    uniforms: {
      view: regl.prop('view'),
      invertedPointColor: regl.prop('invertedPointColor'),
      minimumPointDeviceSize: regl.prop('minimumPointDeviceSize'),
      opacity: regl.prop('opacity'),
      pointSize: regl.prop('pointSize')
    },
    primitive: 'points',
    count: regl.prop('N'),
    depth: { enable: false },
    scissor: { enable: true, box: regl.prop('scissor') },
    viewport: regl.prop('viewport')
  });
}

// Mutable state - all in one cell to avoid cross-cell reassignment issues
let pointsBuffer = regl.buffer(generatePoints(plotData, N, { x: initialXDomain, y: initialYDomain }));
let fbo = regl.framebuffer({
  width: Math.floor(stack.width * pixelRatio),
  height: Math.floor(stack.height * pixelRatio),
  colorType: colorType
});
let drawPoints = createDrawPoints(circularPoints);
let currentPlotData = plotData;
let currentN = N;
let currentColorType = colorType;
let currentPixelRatio = pixelRatio;
let currentCircularPoints = circularPoints;

const loop = regl.frame(() => {
  // Check for parameter changes that require recreation
  if (plotData !== currentPlotData || N !== currentN) {
    pointsBuffer.destroy();
    pointsBuffer = regl.buffer(generatePoints(plotData, N, { x: initialXDomain, y: initialYDomain }));
    currentPlotData = plotData;
    currentN = N;
    render.dirty = true;
  }

  if (colorType !== currentColorType || pixelRatio !== currentPixelRatio) {
    fbo.destroy();
    fbo = regl.framebuffer({
      width: Math.floor(stack.width * pixelRatio),
      height: Math.floor(stack.height * pixelRatio),
      colorType: colorType
    });
    currentColorType = colorType;
    currentPixelRatio = pixelRatio;
    render.dirty = true;
  }

  if (circularPoints !== currentCircularPoints) {
    drawPoints = createDrawPoints(circularPoints);
    currentCircularPoints = circularPoints;
    render.dirty = true;
  }

  // Check if any rendering parameters changed
  if (pointYAxisSize || rho || pointScreenSizeMin || pointScreenSizeMax ||
      minimumPointDeviceSize || pointColor) {
    render.dirty = true;
  }

  if (!render.dirty) return;

  // Resize framebuffer if needed
  const targetWidth = Math.floor(stack.width * pixelRatio);
  const targetHeight = Math.floor(stack.height * pixelRatio);
  if (fbo.width !== targetWidth || fbo.height !== targetHeight) {
    fbo.resize(targetWidth, targetHeight);
  }

  // Compute viewport bounds from axes
  const viewport = {
    x: Math.min(axes.xRange[0], axes.xRange[1]) * pixelRatio,
    y: targetHeight - Math.max(axes.yRange[0], axes.yRange[1]) * pixelRatio,
    width: Math.abs(axes.xRange[1] - axes.xRange[0]) * pixelRatio,
    height: Math.abs(axes.yRange[0] - axes.yRange[1]) * pixelRatio
  };

  // Create context object for opacity/size calculations
  const ctx = {
    view: axes.view,
    viewportWidth: viewport.width,
    viewportHeight: viewport.height,
    pixelRatio: pixelRatio
  };

  const props = {
    N,
    pointYAxisSize,
    pointScreenSize: [pointScreenSizeMin, pointScreenSizeMax],
    minimumPointDeviceSize,
    rho,
    circularPoints,
    initialAxisDimensions
  };

  const pointColorRGB = hexToRgb(pointColor);
  const pointColorLinear = sRGBToLinear(pointColorRGB);

  // Render to framebuffer
  fbo.use(() => {
    regl.clear({ color: [1, 1, 1, 1] });
    drawPoints({
      view: axes.view,
      pointsBuffer,
      N,
      invertedPointColor: [
        1 - pointColorLinear[0],
        1 - pointColorLinear[1],
        1 - pointColorLinear[2]
      ],
      minimumPointDeviceSize,
      opacity: pointOpacity(ctx, props),
      pointSize: pointSizeDevicePixels(ctx, props),
      viewport,
      scissor: viewport
    });
  });

  // Copy to screen with linear → sRGB conversion
  copyToScreen({ src: fbo });

  render.dirty = false;
});

invalidation.then(() => {
  loop.cancel();
  pointsBuffer.destroy();
  fbo.destroy();
});

Selecting the Right Opacity for 2D Point Clouds

First, CSS versus device pixels

Selecting a point size

Computing the opacity

Additional considerations

Data types

Gamma

Antialiasing

Blending

Rendering optimization

Axes and interactions

Observations

WebGL Implementation