Cryptomatte - What Crypto00, Crypto01, Crypto02 means?

In order to explain to sockets and their numbering I'll have to explain how Cryptomatte and some aspects of path tracing work.

---

The Idea

Cryptomatte generates ID mattes which try to preserve for each pixel of the rendered image what elements of the 3D scene contributed to it. For instance in a render with motion blur, pixels where the moving object and background object blend into each other are influenced by both objects. In order to understand how Cryptomatte identifies and stores this information, we have to take a look at how the rendered image is created.

Rendering using a path tracer such as Cycles works roughly in the following way:

In order to create the rendered image we sample multiple paths per pixel at subpixel location from the camera into the scene. The sampling is quasi-random using either the Sobol sequences or correlated multi-jitter. For each path we check if it intersects with an object, if it does, we remember the hit point. Next we chose a random point on one of the light source and try to connect it to the hit point. This is an optimization known as Next-event prediction. If a direct path from the hit point to a light source is possible we can evaluate the shaders along the path to the camera and we receive an RGBA value. In case a connection to a light source isn't possible, we bounce with a new path segment from the hit point in a quasi-random direction and repeat the process until we either hit a light source or exceeded the maximum bounces. Cycles therefore uses Quasi-Monte-Carlo integration. In order to fuse the multiple samples per pixel in the camera plane into a final RGBA value we use a pixel filter which weights and sums the subpixel samples. Cycles uses either a Blackman-Harris, Gaussian or Box filter.

This is a very simplified and shortened explanation, for a more detailed introduction I recommend watching Lukas Stockner's talk from the Blender Conference 2019. The important part for Cryptomatte is that there are multiple samples per pixel, that may hit different objects and which therefore contribute to the final pixel.

The idea behind Cryptomatte is that we can leverage the scene and path tracing information to create ID and coverage pairs per pixel, where the ID references the element that contributed to the pixel and the coverage tells how much. This allows us later on to determine which pixel and by what amount was caused by a certain object or material.

There are three elements Cryptomatte can store ID mattes for: Objects, materials and namespaces (hierarchical groups of objects, which is the Assets option in Blender). IDs are generated by hashing the object/material/namespace's name and are stored per sample in so called arbitrary output variables (AOV).

---

Filtering and Encoding

Each pixel in the render is the result of the weighted sum of its samples, where each sample belongs to an ID. The weights are given by the pixel filter, such as Blackman-Harris or Gaussian filter. There can be several samples per pixel that belong to the same ID.

The weight of the samples that belong to the same ID are accumulated per pixel. Normalizing the value gives the coverage of the ID-coverage pair for that pixel. For every pixel the pairs get ranked by their coverage, rank 0 is for the highest coverage. They are then assigned to the corresponding pixel in the layer that contains said rank. Note: Each rank is essentially a matrix that contains an ID-coverage pair for every pixel. Since the assignment of ID-coverage pairs to ranks is done per pixel, IDs for a specific object may be found across several ranks for different pixels.

Multi-layer OpenEXR files are used as data structure for storing the files. Each of the AOVs is stored in a separate file. IDs and coverages are stored each in their own channel. Since OpenEXR has RGBA channels there are two ranks per layer.

Layer 1

CryptoObject00.r - ID rank 0
CryptoObject00.g - Coverage for ID rank 0
CryptoObject00.b = ID rank 1
CryptoObject00.a = Coverage for ID rank 1

Layer 2

CryptoObject01.r - ID rank 2
CryptoObject01.g - Coverage for ID rank 2
CryptoObject01.b = ID rank 3
CryptoObject01.a = Coverage for ID rank 3

Layer 3

CryptoObject02.r - ID rank 4
CryptoObject02.g - Coverage for ID rank 4
CryptoObject02.b = ID rank 5
CryptoObject02.a = Coverage for ID rank 5

Metadata in the file stores the original name of the elements and the hash method used along other information.

---

Matte

Given a name of an object/material/namespace we can compute the ID using the hash function documented in the metadata. All that is necessary to create the matte is to iterate through the ID coverage pairs and retrieve the coverage per pixel. If a single matte for multiple IDs is requested the coverage of the IDs is summed.

---

Answering your questions

What are levels?

Levels are the number of ID-coverage pairs per pixel. Therefore the levels determine how many unique objects can be differentiated per pixel. It should be a multiple of two, since levels divided by two is the number of layers in the Multi-layer OpenEXR.

---

Why are there three input sockets on the Cryptomatte node?

The three sockets are for the recommended three layers to store the six ID-coverage pairs per pixel. You can add additional layers in the sidebar in the Item tab in the Properties panel.

---

Why are the output sockets of the Render Layers named like that?

I believe this is an error, since they get properly named internally in BlenderSync::sync_render_passes() (blender_sync.cpp) however register_passes() (engine.py) does an increment by two. This is correct in regards to the number of render passes, since there are two ranks per layer. However the names should still use consecutive numbers and match the ones of the Cryptomatte node.

I've created a post on DevTalk to discuss this with the developers.

Update: The numbering has been fixed by commit rB3f3d1ad4800c6d5e9d1a45912841b118cbac9f39

---

Why should I connect more than the first render pass and why does it work when only connecting the first?

It depends how many unique objects you'd like to differentiate per pixel. If you connect the first render pass, then the cryptomatte node can only create a matte which contains the coverage of two objects per pixel at most, since you pass two ranks into it. This will likely be enough for most simple compositing tasks. For more challenging scenes with multiple objects that have motion blur and cross paths which each other, you'll want to use the additional passes as well.

---

References

Cryptomatte Specification Version 1.2.0

Friedman, Jonah, and Andrew C. Jones. "Fully automatic id mattes with support for motion blur and transparency." ACM SIGGRAPH 2015 Posters. ACM, 2015.