Reads alembic file format (currently HDF5 based up to version 1.1) and outputs compiled frame data into FIFOs for the encoder. TODO: Alembic 1.5 introduced a new data format which should greatly benefit read speed.
Supported alembic features
We support static and dynamic transforms with static or homogeneous poly meshes as leafs. Heterogeneous meshes (with changing topology) are not currently supported as they are not suitable for our compression scheme.
Alembic supports different frame spacing per object in the scene tree. E.g. a mesh can have frames at irregular intervals and a transform instead has a fixed 1/30s time step in the same file. CryEngine geometry caches do not support this. Each frame contains data for all objects in the scene to make realtime decoding simpler. To achieve this the alembic compiler first creates the superset of all frame times of the objects in the scene ( AlembicCompiler::CheckTimeSampling). For each unique found frame time a frame will be output.
Alembic has seperate arrays for each vertex data element. This for example allows two triangles to share the same position data for a vertex, but don't share the texture coordinates. GPUs only support a single index buffer for all elements therefore in this case we need to also duplicate the position data. This even is true if the UVs would only be different in one frame of the entire animation because we need the topology to stay the same for all frames.
To achieve this we first calculate a 64 bit hash for each vertex over all frames of the mesh's animation (AlembicCompiler::ComputeVertexHashes). If the hash of a vertex is different for two triangles it needs to be split. Note that there is a chance of a hash collision, but this is pretty unlikely with 64 bits. Checking for collisions would be pretty slow, so we don't do this at the moment.
Smooth normal generation
If the mesh is does not contain normals we need to calculate smooth normals for the mesh in those cases (AlembicCompiler::CalculateSmoothNormals). We couldn't drop support meshes without normals because Maya doesn't store any normal information for a mesh if it is completely smooth. The used algorithm averages the normals of the adjacent triangles multiplied by angle of the respective corners.
Tangent space calculation
We use Mikkelson's tangent space algorithm for calculating a tangent space for each frame of the animation from the normals and UVs of the mesh.
Meshes are optimized with Forsyth's face reorder algorithm to better utilize the GPU's transform cache. This is also necessary as a preparation for inter prediction which assumes the index order follows strips of triangles over the mesh.
The engine format supports multiple materials per mesh. For this the index buffer will be split into chunks for each material. Currently this is done by parsing the name of the face sets of the mesh in the alembic file where the first number in the name is interpreted as the material ID (AlembicCompiler::GetMeshFaceSets).
Vertex quantization and conversion
For compression purposes all data is quantized to integer values (AlembicCompiler::CompileVertices).
Positions are first normalized to [0,1] per axis where 0 and 1 represent the minimum and maximum extent for that axis of the AABB of the mesh's entire animation. After that the position is quantized to uint16. The value representing 1 is chosen so that enough bits are allocated to be better or good enough to match the specified position precision which can be set by RC command line. Smaller meshes therefore use a smaller range of values and therefore compress better.
Texcoords are quantized to uint16. The range is 0-63. Values wrap outside of that range.
Colors are stored as four individual streams for red, green and blue where each value is coded as one uint8 representing range 0-1.
Normals & tangents
Tangent frames consisting of normal, bitangent and tangent are converted to QTangents and then each component is quantized to int16 where 10 bits are used for range [-1, 1]. The reflection of the first frame is enforced for all successive frames to avoid interpolation issues. This only happens in edge cases where the tangent frame calculation produces different orientations for different frames. (AlembicCompiler::EncodeQTangent)
Static and animated data
Each vertex channel of a mesh (positions, uv, color, QTangent) can either be static or dynamic. Static data is only stored once in the beginning of the file, so only animated channels are stored per frame.
Transform hierarchy optimization
As an optimization the alembic transform hierarchy is flattened as far as possible. All unnecessary transforms in the hierarchy are removed, e.g. a mesh with two dynamic transforms and no siblings will only have a single dynamic transform in the engine format (AlembicCompiler::CompileStaticDataRec). RC with verbose output will print the resulting tree for debugging purposes.
Transforms are converted to QuatTNS (Quaternion with translation and non-uniform scale) and stored per frame if they are dynamic. This is not optimized yet because the data rate was not an issue compared to vertex animation.
Parallel frame processing
Dynamic meshes and transforms are updated for each frame. For transforms the influences for that transform are read from the Alembic scene each frame and the final result is computed. For meshes the vertex data is updated, smooth normals are recompiled if required and the tangent space is recomputed. Finally the vertices are converted to engine format again (see "Vertex quantization and conversion").
For output there is one FIFO buffer per mesh and transform which is written by the compiler and read from the encoder in frame order.
As this is an expensive operation the updates are run as jobs. One job per mesh and one job for all frame transform updates is spawned. To get enough parallelism data for multiple frames is processed in parallel. As the output must be in order a reservation buffer is used for each mesh and transform and data is only passed to the encoder if the next frame is completed. In case the reservation buffer runs full, the compiler stalls.