TL;DR
This project presents an offline, automated motion-capture pipeline for documenting and visualizing Indonesian traditional dances using OptiTrack MotiveBody, Blender, and Unreal Engine 5. Four dances, Tari Leu Lenggo, Tari Asih Trasne, Tari Burung Enggang, and Tari Saman were captured, cleaned, retargeted, and visualized to explore both technical challenges and cultural representation. The workflow highlights limitations of single-actor capture, marker occlusion, and pose stitching, while demonstrating the feasibility of preserving complex cultural movement through real-time animation tools.
Introduction
OptiTrack MotiveBody is an optical motion capture software that utilizes an array of infrared cameras to track retroreflective markers attached to performers and objects. Within the Faculty of Informatics at Masaryk University, this project explores the use of MotiveBody to capture Indonesian traditional dances and to develop an offline pipeline that transfers motion data from OptiTrack into Unreal Engine 5 for visualization and analysis.
Workflow
This project focuses on capturing Indonesian traditional dances to highlight both the diversity and unity of regional cultural expression, in line with the national principle of Bhinneka Tunggal Ika (“Unity in Diversity”). The workflow encompasses data acquisition, cleanup, skeletal preprocessing, retargeting, and final visualization, as illustrated below.
OptiTrack Setup
Prior to recording, OptiTrack requires a standard calibration process, including wanding to define the three-dimensional capture volume and aligning all cameras into a unified coordinate system, followed by ground-plane definition. Achieving a high calibration score is essential to ensure accurate capture results.
Marker placement was performed using a full-body OptiTrack suit equipped with over 40 markers, enabling full-body motion tracking. Hand and finger movements were not captured due to missing dedicated hand-tracking equipment. Each performer was defined as a humanoid entity within MotiveBody, allowing skeletal properties to be configured as required.
OptiTrack Capture
The motion capture sessions were conducted with two performers, Fitri and Ida, both Indonesian students and volunteers of the Kintari Foundation , an organization that promotes Indonesian cultural heritage and education. Each dance was captured in multiple takes to ensure sufficient coverage and motion fidelity.
The dances captured include:
- Tari Leu Lenggo (Sumbawa, Bima region)
- Tari Asih Trasne (Lombok, Sasak region)
- Tari Burung Enggang (East Kalimantan, Dayak region)
- Tari Saman (Aceh, Sumatra)
During capture, careful attention was paid to the positioning of the performers within the OptiTrack volume to minimize marker occlusion and preserve the integrity of the choreography. Limitations of the setup included the inability to capture finger articulation and some marker occlusions during close-contact movements. Despite these constraints, the captures successfully recorded the full-body motion necessary for downstream processing in Blender and Unreal Engine 5.
Each take was labeled and organized to facilitate modular editing, pose matching, and animation stitching during visualization, with additional care to maintain the temporal and spatial integrity of the cultural movements.
OptiTrack Cleanup
Following capture, the recorded takes required post-processing to address noise, occlusions, and missing marker data. OptiTrack MotiveBody’s built-in cleanup tools were used to fill gaps both automatically and manually, employing different interpolation methods depending on the characteristics of each take.
OptiTrack Export
Once the motion capture data was cleaned and validated in OptiTrack, it was exported in FBX format for further processing. The FBX export converts the tracked marker positions into a skeletal armature, which can then be imported into 3D software such as Blender or Maya for additional cleanup, rigging, and preparation before integration into Unreal Engine 5. This step ensures that the motion data retains its structural integrity while enabling subsequent modifications and visualization workflows.
Blender Import
The exported FBX files were imported into Blender by opening the files directly or dragging them into the workspace. During import, the Armature Settings were configured to use Automatic Bone Orientation, which optimizes bone alignment and improves visualization of the skeletal hierarchy. This step facilitates downstream processes such as proxy geometry placement, rigging, and animation refinement.
Blender Bone Proxy Setup
Because the exported FBX files contained only skeletal data without mesh geometry, they could not be directly imported into Unreal Engine. To address this, a custom Blender script was developed to automatically generate proxy geometry parented to each bone. These primitives serve as visual guides and are color-coded to correspond with the bone chains required for inverse kinematics setup in Unreal Engine.
Blender Rigging Cleanup
After importing the FBX files into Blender, additional cleanup was performed to enhance the quality and fidelity of the captured animation. For this purpose, the tool RigOnTheFly by Dypsloom was used to generate a Control Rig, enabling both forward and inverse kinematics adjustments. This process allows for precise refinement of the skeletal motion while preserving the integrity of the original capture.
Blender Export
Once the animation data was cleaned and the proxy geometry appropriately positioned, the scene was exported back into FBX format, including both the skeletal armature and associated mesh. This consolidated FBX file serves as the standardized asset for import into Unreal Engine 5.
Unreal Engine Import
The FBX files exported from Blender were then imported into Unreal Engine 5. If a Skeletal Mesh has not yet been created, the “Import Animations Only” option was selected during import. This allows Unreal Engine to generate the required Skeletal Mesh asset, which is subsequently used for creating an IK Rig and Retargeter asset. This step ensures that the motion data is fully compatible with Unreal Engine’s animation and retargeting systems.
Unreal Engine Rigging
Although Unreal Engine 5 provides built-in automatic tools for rigging and retargeting, this project intentionally implements the process manually. This approach facilitates a deeper understanding of the pipeline and enables the development of custom Python-based Editor Utilities tailored specifically to OptiTrack-derived skeletal data.
With this, we can create an Interface that takes our Skeletal Mesh Asset and automatically create an IK Rig Asset.
Unreal Engine Retargeting
Once the IK Rig Asset was created, it was retargeted to a Skeletal Mesh of choice. For this experiment, the default Quinn Skeletal Mesh was used as the target, with the IK Rig assigned as required. Within the Retargeting Asset, the alignment and mapping between the source and target skeletons were configured to ensure accurate transfer of motion data. Certain animations necessitated manual adjustment of retargeting parameters due to differences in skeletal proportions or complex poses. These adjustments were performed on a case-by-case basis to optimize fidelity prior to exporting the final animation to the target Skeletal Mesh.
Unreal Engine Visualization
After exporting all animations to the target Skeletal Mesh, the Level Sequencer in Unreal Engine 5 was used to direct and orchestrate the choreography. The Sequencer provides advanced root-motion matching and animation control, which is particularly valuable when reconstructing animations captured in separate segments. By enabling precise timing, alignment, and blending of multiple animation clips, the Level Sequencer allows for a coherent representation of the full choreography.
Beyond technical alignment, the Sequencer also provides artistic flexibility, allowing for adjustments in cinematography, staging, and overall presentation to enhance the visual narrative of the performance.
Result
With the complete workflow from OptiTrack to Unreal Engine 5 established, the resulting animations were analyzed and visualized. The following sections present the outcomes for each traditional Indonesian dance captured, along with observations on technical challenges, motion fidelity, and cultural representation.
Tari Leu Lenggo (Sumbawa, Bima Region)
Tari Leu Lenggo, a Traditional Dance from the island of Sumbawa, specifically from the Bima Region. This dance is characterized by graceful and refined movements that symbolize the elegance, politeness, and inner beauty of Bima women. Tari Leu Lenggo is traditionally performed during welcoming ceremonies, cultural events, and important communal occasions, serving as a representation of local values and social harmony.
The entire choreography was captured across multiple takes, with the best performance selected following cleanup and retargeting. The choreography was was modified to accommodate only two dancers. Both performers executed the capture simultaneously.
Although the choreography involves multiple performers and a wide range of full-body motion, the absence of finger tracking limited the capture of fine hand articulation. Additionally, pelvis marker noise introduced minor jitter in certain segments of the animation. Physical interaction between performers, which is intrinsic to the dance, was modified to reduce marker occlusion. These adjustments reflect a common trade-off between cultural fidelity and capture reliability in optical motion capture systems.
Tari Asih Trasne (Lombok, Sasak Region)
Tari Asih Trasne is a traditional dance from the island of Lombok, originating from the Sasak tribe. This dance expresses themes of affection, sincerity, and emotional devotion, conveyed through soft, flowing movements and gentle gestures. The choreography emphasizes grace and subtle body language, reflecting the values of love, harmony, and emotional restraint within Sasak culture. Tari Asih Trasne is commonly performed at cultural events and traditional ceremonies.
The entire choreography was captured across multiple takes, with the best performance selected following cleanup and retargeting. The choreography was was modified to accommodate only one dancer (Ida).
Since Tari Asih Trasne is often performed as a communal folk dance, multiple instances of the animation were introduced during visualization. Variation was achieved by temporally offsetting segments within the Level Sequencer rather than recording additional takes. While animation blending could further enhance realism, simple temporal shifts were sufficient for this experimental context.
Tari Burung Enggang (Kalimantan Timur, Dayak Region)
Tari Burung Enggang is a traditional dance from the island of Borneo (Kalimantan), originating from the Dayak Tribe. This dance resembles the movements of the hornbill bird (Burung Enggang), which is considered a sacred and noble symbol in Dayak culture. The dance emphasizes wide, flowing arm gestures that mimic the flapping of wings, expressing grace, strength, and a deep connection between humans and nature. It is commonly performed during traditional ceremonies, cultural celebrations, and welcoming events.
Our dancer, Fitri was the one performing this dance alone. We captured parts by parts of the choreograph in hopes to get the best takes of each part and stitching them together during visualization in Unreal Engine 5 using the Level Sequencer. The entire choreograph is divided into 5 Parts from the Opening to the Ending. The splitting allows the choreograph to be modularly reenacted in Unreal Engine 5 with Pose Matching and Animation Transitions within the Sequencer. However, as good as we can get it at this current state, pose matching seems to be rather challenging with complex poses. We minimize the complexity by starting and ending each take in a T-Pose, but this makes the stitches look to visible. With the best of my current ability, the end result looks decent albeit a few jumps and discontuinities.
The entire choreography was split into multiple routines and captured across multiple takes, with the best routine selected following cleanup and retargeting. The choreography was was modified to accommodate only one dancer (Fitri).
Segmenting the choreography enabled modular reconstruction within Unreal Engine; however, pose matching between complex transitional movements proved challenging. Although starting and ending each segment in a T-pose reduced alignment ambiguity, it introduced visible discontinuities. Despite these limitations, the resulting animation remains visually coherent for exploratory visualization purposes.
Tari Saman (Sumatera Utara, Aceh Region)
Tari Saman is a traditional dance from the island of Sumatra, originating from the Aceh region. This dance is performed in a seated position, with dancers sitting closely side by side in a straight line. It is renowned for its rapid, synchronized hand, arm, and upper-body movements, often described as creating a “thousand hands” effect. Tari Saman emphasizes unity, rhythm, and precision, reflecting strong values of cooperation and collective harmony within Acehnese culture.
The entire choreography was split into multiple routines and captured across multiple takes, with the best routine selected following cleanup and retargeting. The choreography was was modified to accommodate only one dancer (Ida).
Tari Saman posed the greatest challenge due to its rapid, synchronized movements and seated posture. Capturing the choreography using a single performer significantly constrained post-capture synchronization. Pose matching fast hand movements across segments proved particularly difficult, resulting in visible discontinuities. These findings indicate that future capture sessions for Tari Saman should involve at least three performers recorded simultaneously to preserve synchronization integrity.
Conclusion
Overall, the motion capture process successfully produced high-quality animations despite limited prior experience with motion capture and performance capture acting. From a cultural perspective, this study represents only a small subset of Indonesia’s extensive traditional dance heritage. Furthermore, visualization using a default skeletal mesh does not yet fully convey the richness of Indonesian identity, including traditional costumes and environmental context. These limitations present clear opportunities for future work, particularly in multi-actor capture, hand tracking, and culturally accurate character representation.