& Construction

Integrated BIM tools, including Revit, AutoCAD, and Civil 3D
& Manufacturing

Professional CAD/CAM tools built on Inventor and AutoCAD
Markerless motion capture utilizes machine learning (ML) and other artificial intelligence (AI) technology to record and interpret the data needed for motion capture without requiring the motion capture suits, markers, or sensor hardware that traditional motion capture systems need. AI video motion capture uses computer vision technology to derive motion capture data from well-lit video footage recorded with high-resolution video cameras.
This process relies on markerless motion capture software, which identifies and tracks the relevant points on a person’s body and face to convert 2D video into 3D motion data. By removing elaborate setups and cumbersome equipment, markerless motion capture saves time and can be much less expensive compared to traditional methods.
AI-powered markerless motion capture has become so effective that even markerless facial motion capture can achieve results comparable to traditional marker-based systems. It also provides the flexibility to set up motion capture and performance capture in a variety of environments, including the outdoors and other non-studio spaces. That effectiveness and flexibility have made it attractive to various entertainment industries, such as film, gaming, advertising, and even sports for analyzing athletes’ movement.
Markerless motion capture works by analyzing video footage with computer vision and other AI solutions to track and record body and facial movement without motion capture suits or sensor markers. The videos fed into markerless motion capture software can be recorded with a single camera or multiple cameras; common video cameras will suffice, including webcams and smartphone cameras.
AI mocap from video may work better with video footage from multiple cameras at different angles, but a single video angle can work if that’s all that’s available. However, if there are multiple camera angles, a tool like Wonder Dynamics’ Wonder Animation AI motion capture software can convert a multi-angle video scene into a fully-editable 3D scene incorporating all the camera angles and camera movements from the original video.
All markerless motion capture from video works by processing the original video footage with AI video mocap and computer vision technology to locate and track significant points on the subject’s body and face, such as limbs, joints, eyes, nose, mouth, and more. The AI mocap tool then converts the 2D video footage into 3D motion data, which can then be used in 3D animation software to create characters and scenes for games, films, advertisements, VR media, and so on. Using Wonder Animation, artists save even more time because the software automatically maps the 3D motion data onto a rigged 3D character, which can then be exported to Autodesk Maya, Blender, Unreal Engine, or OpenUSD.
Markerless facial motion capture is also now highly accessible and comparable to marker-based facial motion capture, thanks to robust AI mocap detection and processing. This AI motion capture software identifies and tracks the important facial movements detected in video footage, such as the eyes/eyebrows, lips/mouth, and nose, and converts it into motion data compatible with 3D animation software. The facial motion data is then mapped onto 3D character models to produce lifelike facial expressions.
Capturing the subtle facial expressions of human actors is a higher-level task than capturing their major body movements. As such, it can be helpful to use footage taken from multiple angles and multiple cameras in well-lit conditions. That said, today’s AI software for markerless facial motion capture is powerful enough to produce usable results from low-light conditions and from a single camera.
For example, Wonder Dynamics’ Wonder Studio performs markerless facial and bodily motion capture from live-action footage shot with any camera, including webcams and smartphones. It can also take a video cut from multiple camera angles and convert it into a full 3D scene including rigged 3D characters with the video’s motion data applied, full 3D environments, and lighting and camera movements matching the original input video. Wonder Studio’s 3D scenes can then be imported into Autodesk Maya, Blender, Unreal Engine, or OpenUSD. This level of motion capture automation saves 3D artists a huge amount of time and expense over traditional motion capture, democratizing the accessibility of professional motion capture animation for nearly any budget.
AI mocap allows creators of almost any skill level or budget to use markerless motion capture, which can be simpler, faster, and less expensive than traditional motion capture. Benefits include:
Markerless motion capture does not require cumbersome motion capture suits or physical markers that are adhered to or carried on an actor’s body. That makes it much easier to set up a markerless motion capture shoot with minimal requirements.
Removing expensive motion capture suits, sensor markers, and other physical markers from the equation makes markerless motion capture significantly more affordable than traditional mocap solutions.
With markerless motion capture, production teams do not need specialized equipment or studios, so they have the freedom and flexibility to capture actors’ performances in most environments.
In addition to faster, more flexible setups for motion capture shoots, the high-powered AI processing software used for markerless motion capture speeds up the process of converting video footage to 3D character models with motion data applied.
Markerless AI mocap brings value to a variety of areas in media and entertainment, including:
Markerless AI motion capture empowers entertainment projects like animated films and TV, video games, VR, and even music videos to create realistic 3D characters based on actual human movement.
Postproduction artists enjoy the immediacy and flexibility of AI mocap from video, which analyzes movements directly from video footage in real time.
The accessibility of markerless motion AI capture makes mocap more attractive to smaller projects like commercials, even at a grassroots level for small businesses.
Markerless motion capture is just one aspect of visual effects (VFX) that AI animation software can help automate to save time and resources. Other valuable AI VFX functionality may be packaged within the same AI animation tools that perform markerless mocap, significantly boosting VFX workflow efficiency.
For example, Wonder Dynamics’ Wonder Studio VFX AI software can automate several VFX processes based on standard video footage. VFX artists can upload a single- or multi-camera video scene to Wonder Studio, and the cloud-based software uses its Video to 3D Scene technology to convert it into a full 3D animation scene compatible with industry-standard 3D animation software. Wonder Studio’s converted 3D scenes automate the creation of markerless motion capture data from video and also automate the conversion of the camera tracking, lighting, and rigging of 3D characters based on the actors in the scene, and the compositing of the 3D characters into a full 3D environment.
Once converted, Wonder Studio’s 3D scenes offer individual element export for elements such as motion capture data, alpha mask, character pass, camera tracking, clean plate, and a full scene render from Wonder Dynamics’ cloud servers. The 3D scenes are also fully editable by exporting them to popular 3D animation software, including Autodesk Maya, Blender, Unreal Engine, and OpenUSD.
All of these advanced AI VFX tools accelerate VFX workflows while simplifying VFX processes, saving time and cost for VFX post-production teams. These VFX AI tools make realistic animations more accessible to teams working in film, video games, and virtual reality (VR), and augmented reality (AR) applications, while greatly reducing their need to make tedious manual adjustments. At the same time, VFX artists retain their full creative potential by getting fully editable 3D scenes.
Discover all that Wonder Studio and its Video to 3D Scene technology can do in addition to AI-powered motion capture: camera tracking, lighting, individual element export, and more.
Virtual production blends traditional filmmaking with digitally created characters and environments. It uses techniques like performance capture and motion capture to give virtual characters convincing movements and facial expressions. It’s also used to create live LED walls, which project photorealistic imagery onto LED screens behind physical sets.
Read about the many animation and VFX tasks AI is helping artists accomplish faster. Also, discover the difference between AI animation tools that produce uneditable outputs and those that save artists time but leave them with complete creative freedom.
Hear from the Wētā FX VFX Supervisor on the latest in motion capture and virtual production technologies that brought lifelike simians to the screen in 2024’s Kingdom of the Planet of the Apes.
Learn more about working with performance capture, a more nuanced version of motion capture that records aspects of an actor’s performance in real time, including voice, major body movements, and subtle movements such as small gestures and facial expressions.
Find out how the latest Autodesk AI innovations in media and entertainment software take tedious work off artists’ plates so they can focus more on creative storytelling. These features automate repetitive, mundane tasks related to VFX finishing, task scheduling, character model deforming and rigging, and more.
Traditional motion capture uses different kinds of markers or sensors adhered or placed on an actor’s body and face, which generate data that are translated into the digital skeleton of rigged 3D character models. These methods require performers to wear motion capture suits and/or sensors like accelerometers, gyroscopes, and magnetometers—all of which can impede the actors’ movements and sense of freedom.
Markerless motion capture, on the other hand, does not require performers to wear markers, suits, or data sensors. It utilizes advanced AI motion capture technology such as computer vision and performance capture software that analyzes video frames like a 3D scanner to track movement and digitize the image into manipulable 3D data. Markerless motion capture allows performers unfettered movement and democratizes access to motion capture so that creators with nearly any budget and skill level can use it.
Markerless facial motion capture, using the most advanced machine learning and AI motion capture algorithms, is comparable in accuracy to traditional marker-based motion capture. However, there are some concerns with markerless motion capture to be aware of when recording, such as the quality of lighting, the complexity of the facial expressions, and whether any part of the face is hidden from view. Multiple camera angles may be necessary to capture the entire face.
Advanced AI motion capture processing has closed the gap between markerless facial mocap and traditional, marker-based facial mocap. Marker-based techniques may still offer the highest level of precision for now, but AI motion capture will only continue to improve.
Yes, AI mocap can be used for real-time motion tracking, which makes it useful for real-time applications like live performances, interactive gaming, and different kinds of XR technologies like virtual reality (VR) and augmented reality (AR) applications. AI mocap has progressed enough to offer reliable real-time processing for finding and tracking the relevant movement points on a body or face. It can quickly convert the movement of those points in real time into 3D-modeled characters that mirror the motion capture data.
To use markerless AI motion capture software to its fullest ability, you will need at least one high-resolution video camera, a computer or mobile device, and a reliable, high-speed internet connection. Depending on the AI motion capture software used, the requirements may vary.
For example, Wonder Studio only requires a video uploaded to its cloud servers through the Google Chrome or Apple Safari browsers. However, depending on the scope of the production, the desired results, and the markerless motion capture software used, a production may want to utilize four or more high-res cameras to capture multiple facial angles, professional lighting, and high-end computers with solid-state drive storage and 16GB or more of RAM.
Yes, markerless motion capture is suitable for the complex movements in dance and sports for several reasons. First, it does not require people to wear any kind of adhesive markers or motion capture suits that could restrict the athlete’s movement. Also, markerless motion capture systems for the most part are easier to use and to set up in settings outside of a controlled studio environment.
Provided that the markerless system utilizes advanced enough computer vision and other AI motion capture algorithms, it can more accurately track a dancer’s or athlete’s dynamic, fast, and nuanced movements in real time, making it suitable for performance analysis and training.
AI motion capture processing’s reliability in low-light conditions is getting markedly better thanks to AI-assisted enhancements to low-light image processing like noise reduction and better low-light accuracy. However, well-lit conditions still ensure the best-case scenario for more accurate performance capture and motion capture results.
Markerless facial motion caption for character animation works in the same way that markerless body motion capture works, with a higher level of AI movement detection and tracking.
Markerless motion capture software takes video footage shot on any high-resolution video camera and uses advanced AI algorithms to identify and track key facial movements. It then converts that movement into motion data, which animation software can map onto 3D character models to result in realistic 3D characters that mirror the actual facial movements of human actors.
Deriving AI mocap from video improves postproduction workflow efficiency because its markerless motion capture automates much of the work of capturing and processing motion data, making it faster and less labor-intensive.
Without the need for traditional motion capture suits and markers, AI mocap also saves budget and can be done easily in a variety of settings beyond a production studio. If the AI mocap system is capable of real-time feedback, as it often is, that instant feedback allows a production team to make immediate adjustments that improve the overall results.
Finally, AI motion capture systems like Wonder Dynamics’ Wonder Studio offer seamless integration with popular 3D animation tools, including Autodesk Maya, Blender, Unreal Engine, and OpenUSD.