Decoding China’s tech world.

Brief

Nvidia trained new robot-hand model on Beijing firm’s data, company says

A head-mounted camera’s view of someone gripping a saucepan, with EgoHand’s overlay tracking the joints of each hand. The free AI model, trained on 14 million images, reconstructs the 3D shape and movement of hands from ordinary first-person video, and anyone can use it, including commercially.

A head-mounted camera’s view of someone gripping a saucepan, with EgoHand’s overlay tracking the joints of each hand. The free AI model, trained on 14 million images, reconstructs the 3D shape and movement of hands from ordinary first-person video, and anyone can use it, including commercially.

Beijing motion-capture company Noitom says Nvidia used its hand-tracking data to train EgoHand, Nvidia’s new model for reconstructing hands in 3D from first-person video.

Noitom Robotics released the dataset, called HiPHI-Hand, on Hugging Face on Tuesday and said it matches the data it delivered to Nvidia. It holds 11 hours of video from headset cameras, paired with sub-millimetre optical motion capture of hands handling objects. Noitom has licensed the public release for non-commercial research only.

Nvidia’s model card for EgoHand does not name Noitom. It says training drew on about 16 hours of marker-based motion-capture data bought from third-party vendors, along with real-world video and synthetic images, about 14 million images in all. Nvidia has released EgoHand under a licence that allows commercial use.

Noitom said Nvidia converted its data into a format called SOMA-Hand before training. The company released a full-body version of the dataset, covering whole-body motion and how people handle objects, in August.

Source: Sina Tech · Nvidia EgoHand model card

Tagged: · ·

Keep reading