hold

Getting started

Get a copy of the code:

git clone https://github.com/zc-alexfan/hold.git
cd hold; git submodule update --init --recursive

1. Setup environments

   • Follow the instructions here: docs/setup.md.
   • You may skip external dependencies for now.

2. Train on a preprocessed sequence

   • Start with one of our preprocessed in-the-wild sequences, such as hold_bottle1_itw.
   • Familiarize yourself with the usage guidelines in docs/usage.md for this preprocessed sequence.
   • This will enable you to train, render HOLD, and experiment with our interactive viewer.
   • At this stage, you can also explore the HOLD code in the ./code directory.

3. Set up external dependencies and process custom videos

   • After understanding the initial tools, set up the "external dependencies" as outlined in docs/setup.md.
   • Preprocess the images from the hold_bottle1_itw sequence by following the instructions in docs/custom.md.
   • Train on this sequence to learn how to build a custom dataset.
   • You can capture your own custom video and reconstruct it in 3D at this point.
   • Most preprocessing artifact files are documented in docs/data_doc.md, which you can use as a reference.

4. Two-hand setting: Bimanual category-agnostic reconstruction

   • At this point, you can preprocess and train on a custom single-hand sequence.
   • Now you can take on the bimanual category-agnostic reconstruction challenge!
   • Following the instruction in docs/arctic.md to reconstruct two-hand manipulation of ARCTIC sequences.