talking-head-anime-2-demo

Hardware Requirements

Both programs require a recent and powerful Nvidia GPU to run. I could personally ran them at good speed with the Nvidia Titan RTX. However, I think recent high-end gaming GPUs such as the RTX 2080, the RTX 3080, or better would do just as well.

The ifacialmocap_puppeteer requires an iOS device that is capable of computing blend shape parameters from a video feed. This means that the device must be able to run iOS 11.0 or higher and must have a TrueDepth front-facing camera. (See this page for more info.) In other words, if you have the iPhone X or something better, you should be all set. Personally, I have used an iPhone 12 mini.

---

Software Requirements

Both programs were written in Python 3. To run the GUIs, the following software packages are required:

• Python >= 3.8
• PyTorch >= 1.7.1 with CUDA support
• SciPY >= 1.6.0
• wxPython >= 4.1.1
• Matplotlib >= 3.3.4

In particular, I created the environment to run the programs with Anaconda, using the following commands:

> conda create -n talking-head-anime-2-demo python=3.8
> conda activate talking-head-anime-2-demo
> conda install pytorch torchvision cudatoolkit=10.2 -c pytorch
> conda install scipy
> pip install wxPython
> conda install matplotlib

Note: You may find that the particular versions of Python (3.8) and CUDA Toolkit (10.2) might not work for your particular computer setup. When this happens, replace these versions with those that work with your hardware. The command would become:

> conda create -n talking-head-anime-2-demo python=[YOUR-PYTHON-VERSION]
> conda activate talking-head-anime-2-demo
> conda install pytorch torchvision cudatoolkit=[YOUR-CUDA-TOOLKIT-VERSION] -c pytorch
> conda install scipy
> pip install wxPython
> conda install matplotlib

In general, the latest version of Python and the latest version of CUDA Toolkit shown on PyTorch's website would work.

To run the Jupyter notebook version of the manual_poser, you also need:

• Jupyter Notebook >= 6.2.0
• IPyWidgets >= 7.6.3

This means that, in addition to the commands above, you also need to run:

> conda install -c conda-forge notebook
> conda install -c conda-forge ipywidgets
> jupyter nbextension enable --py widgetsnbextension

Lastly, the ifacialmocap_puppeteer requires iFacialMocap, which is available in the App Store for 980 yen. You also need to install the paired desktop application on your PC or Mac. (Linux users, I'm sorry!) Your iOS and your computer must also use the same network. (For example, you may connect them to the same wireless router.)