# TetraTSDF

**Repository Path**: aiacLab/TetraTSDF

## Basic Information

- **Project Name**: TetraTSDF
- **Description**: No description available
- **Primary Language**: Unknown
- **License**: Not specified
- **Default Branch**: master
- **Homepage**: None
- **GVP Project**: No

## Statistics

- **Stars**: 0
- **Forks**: 0
- **Created**: 2020-10-30
- **Last Updated**: 2022-03-09

## Categories & Tags

**Categories**: Uncategorized

**Tags**: None

## README

# TetraTSDF

Volumetric TSDF regression with tetrahedral representation for 3D human body prediction from single color image.


## Requirements

In this code, we used 2 versions of the python.

Dataset and mesh generation

- python 2.7

- binascii

- pymesh (https://pymesh.readthedocs.io/en/latest/installation.html#download-the-source)

- pyflann

Network training and testing

- python 3.6

- CUDA 10.0  (Our code maybe works on other CUDA versions)

- keras with TensorFlow background

Evaluation

- python 3.6 and 2.7

- shutil

- Open3D

Model data
- Download data for coarse models at:  https://archive.iii.kyushu-u.ac.jp/public/ljLUAANJM8BACNsBxqd16-g-mb1eC1VTBZXKTCR30CvL
- Pretrained (on ARTICULATED) model is available at: https://archive.iii.kyushu-u.ac.jp/public/4j1EAAUJ6YBAuSYB2bZ15AE_hUMJnm0thlds6huPCsI5

## Dataset generation

- Coarse human outer shell

We need to prepare coarse human outer shell for dataset generation.

In our paper, we generated it based on male SMPL model by using Blender.

(the coarse human is already contained in the code)

if you want to use another coarse human (generated by yourself), see the README in ./coarsehuman folder

### Articulated dataset
1. Download Articulated dataset from http://people.csail.mit.edu/drdaniel/mesh_animation/ to somewhere in your computer  
We only used crane, bouncing, jumping, march 2 sequence that we could get correct SMPL ground truth pose parameters and weights for re-pose.

2. Unzip images and meshes folder  

```
[Articulated dataset root]  
      ├─D_bouncing
      ├─D_march
      ├─I_crane
      └─I_jumping
              ├─images
              └─meshes
```

3. Generate TSDF

```
cd [TetraTSDF root]/coarsehuman/gendata_Articulated
python genTSDF.py
```

4. Crop images into 256x256  
Execute cropimg.py on "images" directory in each sequence
```
python cropimg.py --imgdir [path to images directory] --skeletondir ./smplfitresults_articulated/[sequence]/openpose_joints
```
We used OpenPose joints to centering & scaling the images (The person in the image covers about 65% of the image height).
Articulated dataset contains images from 8 views for each sequence. We selected one of them for training.

## Network training
Run main.py with python3.6 environment    
```
cd [TetraTSDF root]/network
python main.py --mode 0 --datasetroot [Path to Articulated dataset root]
```

## Testing
```
cd [TetraTSDF root]/network
python main.py --mode 1 (--imgpath_pred [Path to test images(file or dir)])
```
TSDF data (.bin) is saved to [TetraTSDF root]/network/result  
Then execute TSDF2mesh.py to visualize the tsdf
```
(python2.7 environment)
cd [TetraTSDF root]/visualize
python TSDF2mesh.py (--tsdfpath [path to tsdf] --parampath [path to smplparam])
```

## Evaluation
```
cd [TetraTSDF root]/network
python main.py --mode 2 --datasetroot [Path to Articulated dataset root]
```
Then execute TSDF2mesh.py to visualize the tsdf
```
(python2.7 environment)
cd [TetraTSDF root]/visualize
python TSDF2chamfer.py --TSDFdir_pred ../network/for_evaluation/TSDF_pred --TSDFdir_GT ../network/for_evaluation/TSDF_GT --paramdir_pred ../network/for_evaluation/params --paramdir_GT ../network/for_evaluation/params
```

## Reference
This work is to be used for educational and research purpose only. If you use part of this work, you are required to refer our paper:

```
@inproceedings{Onizuka2020,
  title={TetraTSDF: 3D human reconstruction from a single image with a tetrahedral outer shell},
  author={Onizuka, Hayato and Haiyrci, Zehra and Thomas, Diego and Sugimoto, Akihiro and Uchiyama, Hideaki and Taniguchi, Rin-Ichiro},
  booktitle={Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition},
  pages={--},
  year={2020}
}
```

## Issues
Since our system has been trained on small dataset, the generalizability is very low. To have more generalizability, our system needs more dataset.