Our face reconstruction C++ pipeline aims to obtain parameters for a morphable face model that matches a given RGB/RGB-D image. It utilizes a PCA-based Basel Face Model encompassing shape, expression, albedo, illumination, and pose parameters.
The reconstruction process follows an analysis-by-synthesis approach, updating parameters to minimize an overall energy function using Ceres. The energy function includes dense and sparse terms, incorporating geometry and color comparisons between the rendered face and the actual image.
- OpenCV
- Eigen3
- Glog
- GLFW
- PCL
- Boost
- Ceres
Transfer expression example. We find parameters independently using source (top left) and target (bottom left) actors. Then, we transfer expression coefficients from the source into the target. Finally, we project the source image into the mesh and visualize the mask above the source image.
Results of the optimization pipeline using RGB only. In the center column, we apply a BFM model texture. The last column obtains texture from the projection mesh into the image.
mkdir build && cd build
cmake ..
makeYou can read the final report where explained our method here.
- Gökçe Şengün
- Dmitrii Pozdev
- Biray Sütçüoğlu
- David Gichev
