Creating real-time facial and emotion recognition software to pair with Unreal Engine 5 Metahuman to predict and mimic user emotions.
Our objective is to develop a 3D Virtual Human (3DVH) prototype that can mirror a user’s emotional reaction. The user emotional reaction is captured using a computer vision-based technique and a camera system. The virtual human will mimic user emotions in real-time.
- Python Scripting: Prem Patel (@Prem-ium)
- Unreal Engine Integration: Gabe Vindas (@GabeV95) & Dustin Lynn (@Onemorehell)
- Emotion Animations in Unreal Engine: Matthew Goetz
- Anger 😡
- Fear 😨
- Happy 😄
- Neutral 😐
- Sad 😢
- Surprised 😲
They're terrifying, we know.
(Showcasing Happy & Surprised)
- Emotion Recognition
- Age & Gender Prediction
- Enviornmental Variables Customization
- Headless mode
- Flexible Emotion Selection: Choose between the most common emotion every X captured emotions or the latest emotion.
Run locally:
- Clone this repository, cd into it, and install dependancies:
git clone https://github.com/Prem-ium/Metahuman-Emotion-Recognition.git
cd EmotionDetection
pip install -r requirements.txt- Configure your
.envfile (See below and example for options) - Run the main script:
python emotional-detection-main.py- Open Unreal Engine Project & Run the Blueprint
- Click the button to trigger the text reader to process the most common emotion recorded.
- The Metahuman mimics the user's most common emotion.
- Repeat Steps 5-6 until desired termination
Configure your variables in a .env file within the same directory. All .env variables are optional and have default values if not specified.
| Variable | Description | Default Value |
|---|---|---|
| HEADLESS | True or False. Whether to open a GUI for testing webcam accuracy. | True |
| PRODUCTION | True or False. Whether the program is running in Unreal Engine. | False |
| DELAY | Integer value of how many seconds the program will wait before starting the next iteration | - |
| FILE_PATH | Path of directory containing the model and weights | - |
| WEIGHTS | Name of the model being used | - |
I regret to inform you that I'm currently unable to respond to any issues for this repository. I have disabled the ability to start an issue in this repository. Please refrain from contacting me regarding any issues at this time.
