TLDR

Heimdall is an open source face recognition plattform. It's build on a microservice architecture based on docker and uses the powerful face recognition model from the Dlib-library by Davis King, as well as the Histogram of Oriented Gradients face detection model. Classification is done with a SVM classifier from Scikit-Learn. It also features a native Android App as well as an WebInterface written in React. Installation Instructions can be found here: Installation.md

Heimdall

Heimdall is an open source face recognition plattform. It was designed especially to work with a peephole camera in order to recognize persons ringing the doorbell. This is not only very useful for people who are too lazy to get up just to know who is in front of their door, but also for elderly people that suffer from dementia. A first prototype of heimdall was created as part of the Masterthesis "Neuronale Netze und andere Verfahren zur Gesichtserkennung im Heimautomatisierungsfumeld" by Constantin Kirsch for the Master of Science in Computer Science at the Bonn-Rhine-Sieg University of Applied Sciences at the Multimedia Communication Laboratory. This fork continous his work by implementing a microservice architecture, several security features as well as improving the usability. It uses the powerful face recognition model from the Dlib-library by Davis King, as well as the Histogram of Oriented Gradients face detection model. Classification is done with a SVM classifier from Scikit-Learn. The face detection, feature extraction and training is embedded into a Flask based Webapplication.

Installation & Usage

Installation and Usage Instructions can be found in the corresponding Installation instructions. If you have docker installed it's basically only one command.

TODO

• Camera

  • Disable DHCP to allow the first image to be transmitted earlier/faster

• UI

  • Improve visibility of progressbar in React App (training progress)
  • Blue mark stays on even after classifying an image
  • Improve instructions on what to do after opening the timeline
  • LiveView - naming is misleading/confusing - change to something like "DetectionView"?
  • Reload page when clicking on the icon in the sidebar (if the page is already open the page won't reload)
  • Update API access to use the getImageUrl from HTTPClient
  • only show images with one person (in classification view)

• remove heimdall-backend, heimdall-frontend and emq-docker dependency from this repo (the images are included via dockerhub - dependency is only used for debugging purposes)

• Implement some kind of housekeeping operation (delete old images) e.g.

  • after a training delete all images (good for performance)
  • delete images that are older than 10 days
  • keep 100 (unclassified) images

• Disable redis public port

• Facial blur detection - Remove blurry images (difficult to implement since amount of blur depends on many factors)

• Write frontend tests (read readme in heimdall-frontend for information on react testing)

• Test resized images from API / Add special api for resized images

• Check exposed ports in Dockerfiles / Docker-Compose

• Cleanup backend (Remove unused imports / Refactor code)

• Better error handling (Push error messages to the client)

• Use EMQ - MQTT Broker EMQTT

  • Docker Container
  • Example of sending images from RPI via MQTT

• API Endpoint /events very slow

• Update Mosquitto to 1.4.14

• Use paho-mqtt in backend and browser

• Setup nginx in front of backend (flask)

• Informationen in Android-App sollen nach 5min verschwinden

• Weißer Rand am unteren Bildschirmrand in Webseite. Sonst sieht es so aus, als ob es noch weiter ginge

• Automatisch zu angeklicktem Bild springen (und nicht zu erstem aus dem Event)

• Beim Anlegen der Person diese auch automatisch markieren

• Bilder laden geht nicht immer korrekt!! (In der Klassifizierungsansicht) (image key is missing)

• Löschen einer Klassifikation löschen können (nochmal auf Namen drücken können um ein Bild löschen zu können)

• Gallerie --> Popup einbauen und Bilder entfernen können --> Und neu zuordnen können

• Use VisualStudio debugger-for-chrome