Week 4 2023 Spring

Homework:

• Reading:

  • face-api.js — JavaScript API for Face Recognition in the Browser with tensorflow.js
  • Face and hand tracking in the browser with MediaPipe and TensorFlow.js

• Coding:

  • Build an interactive browser experiment on face or hand data
  • Publish your project on GitHub or your own blog, or record a video and put it on your blog. Add your project link below.

See demos live

• Face-api
  • FaceApi_Image_Landmarks
  • FaceApi_Video_Landmarks
• Face Mesh
  • Facemesh_Image
  • Facemesh_Webcam
  • Cat face filter, code
• HandPose
  • Handpose_Image
  • Handpose_Webcam
• Training hand pose model
  • Hand pose tracking + KNN Classification, p5 web editor code, demo video
  • Hand pose tracking + Neural Network, demo video
    • Collect data
    • Train the model
    • Run the model
• Multiple hands detection:
  • Handpose multiple hands with mediapipe If the webcam video is not loading, try open a incognito window, or set the camera access for the browser.
  • KNN Classifier on multiple hands: Change HAND_NUM = 2 if you would like to detect other hand numbers. Once clicking on "Add example" button, it will start collecting data in 5 seconds for 5 seconds.

Resource

• ml5js - faceapi
• ml5js - faceMesh source
• ml5js - handPose source
• tfjs - faceapi-js
• tfjs - faceMesh
• tfjs - handPose

Projects

• LipSync game using Face Mesh model
• Customizable AR face masks - Made with TensorFlow.js
• fingerspelling

Questions?

• add your question here

Add your name / link to your assignment below

Spencer Harris: https://www.notion.so/spencerharris/Week-4-Handpose-Generative-Poetry-9498c25767d74af592e34eb042c2f74a

Dror https://www.drormargalit.com/post/i-will-always-love-you

Athena, Color Edit

Sim, Traffic Controller Simulator, Blog Post

Annan, [Blowing balloon]

Yizhi, Buzz

Keyi, Week 5

Shauna, Laser Eyes