Skip to content

Commit

Permalink
Facemesh v2 (#1437)
Browse files Browse the repository at this point in the history
* feat(Facemesh): it now looks in the same direction as the face

* fix: avoiding creating disposable objects imperatively

* feat: origin landmark prop

* fix: initial useState value as function initializer
  • Loading branch information
abernier authored May 10, 2023
1 parent a34aff0 commit f8fa46c
Show file tree
Hide file tree
Showing 3 changed files with 210 additions and 227 deletions.
12 changes: 8 additions & 4 deletions .storybook/stories/Facemesh.stories.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -20,28 +20,32 @@ export default {
],
}

export const FacemeshSt = ({ width, flipY, wireframe, flat, skin }) => (
export const FacemeshSt = ({ depth, origin, wireframe, flat, skin, debug }) => (
<>
<color attach="background" args={['#303030']} />
<axesHelper />

<Facemesh width={width} rotation-y={Math.PI}>
<Facemesh depth={depth} origin={origin} debug={debug} rotation-z={Math.PI}>
<meshStandardMaterial side={THREE.DoubleSide} color={skin} flatShading={flat} wireframe={wireframe} />
</Facemesh>
</>
)
FacemeshSt.args = {
width: undefined,
depth: undefined,
origin: undefined,
wireframe: false,
flat: true,
skin: '#cbcbcb',
debug: true,
}

FacemeshSt.argTypes = {
width: { control: { type: 'range', min: 0, max: 6.5, step: 0.01 } },
depth: { control: { type: 'range', min: 0, max: 6.5, step: 0.01 } },
origin: { control: 'select', options: [undefined, 168, 9] },
wireframe: { control: { type: 'boolean' } },
flat: { control: { type: 'boolean' } },
skin: { control: { type: 'color' } },
debug: { control: { type: 'boolean' } },
}

FacemeshSt.storyName = 'Default'
33 changes: 32 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -961,13 +961,44 @@ const face = {
type FacemeshProps = {
/** a MediaPipeFaceMesh object, default: a lambda face */
face?: MediaPipeFaceMesh
/** width of the mesh, default: 1 */
/** width of the mesh, default: undefined */
width?: number
/** or height of the mesh, default: undefined */
height?: number
/** or depth of the mesh, default: 1 */
depth?: number
/** a landmarks tri supposed to be vertical, default: [159, 386, 200] (see: https://github.com/tensorflow/tfjs-models/tree/master/face-landmarks-detection#mediapipe-facemesh-keypoints) */
verticalTri?: [number, number, number]
/** a landmark index to be the origin of the mesh. default: undefined (ie. the bbox center) */
origin?: number
/** debug mode, default: false */
debug?: boolean
}
```

Ref-api:

```tsx
const api = useRef<FacemeshApi>()
const {meshRef, outerRef} = api
<Facemesh ref={api} face={face} />
```

```tsx
type FacemeshApi = {
meshRef: React.RefObject<THREE.Mesh>
outerRef: React.RefObject<THREE.Group>
}
```

NB: `outerRef` group is oriented as your `face`. You can for example gets its direction:

```tsx
meshRef.current.localToWorld(new THREE.Vector3(0, 0, -1))
```

# Abstractions

#### Image
Expand Down
Loading

1 comment on commit f8fa46c

@vercel
Copy link

@vercel vercel bot commented on f8fa46c May 10, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please sign in to comment.