Commit a72ec31c by vincent

added sections for face expression recognition

parent ffe26b32
...@@ -16,6 +16,7 @@ Table of Contents: ...@@ -16,6 +16,7 @@ Table of Contents:
* **[Face Detection Models](#models-face-detection)** * **[Face Detection Models](#models-face-detection)**
* **[68 Point Face Landmark Detection Models](#models-face-landmark-detection)** * **[68 Point Face Landmark Detection Models](#models-face-landmark-detection)**
* **[Face Recognition Model](#models-face-recognition)** * **[Face Recognition Model](#models-face-recognition)**
* **[Face Expression Recognition Model](#models-face-expression-recognition)**
* **[Getting Started](#getting-started)** * **[Getting Started](#getting-started)**
* **[face-api.js for the Browser](#getting-started-browser)** * **[face-api.js for the Browser](#getting-started-browser)**
* **[face-api.js for Nodejs](#getting-started-nodejs)** * **[face-api.js for Nodejs](#getting-started-nodejs)**
...@@ -61,6 +62,10 @@ Check out my face-api.js tutorials: ...@@ -61,6 +62,10 @@ Check out my face-api.js tutorials:
![preview_face-similarity](https://user-images.githubusercontent.com/31125521/40316573-0a1190c0-5d1f-11e8-8797-f6deaa344523.gif) ![preview_face-similarity](https://user-images.githubusercontent.com/31125521/40316573-0a1190c0-5d1f-11e8-8797-f6deaa344523.gif)
## Face Expression Recognition
![preview_face-expression-recognition](https://user-images.githubusercontent.com/31125521/50575270-f501d080-0dfb-11e9-9676-8f419efdade4.png)
## Face Landmark Detection ## Face Landmark Detection
![face_landmarks_boxes_2](https://user-images.githubusercontent.com/31125521/46063404-00928b00-c16d-11e8-8f29-e9c50afd2bc8.jpg) ![face_landmarks_boxes_2](https://user-images.githubusercontent.com/31125521/46063404-00928b00-c16d-11e8-8f29-e9c50afd2bc8.jpg)
...@@ -161,6 +166,12 @@ The neural net is equivalent to the **FaceRecognizerNet** used in [face-recognit ...@@ -161,6 +166,12 @@ The neural net is equivalent to the **FaceRecognizerNet** used in [face-recognit
The size of the quantized model is roughly 6.2 MB (**face_recognition_model**). The size of the quantized model is roughly 6.2 MB (**face_recognition_model**).
<a name="models-face-expression-recognition"></a>
## Face Expression Recognition Model
The face expression recognition model is lightweight, fast and provides reasonable accuracy. The model has a size of roughly 310kb and it employs depthwise separable convolutions and densely connected blocks. It has been trained on a variety of images from publicly available datasets as well as images scraped from the web. Note, that wearing glasses might decrease the accuracy of the prediction results.
<a name="getting-started"></a> <a name="getting-started"></a>
# Getting Started # Getting Started
...@@ -228,6 +239,7 @@ await faceapi.loadSsdMobilenetv1Model('/models') ...@@ -228,6 +239,7 @@ await faceapi.loadSsdMobilenetv1Model('/models')
// await faceapi.loadFaceLandmarkModel('/models') // await faceapi.loadFaceLandmarkModel('/models')
// await faceapi.loadFaceLandmarkTinyModel('/models') // await faceapi.loadFaceLandmarkTinyModel('/models')
// await faceapi.loadFaceRecognitionModel('/models') // await faceapi.loadFaceRecognitionModel('/models')
// await faceapi.loadFaceExpressionModel('/models')
``` ```
All global neural network instances are exported via faceapi.nets: All global neural network instances are exported via faceapi.nets:
...@@ -354,6 +366,42 @@ Detect the face with the highest confidence score in an image + computes 68 Poin ...@@ -354,6 +366,42 @@ Detect the face with the highest confidence score in an image + computes 68 Poin
const result = await faceapi.detectSingleFace(input).withFaceLandmarks().withFaceDescriptor() const result = await faceapi.detectSingleFace(input).withFaceLandmarks().withFaceDescriptor()
``` ```
### Recognizing Face Expressions
**Face expressions recognition can be performed for detected faces as follows:**
Detect all faces in an image + recognize face expressions. Returns **Array<[WithFaceExpressions<WithFaceDetection<{}>>](#usage-utility-classes)>**:
``` javascript
const detectionsWithExpressions = await faceapi.detectAllFaces(input).withFaceExpressions()
```
Detect the face with the highest confidence score in an image + recognize the face expression for that face. Returns **[WithFaceExpressions<WithFaceDetection<{}>>](#usage-utility-classes) | undefined**:
``` javascript
const detectionWithExpressions = await faceapi.detectSingleFace(input).withFaceExpressions()
```
### Composition of Tasks
**Tasks can be composed as follows:**
``` javascript
// all faces
await faceapi.detectAllFaces(input)
await faceapi.detectAllFaces(input).withFaceExpressions()
await faceapi.detectAllFaces(input).withFaceLandmarks()
await faceapi.detectAllFaces(input).withFaceExpressions().withFaceLandmarks()
await faceapi.detectAllFaces(input).withFaceExpressions().withFaceLandmarks().withFaceDescriptors()
// single face
await faceapi.detectSingleFace(input)
await faceapi.detectSingleFace(input).withFaceExpressions()
await faceapi.detectSingleFace(input).withFaceLandmarks()
await faceapi.detectSingleFace(input).withFaceExpressions().withFaceLandmarks()
await faceapi.detectSingleFace(input).withFaceExpressions().withFaceLandmarks().withFaceDescriptor()
```
### Face Recognition by Matching Descriptors ### Face Recognition by Matching Descriptors
To perform face recognition, one can use faceapi.FaceMatcher to compare reference face descriptors to query face descriptors. To perform face recognition, one can use faceapi.FaceMatcher to compare reference face descriptors to query face descriptors.
...@@ -611,6 +659,23 @@ export type WithFaceDescriptor<TSource> TSource & { ...@@ -611,6 +659,23 @@ export type WithFaceDescriptor<TSource> TSource & {
} }
``` ```
<a name="with-face-expressions"></a>
### WithFaceExpressions
``` javascript
export type FaceExpression = 'neutral' | 'happy' | 'sad' | 'angry' | 'fearful' | 'disgusted' | 'surprised'
export type FaceExpressionPrediction = {
expression: FaceExpression,
probability: number
}
export type WithFaceExpressions<TSource> TSource & {
expressions: FaceExpressionPrediction[]
}
```
<a name="other-useful-utility"></a> <a name="other-useful-utility"></a>
## Other Useful Utility ## Other Useful Utility
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment