Products

SIGN UPLOG IN

Models / Face attributes Detection

Face attributes Detection

Overview

The Faces Attributes Detection model is useful to determine if an image contains faces and get descriptive attributes on those faces (gender, sex, face landmarks, presence of sun glasses etc).

Face Detection

Face Detection has been designed to be efficient and robust, with a high precision/recall, supporting different face sizes, arbitrary face orientations as well as partially occluded faces, faces with sunglasses, hats or sanitary masks.

Use-cases

  • Detect people or faces in images or videos
  • Force users to upload pictures of their face
  • Prevent users from uploading group pictures
  • Group or classify your images/videos

To help you locate each individual face, the corners of the boxes containing those faces will be returned (x1,x2,y1,y2 values).

Face Features / Face Landmarks

For each detected face, the positions of the main face features are returned. Face features include:

  • Left and right eyes
  • Left and right mouth corners
  • Nose tip

Use-cases

  • Apply masks, transforms or other augmented-reality elements on faces
  • Adjust face crops based on the relative locations of key face landmarks

Gender Detection

For each detected face, the Face Attribute Model will return a "gender" field that will help you determine if a face is a male or female face, solely based on the characteristics of this face.

Gender properties are determined solely using the face. So other signs such as clothes or context will not influence the result. Males dressing up as females or females dressing up as males should therefore be correctly classified.

The API returns a "female" value and a "male" value. The sum of those are 1, each one being between 0 and 1. The largest of the two corresponds to the expected gender of the face. The closer to 1 it is, the higher the API's confidence.

Use-cases

  • Create profile data based on profile pictures
  • Check that users correctly entered their gender
  • Group or classify your images

Limitations

  • Gender detection is less accurate for babies and young infants
  • Make-up and fake mustaches or fake beards might trick the API, just like they might trick humans
  • Gender detection is less accurate if a very large proportion of the face is veiled or occluded

Example results

Female: (female value 0.99, male value 0.01)
Male: (female value 0.05, male value 0.95)

Age group Detection

For each detected face, the Face Attribute Model will return a "minor" field that will help you determine if a given face belongs to someone that is less than 18 years or more than 18 years old.

The 18-year threshold corresponds to the legal age of majority (adulthood) in many countries.

The Age Group information is determined solely using the face. Other signs such as clothes or context will not influence the result.

The returned value is between 0 and 1, face with a minor value closer to 1 indicate that the person is minor while faces with a minor value closer to 0 indicate that the person is major

Use-cases

  • Remove profile pictures belonging to minors
  • Prevent users from posting pictures /videos of their children
  • Blur child faces in images or videos
  • Group or classify your images based on content

15-20 year olds

Determining if someone is 17 or 19 based on a single face can be tricky. The API has therefore been developed to show that its confidence is low whenever it encounters pictures of users that are visually close to 18. The "minor" value would in this case be close to 0.5.

Minor: (minor value 0.93)
Major: (minor value 0.04)

Sunglasses

For each detected face, the Face Attribute Model will return a "sunglasses" field that will help you determine if a face is covered with sunglasses or not.

The returned value is between 0 and 1, face with a sunglasses value closer to 1 indicate that the person wear sunglasses while faces with a sunglasses value closer to 0 indicate that the person doesn't wear sunglasses.

Use-cases

  • Detect if a person is wearing sunglasses
  • Prevent users from hiding their faces with sunglasses
  • Group or classify your images

Limitations

  • The limitations applicable to face detection remain relevant
Sunglasses: (sunglasses value 0.92)
No Sunglasses: (sunglasses value 0.01)

Use the model (images)

If you haven't already, create an account to get your own API keys.

Detect faces in images

Let's say you want to moderate the following image:

You can either share a URL to the image, or upload the raw binary image.

Option 1: Send image URL

Here's how to proceed if you choose to share the image URL:


curl -X GET -G 'https://api.sightengine.com/1.0/check.json' \
    -d 'models=face-attributes' \
    -d 'api_user={api_user}&api_secret={api_secret}' \
    --data-urlencode 'url=https://sightengine.com/assets/img/examples/example7.jpg'


# this example uses requests
import requests
import json

params = {
  'url': 'https://sightengine.com/assets/img/examples/example7.jpg',
  'models': 'face-attributes',
  'api_user': '{api_user}',
  'api_secret': '{api_secret}'
}
r = requests.get('https://api.sightengine.com/1.0/check.json', params=params)

output = json.loads(r.text)


$params = array(
  'url' =>  'https://sightengine.com/assets/img/examples/example7.jpg',
  'models' => 'face-attributes',
  'api_user' => '{api_user}',
  'api_secret' => '{api_secret}',
);

// this example uses cURL
$ch = curl_init('https://api.sightengine.com/1.0/check.json?'.http_build_query($params));
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
$response = curl_exec($ch);
curl_close($ch);

$output = json_decode($response, true);


// this example uses axios
const axios = require('axios');

axios.get('https://api.sightengine.com/1.0/check.json', {
  params: {
    'url': 'https://sightengine.com/assets/img/examples/example7.jpg',
    'models': 'face-attributes',
    'api_user': '{api_user}',
    'api_secret': '{api_secret}',
  }
})
.then(function (response) {
  // on success: handle response
  console.log(response.data);
})
.catch(function (error) {
  // handle error
  if (error.response) console.log(error.response.data);
  else console.log(error.message);
});

See request parameter description

ParameterTypeDescription
mediabinaryimage to analyze
modelsstringcomma-separated list of models to apply
api_userstringyour API user id
api_secretstringyour API secret

Option 2: Send raw image

Here's how to proceed if you choose to upload the raw image:


curl -X POST 'https://api.sightengine.com/1.0/check.json' \
    -F 'media=@/path/to/image.jpg' \
    -F 'models=face-attributes' \
    -F 'api_user={api_user}' \
    -F 'api_secret={api_secret}'


# this example uses requests
import requests
import json

params = {
  'models': 'face-attributes',
  'api_user': '{api_user}',
  'api_secret': '{api_secret}'
}
files = {'media': open('/path/to/image.jpg', 'rb')}
r = requests.post('https://api.sightengine.com/1.0/check.json', files=files, data=params)

output = json.loads(r.text)


$params = array(
  'media' => new CurlFile('/path/to/image.jpg'),
  'models' => 'face-attributes',
  'api_user' => '{api_user}',
  'api_secret' => '{api_secret}',
);

// this example uses cURL
$ch = curl_init('https://api.sightengine.com/1.0/check.json');
curl_setopt($ch, CURLOPT_POST, true);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_POSTFIELDS, $params);
$response = curl_exec($ch);
curl_close($ch);

$output = json_decode($response, true);


// this example uses axios and form-data
const axios = require('axios');
const FormData = require('form-data');
const fs = require('fs');

data = new FormData();
data.append('media', fs.createReadStream('/path/to/image.jpg'));
data.append('models', 'face-attributes');
data.append('api_user', '{api_user}');
data.append('api_secret', '{api_secret}');

axios({
  method: 'post',
  url:'https://api.sightengine.com/1.0/check.json',
  data: data,
  headers: data.getHeaders()
})
.then(function (response) {
  // on success: handle response
  console.log(response.data);
})
.catch(function (error) {
  // handle error
  if (error.response) console.log(error.response.data);
  else console.log(error.message);
});

See request parameter description

ParameterTypeDescription
mediabinaryimage to analyze
modelsstringcomma-separated list of models to apply
api_userstringyour API user id
api_secretstringyour API secret

API response

The API will then return a JSON response with the following structure:

                  
                  
{
  "status": "success",
  "request": {
      "id": "req_0MsK5ptZx713xt5aRmckl",
      "timestamp": 1494406445.3718,
      "operations": 1
  },
  "faces": [
      {
          "x1": 0.5121,
          "y1": 0.1879,
          "x2": 0.6926,
          "y2": 0.6265,
          "features": {
              "left_eye": {
                  "x": 0.6438,
                  "y": 0.3634
              },
              "right_eye": {
                  "x": 0.5578,
                  "y": 0.3714
              },
              "nose_tip": {
                  "x": 0.6047,
                  "y": 0.4801
              },
              "left_mouth_corner": {
                  "x": 0.6469,
                  "y": 0.5305
              },
              "right_mouth_corner": {
                  "x": 0.5719,
                  "y": 0.5332
              }
          },
          "attributes": {
              "female": 0.96,
              "male": 0.04,
              "minor": 0.01,
              "sunglasses": 0.01
          }
      }
  ],
  "media": {
      "id": "med_0MsK3A6i2vNxQgHkc11j9",
      "uri": "https://sightengine.com/assets/img/examples/example7.jpg"
  }
}


              

Use model (Videos)

Detect faces in videos

Option 1: Short video

Here's how to proceed to analyze a short video (less than 1 minute):


curl -X POST 'https://api.sightengine.com/1.0/video/check-sync.json' \
  -F 'media=@/path/to/video.mp4' \
  -F 'models=face-attributes' \
  -F 'api_user={api_user}' \
  -F 'api_secret={api_secret}'


# this example uses requests
import requests
import json

params = {
  # specify the models you want to apply
  'models': 'face-attributes',
  'api_user': '{api_user}',
  'api_secret': '{api_secret}'
}
files = {'media': open('/path/to/video.mp4', 'rb')}
r = requests.post('https://api.sightengine.com/1.0/video/check-sync.json', files=files, data=params)

output = json.loads(r.text)


$params = array(
  'media' => new CurlFile('/path/to/video.mp4'),
  // specify the models you want to apply
  'models' => 'face-attributes',
  'api_user' => '{api_user}',
  'api_secret' => '{api_secret}',
);

// this example uses cURL
$ch = curl_init('https://api.sightengine.com/1.0/video/check-sync.json');
curl_setopt($ch, CURLOPT_POST, true);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_POSTFIELDS, $params);
$response = curl_exec($ch);
curl_close($ch);

$output = json_decode($response, true);


// this example uses axios and form-data
const axios = require('axios');
const FormData = require('form-data');
const fs = require('fs');

data = new FormData();
data.append('media', fs.createReadStream('/path/to/video.mp4'));
// specify the models you want to apply
data.append('models', 'face-attributes');
data.append('api_user', '{api_user}');
data.append('api_secret', '{api_secret}');

axios({
  method: 'post',
  url:'https://api.sightengine.com/1.0/video/check-sync.json',
  data: data,
  headers: data.getHeaders()
})
.then(function (response) {
  // on success: handle response
  console.log(response.data);
})
.catch(function (error) {
  // handle error
  if (error.response) console.log(error.response.data);
  else console.log(error.message);
});

See request parameter description

ParameterTypeDescription
mediabinaryimage to analyze
modelsstringcomma-separated list of models to apply
intervalfloatframe interval in seconds, out of 0.5, 1, 2, 3, 4, 5 (optional)
api_userstringyour API user id
api_secretstringyour API secret

Option 2: Long video

Here's how to proceed to analyze a long video. Note that if the video file is very large, you might first need to upload it through the Upload API.


curl -X POST 'https://api.sightengine.com/1.0/video/check.json' \
  -F 'media=@/path/to/video.mp4' \
  -F 'models=face-attributes' \
  -F 'callback_url=https://yourcallback/path' \
  -F 'api_user={api_user}' \
  -F 'api_secret={api_secret}'


# this example uses requests
import requests
import json

params = {
  # specify the models you want to apply
  'models': 'face-attributes',
  # specify where you want to receive result callbacks
  'callback_url': 'https://yourcallback/path',
  'api_user': '{api_user}',
  'api_secret': '{api_secret}'
}
files = {'media': open('/path/to/video.mp4', 'rb')}
r = requests.post('https://api.sightengine.com/1.0/video/check.json', files=files, data=params)

output = json.loads(r.text)


$params = array(
  'media' => new CurlFile('/path/to/video.mp4'),
  // specify the models you want to apply
  'models' => 'face-attributes',
  // specify where you want to receive result callbacks
  'callback_url' => 'https://yourcallback/path',
  'api_user' => '{api_user}',
  'api_secret' => '{api_secret}',
);

// this example uses cURL
$ch = curl_init('https://api.sightengine.com/1.0/video/check.json');
curl_setopt($ch, CURLOPT_POST, true);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_POSTFIELDS, $params);
$response = curl_exec($ch);
curl_close($ch);

$output = json_decode($response, true);


// this example uses axios and form-data
const axios = require('axios');
const FormData = require('form-data');
const fs = require('fs');

data = new FormData();
data.append('media', fs.createReadStream('/path/to/video.mp4'));
// specify the models you want to apply
data.append('models', 'face-attributes');
// specify where you want to receive result callbacks
data.append('callback_url', 'https://yourcallback/path');
data.append('api_user', '{api_user}');
data.append('api_secret', '{api_secret}');

axios({
  method: 'post',
  url:'https://api.sightengine.com/1.0/video/check.json',
  data: data,
  headers: data.getHeaders()
})
.then(function (response) {
  // on success: handle response
  console.log(response.data);
})
.catch(function (error) {
  // handle error
  if (error.response) console.log(error.response.data);
  else console.log(error.message);
});

See request parameter description

ParameterTypeDescription
mediabinaryimage to analyze
callback_urlstringcallback URL to receive moderation updates (optional)
modelsstringcomma-separated list of models to apply
intervalfloatframe interval in seconds, out of 0.5, 1, 2, 3, 4, 5 (optional)
api_userstringyour API user id
api_secretstringyour API secret

Option 3: Live-stream

Here's how to proceed to analyze a live-stream:


curl -X GET -G 'https://api.sightengine.com/1.0/video/check.json' \
    --data-urlencode 'stream_url=https://domain.tld/path/video.m3u8' \
    -d 'models=face-attributes' \
    -d 'callback_url=https://your.callback.url/path' \
    -d 'api_user={api_user}' \
    -d 'api_secret={api_secret}'


# if you haven't already, install the SDK with 'pip install sightengine'
from sightengine.client import SightengineClient
client = SightengineClient('{api_user}','{api_secret}')
output = client.check('face-attributes').video('https://domain.tld/path/video.m3u8', 'https://your.callback.url/path')


// if you haven't already, install the SDK with 'composer require sightengine/client-php'
use \Sightengine\SightengineClient;
$client = new SightengineClient('{api_user}','{api_secret}');
$output = $client->check(['face-attributes'])->video('https://domain.tld/path/video.m3u8', 'https://your.callback.url/path');


// if you haven't already, install the SDK with 'npm install sightengine --save'
var sightengine = require('sightengine')('{api_user}', '{api_secret}');
sightengine.check(['face-attributes']).video('https://domain.tld/path/video.m3u8', 'https://your.callback.url/path').then(function(result) {
    // The API response (result)
}).catch(function(err) {
    // Handle error
});

See request parameter description

ParameterTypeDescription
stream_urlstringURL of the video stream
callback_urlstringcallback URL to receive moderation updates (optional)
modelsstringcomma-separated list of models to apply
intervalfloatframe interval in seconds, out of 0.5, 1, 2, 3, 4, 5 (optional)
api_userstringyour API user id
api_secretstringyour API secret

Moderation result

The Moderation result will be provided either directly in the request response (for sync calls, see below) or through the callback URL your provided (for async calls).

Here is the structure of the JSON response with moderation results for each analyzed frame under the data.frames array:

            
                  
{
  "status": "success",
    "request": {
    "id": "req_gmgHNy8oP6nvXYaJVLq9n",
    "timestamp": 1717159864.348989,
    "operations": 21
  },
  "data": {
  "frames": [
    {
      "info": {
        "id": "med_gmgHcUOwe41rWmqwPhVNU_1",
        "position": 0
      },
      "faces": [
        {
            "x1": 0.5121,
            "y1": 0.1879,
            "x2": 0.6926,
            "y2": 0.6265,
            "features": {
                "left_eye": {
                    "x": 0.6438,
                    "y": 0.3634
                },
                "right_eye": {
                    "x": 0.5578,
                    "y": 0.3714
                },
                "nose_tip": {
                    "x": 0.6047,
                    "y": 0.4801
                },
                "left_mouth_corner": {
                    "x": 0.6469,
                    "y": 0.5305
                },
                "right_mouth_corner": {
                    "x": 0.5719,
                    "y": 0.5332
                }
            },
            "attributes": {
                "female": 0.96,
                "male": 0.04,
                "minor": 0.01,
                "sunglasses": 0.01
            }
        }
      ],
     },
     ...
    ]
  },
  "media": {
    "id": "med_gmgHcUOwe41rWmqwPhVNU",
    "uri": "yourfile.mp4"
  },
}


            

You can use the classes under the faces object to analyze faces in the video.

Any other needs?

See our full list of Image/Video models for details on other filters and checks you can run on your images and videos. You might also want to check our Text models to moderate text-based content: messages, reviews, comments, usernames...

Was this page helpful?