Products

SIGN UPLOG IN

Models / Drug Detection

Recreational Drug and Medical Drug Detection

Description of concepts

The API returns confidence scores for the following concepts:

ConceptDescription

Recreational drugs

recreational_drug.prob

Photos and illustrations containing recreational drugs. This includes:

  • Powder-like drugs such as cocaine, ketamine, where indicated by context (in line, in pouch) or by usage (snorting)
  • Self-administration through syringes of drugs such as heroin
  • Cannabis buds, joints and related products
  • Cannabis leaf or plant
  • Bongs, pipes, or other paraphernalia related to marijuana consumption
  • Pills and pill bottles in a recreational context (i.e. unlikely to be for medical use, based on quantity, color/shape or context)

Cannabis

cannabis

Any reference to cannabis/marijuana. This includes:

  • Cannabis logo (cannabis leaf)
  • Cannabis plant
  • Cannabis drug (joints, haschich)

Cannabis logo (only)

cannabis_logo_only

Image where the only reference to cannabis is through a symbol designed to represent marijuana (think of the cannabis leaf outline). This typically appears on objects, apparels or as logos super-imposed on the image. If other references to marijuana are present, then this class will not trigger.

Cannabis plant

cannabis_plant

Marijuana plants.

Cannabis drug

cannabis_drug

Displays of weed, marijuana buds, joints, blunts. This also includes marijuana-related paraphernalia such as pipes, bongs.

Recreational drug excluding cannabis

recreational_drug_not_cannabis

Photos and illustrations:

  • Snorting of drugs such as cocaine, ketamine
  • Self-administration through syringes of drugs such as heroin

Medical pills

pills

Pills and pill bottles in a medical context.

Medical paraphernalia

paraphernalia

Medical syringes or medical equipment. This does not include syringes in the context of recreational drugs such as self-administration.

Excluded concepts

The API will ignore situations where regular tobacco is smoked. This means that the following concepts would not be flagged as drug:

  • Regular Tobacco
  • Industrial cigarettes, cigarette packs
  • People smoking or puffing, unless there are references to recreational drugs
  • Smoking pipes used with regular tobacco
  • Hookahs, shishas

To detect tobacco smoking and tobacco products, please refer to the Smoking & Tobacco detection model.

Use the model (images)

If you haven't already, create an account to get your own API keys.

Detect drugs

Let's say you want to moderate the following image:

You can either share a URL to the image, or upload the raw binary image.

Option 1: Send image URL

Here's how to proceed if you choose to share the image URL:


curl -X GET -G 'https://api.sightengine.com/1.0/check.json' \
    -d 'models=recreational_drug,medical' \
    -d 'api_user={api_user}&api_secret={api_secret}' \
    --data-urlencode 'url=https://sightengine.com/assets/img/examples/cann.jpg'


# this example uses requests
import requests
import json

params = {
  'url': 'https://sightengine.com/assets/img/examples/cann.jpg',
  'models': 'recreational_drug,medical',
  'api_user': '{api_user}',
  'api_secret': '{api_secret}'
}
r = requests.get('https://api.sightengine.com/1.0/check.json', params=params)

output = json.loads(r.text)


$params = array(
  'url' =>  'https://sightengine.com/assets/img/examples/cann.jpg',
  'models' => 'recreational_drug,medical',
  'api_user' => '{api_user}',
  'api_secret' => '{api_secret}',
);

// this example uses cURL
$ch = curl_init('https://api.sightengine.com/1.0/check.json?'.http_build_query($params));
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
$response = curl_exec($ch);
curl_close($ch);

$output = json_decode($response, true);


// this example uses axios
const axios = require('axios');

axios.get('https://api.sightengine.com/1.0/check.json', {
  params: {
    'url': 'https://sightengine.com/assets/img/examples/cann.jpg',
    'models': 'recreational_drug,medical',
    'api_user': '{api_user}',
    'api_secret': '{api_secret}',
  }
})
.then(function (response) {
  // on success: handle response
  console.log(response.data);
})
.catch(function (error) {
  // handle error
  if (error.response) console.log(error.response.data);
  else console.log(error.message);
});

See request parameter description

ParameterTypeDescription
mediabinaryimage to analyze
modelsstringcomma-separated list of models to apply
api_userstringyour API user id
api_secretstringyour API secret

Option 2: Send raw image

Here's how to proceed if you choose to upload the raw image:


curl -X POST 'https://api.sightengine.com/1.0/check.json' \
    -F 'media=@/path/to/image.jpg' \
    -F 'models=recreational_drug,medical' \
    -F 'api_user={api_user}' \
    -F 'api_secret={api_secret}'


# this example uses requests
import requests
import json

params = {
  'models': 'recreational_drug,medical',
  'api_user': '{api_user}',
  'api_secret': '{api_secret}'
}
files = {'media': open('/path/to/image.jpg', 'rb')}
r = requests.post('https://api.sightengine.com/1.0/check.json', files=files, data=params)

output = json.loads(r.text)


$params = array(
  'media' => new CurlFile('/path/to/image.jpg'),
  'models' => 'recreational_drug,medical',
  'api_user' => '{api_user}',
  'api_secret' => '{api_secret}',
);

// this example uses cURL
$ch = curl_init('https://api.sightengine.com/1.0/check.json');
curl_setopt($ch, CURLOPT_POST, true);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_POSTFIELDS, $params);
$response = curl_exec($ch);
curl_close($ch);

$output = json_decode($response, true);


// this example uses axios and form-data
const axios = require('axios');
const FormData = require('form-data');
const fs = require('fs');

data = new FormData();
data.append('media', fs.createReadStream('/path/to/image.jpg'));
data.append('models', 'recreational_drug,medical');
data.append('api_user', '{api_user}');
data.append('api_secret', '{api_secret}');

axios({
  method: 'post',
  url:'https://api.sightengine.com/1.0/check.json',
  data: data,
  headers: data.getHeaders()
})
.then(function (response) {
  // on success: handle response
  console.log(response.data);
})
.catch(function (error) {
  // handle error
  if (error.response) console.log(error.response.data);
  else console.log(error.message);
});

See request parameter description

ParameterTypeDescription
mediabinaryimage to analyze
modelsstringcomma-separated list of models to apply
api_userstringyour API user id
api_secretstringyour API secret

API response

The API will then return a JSON response with the following structure:

                  
                  
{
    "status": "success",
    "request": {
        "id": "req_gcTp4s63IAAni0lFOT7KK",
        "timestamp": 1714997478.552115,
        "operations": 1
    },
    "recreational_drug": {
        "prob": 0.99,
        "classes": {
            "cannabis": 0.99,
            "cannabis_logo_only": 0.01,
            "cannabis_plant": 0.01,
            "cannabis_drug": 0.99,
            "recreational_drugs_not_cannabis": 0.01
        }
    },
    "medical": {
        "prob": 0.01,
        "classes": {
            "pills": 0.01,
            "paraphernalia": 0.01
        }
    },
    "media": {
        "id": "med_gcTpqyOZ18IMsiMe4Ar28",
        "uri": "https://sightengine.com/assets/img/examples/cann.jpg"
    }
}


              

Use model (Videos)

Detecting Drugs in videos

Option 1: Short video

Here's how to proceed to analyze a short video (less than 1 minute):


curl -X POST 'https://api.sightengine.com/1.0/video/check-sync.json' \
  -F 'media=@/path/to/video.mp4' \
  -F 'models=recreational_drug,medical' \
  -F 'api_user={api_user}' \
  -F 'api_secret={api_secret}'


# this example uses requests
import requests
import json

params = {
  # specify the models you want to apply
  'models': 'recreational_drug,medical',
  'api_user': '{api_user}',
  'api_secret': '{api_secret}'
}
files = {'media': open('/path/to/video.mp4', 'rb')}
r = requests.post('https://api.sightengine.com/1.0/video/check-sync.json', files=files, data=params)

output = json.loads(r.text)


$params = array(
  'media' => new CurlFile('/path/to/video.mp4'),
  // specify the models you want to apply
  'models' => 'recreational_drug,medical',
  'api_user' => '{api_user}',
  'api_secret' => '{api_secret}',
);

// this example uses cURL
$ch = curl_init('https://api.sightengine.com/1.0/video/check-sync.json');
curl_setopt($ch, CURLOPT_POST, true);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_POSTFIELDS, $params);
$response = curl_exec($ch);
curl_close($ch);

$output = json_decode($response, true);


// this example uses axios and form-data
const axios = require('axios');
const FormData = require('form-data');
const fs = require('fs');

data = new FormData();
data.append('media', fs.createReadStream('/path/to/video.mp4'));
// specify the models you want to apply
data.append('models', 'recreational_drug,medical');
data.append('api_user', '{api_user}');
data.append('api_secret', '{api_secret}');

axios({
  method: 'post',
  url:'https://api.sightengine.com/1.0/video/check-sync.json',
  data: data,
  headers: data.getHeaders()
})
.then(function (response) {
  // on success: handle response
  console.log(response.data);
})
.catch(function (error) {
  // handle error
  if (error.response) console.log(error.response.data);
  else console.log(error.message);
});

See request parameter description

ParameterTypeDescription
mediabinaryimage to analyze
modelsstringcomma-separated list of models to apply
intervalfloatframe interval in seconds, out of 0.5, 1, 2, 3, 4, 5 (optional)
api_userstringyour API user id
api_secretstringyour API secret

Option 2: Long video

Here's how to proceed to analyze a long video. Note that if the video file is very large, you might first need to upload it through the Upload API.


curl -X POST 'https://api.sightengine.com/1.0/video/check.json' \
  -F 'media=@/path/to/video.mp4' \
  -F 'models=recreational_drug,medical' \
  -F 'callback_url=https://yourcallback/path' \
  -F 'api_user={api_user}' \
  -F 'api_secret={api_secret}'


# this example uses requests
import requests
import json

params = {
  # specify the models you want to apply
  'models': 'recreational_drug,medical',
  # specify where you want to receive result callbacks
  'callback_url': 'https://yourcallback/path',
  'api_user': '{api_user}',
  'api_secret': '{api_secret}'
}
files = {'media': open('/path/to/video.mp4', 'rb')}
r = requests.post('https://api.sightengine.com/1.0/video/check.json', files=files, data=params)

output = json.loads(r.text)


$params = array(
  'media' => new CurlFile('/path/to/video.mp4'),
  // specify the models you want to apply
  'models' => 'recreational_drug,medical',
  // specify where you want to receive result callbacks
  'callback_url' => 'https://yourcallback/path',
  'api_user' => '{api_user}',
  'api_secret' => '{api_secret}',
);

// this example uses cURL
$ch = curl_init('https://api.sightengine.com/1.0/video/check.json');
curl_setopt($ch, CURLOPT_POST, true);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_POSTFIELDS, $params);
$response = curl_exec($ch);
curl_close($ch);

$output = json_decode($response, true);


// this example uses axios and form-data
const axios = require('axios');
const FormData = require('form-data');
const fs = require('fs');

data = new FormData();
data.append('media', fs.createReadStream('/path/to/video.mp4'));
// specify the models you want to apply
data.append('models', 'recreational_drug,medical');
// specify where you want to receive result callbacks
data.append('callback_url', 'https://yourcallback/path');
data.append('api_user', '{api_user}');
data.append('api_secret', '{api_secret}');

axios({
  method: 'post',
  url:'https://api.sightengine.com/1.0/video/check.json',
  data: data,
  headers: data.getHeaders()
})
.then(function (response) {
  // on success: handle response
  console.log(response.data);
})
.catch(function (error) {
  // handle error
  if (error.response) console.log(error.response.data);
  else console.log(error.message);
});

See request parameter description

ParameterTypeDescription
mediabinaryimage to analyze
callback_urlstringcallback URL to receive moderation updates (optional)
modelsstringcomma-separated list of models to apply
intervalfloatframe interval in seconds, out of 0.5, 1, 2, 3, 4, 5 (optional)
api_userstringyour API user id
api_secretstringyour API secret

Option 3: Live-stream

Here's how to proceed to analyze a live-stream:


curl -X GET -G 'https://api.sightengine.com/1.0/video/check.json' \
    --data-urlencode 'stream_url=https://domain.tld/path/video.m3u8' \
    -d 'models=recreational_drug,medical' \
    -d 'callback_url=https://your.callback.url/path' \
    -d 'api_user={api_user}' \
    -d 'api_secret={api_secret}'


# if you haven't already, install the SDK with 'pip install sightengine'
from sightengine.client import SightengineClient
client = SightengineClient('{api_user}','{api_secret}')
output = client.check('recreational_drug', 'medical').video('https://domain.tld/path/video.m3u8', 'https://your.callback.url/path')


// if you haven't already, install the SDK with 'composer require sightengine/client-php'
use \Sightengine\SightengineClient;
$client = new SightengineClient('{api_user}','{api_secret}');
$output = $client->check(['recreational_drug', 'medical'])->video('https://domain.tld/path/video.m3u8', 'https://your.callback.url/path');


// if you haven't already, install the SDK with 'npm install sightengine --save'
var sightengine = require('sightengine')('{api_user}', '{api_secret}');
sightengine.check(['recreational_drug', 'medical']).video('https://domain.tld/path/video.m3u8', 'https://your.callback.url/path').then(function(result) {
    // The API response (result)
}).catch(function(err) {
    // Handle error
});

See request parameter description

ParameterTypeDescription
stream_urlstringURL of the video stream
callback_urlstringcallback URL to receive moderation updates (optional)
modelsstringcomma-separated list of models to apply
intervalfloatframe interval in seconds, out of 0.5, 1, 2, 3, 4, 5 (optional)
api_userstringyour API user id
api_secretstringyour API secret

Moderation result

The Moderation result will be provided either directly in the request response (for sync calls, see below) or through the callback URL your provided (for async calls).

Here is the structure of the JSON response with moderation results for each analyzed frame under the data.frames array:

            
                  
{
  "status": "success",
    "request": {
    "id": "req_gmgHNy8oP6nvXYaJVLq9n",
    "timestamp": 1717159864.348989,
    "operations": 21
  },
  "data": {
  "frames": [
    {
      "info": {
        "id": "med_gmgHcUOwe41rWmqwPhVNU_1",
        "position": 0
      },
      "recreational_drug": {
        "prob": 0.01,
        "classes": {
            "cannabis": 0.01,
            "cannabis_logo_only": 0.01,
            "cannabis_plant": 0.01,
            "cannabis_drug": 0.01,
            "recreational_drugs_not_cannabis": 0.01
        }
      },
      "medical": {
        "prob": 0.01,
        "classes": {
            "pills": 0.01,
            "paraphernalia": 0.01
        }
      },
     },
     ...
    ]
  },
  "media": {
    "id": "med_gmgHcUOwe41rWmqwPhVNU",
    "uri": "yourfile.mp4"
  },
}


            

You can use the classes under the recreational_drug and medical objects to detect drugs in the video.

Any other needs?

See our full list of Image/Video models for details on other filters and checks you can run on your images and videos. You might also want to check our Text models to moderate text-based content: messages, reviews, comments, usernames...

Was this page helpful?