Visual Search

Not yet available for batches created through the developer API. The keyframe index this searches is built only by the fastdrop.io subscriber pipeline. A batch you create with POST /v1/batch is never indexed, so POST /v1/batch/{id}/visual-search accepts your key, finds the batch, and returns zero results — not an error. The endpoint is documented here because it exists and will work once API batches are indexed; until then, treat this page as a preview and use transcript search, which does work for API-created batches.For the same reason, visual search is not exposed as an MCP tool. See MCP Setup for the tools that are.

Visual search is in beta. It works well for broad visual descriptions but not for fine-grained details like specific colors or small text.

Visual search lets you find videos by describing what’s in them. It uses CLIP to match your text description against keyframe thumbnails extracted from each video.

How it works

When a batch is processed, keyframe thumbnails are extracted from each video
Each thumbnail is embedded using CLIP (ViT-B/32) into a 512-dimensional vector
When you search, your text query is embedded using the same CLIP model
FastDrop finds thumbnails with the highest cosine similarity to your query

Step 2 is the one that doesn’t run for API-created batches today. Thumbnails are extracted, but nothing embeds them, so there is nothing for step 4 to match against. Visual search requires an API key. It costs 0 credits per query — the CLIP embedding cost is included with thumbnail extraction.

Code examples

import httpx

API_KEY = "fd_live_your_key_here"
BATCH_ID = "a1b2c3d4-5678-90ab-cdef-1234567890ab"

response = httpx.post(
    f"https://api.fastdrop.io/api/v1/batch/{BATCH_ID}/visual-search",
    headers={"X-API-Key": API_KEY},
    json={
        "query": "outdoor establishing shot with trees",
        "max_results": 5,
    },
)

data = response.json()
for result in data["results"]:
    print(f'{result["filename"]} — similarity: {result["similarity_score"]:.3f}')
    print(f'  Frame {result["matching_frame_index"]}: {result["thumbnail_url"]}')

const API_KEY = "fd_live_your_key_here";
const BATCH_ID = "a1b2c3d4-5678-90ab-cdef-1234567890ab";

const response = await fetch(
  `https://api.fastdrop.io/api/v1/batch/${BATCH_ID}/visual-search`,
  {
    method: "POST",
    headers: {
      "Content-Type": "application/json",
      "X-API-Key": API_KEY,
    },
    body: JSON.stringify({
      query: "outdoor establishing shot with trees",
      max_results: 5,
    }),
  }
);

const data = await response.json();
for (const result of data.results) {
  console.log(`${result.filename} — similarity: ${result.similarity_score}`);
  console.log(`  Frame ${result.matching_frame_index}: ${result.thumbnail_url}`);
}

Example response

{
  "query": "outdoor establishing shot with trees",
  "total_results": 3,
  "results": [
    {
      "video_id": "f8e7d6c5-...",
      "filename": "DJI_0042.MP4",
      "role": "B-Roll",
      "similarity_score": 0.7234,
      "matching_frame_index": 2,
      "thumbnail_url": "https://storage.fastdrop.io/thumbs/...",
      "duration_seconds": 15.4
    }
  ]
}

Response fields

Field	Type	Description
`video_id`	string	UUID of the matched video
`filename`	string	Original filename
`role`	string \| null	Classification role (null if not classified)
`similarity_score`	float	0.0 to 1.0 CLIP similarity score
`matching_frame_index`	int	Index of the best-matching keyframe
`thumbnail_url`	string \| null	URL of the matching thumbnail
`duration_seconds`	float \| null	Video duration in seconds

Example queries

Works well

These types of descriptions produce reliable results:

"outdoor establishing shot" — landscape and wide shots
"person at desk" — interview or office setups
"whiteboard with writing" — presentation or lecture frames
"close-up of hands" — detail shots
"drone aerial view" — aerial footage
"dark room with screen" — screen recording or demo setups

Less reliable

CLIP matches broad visual concepts, not fine details:

"blue shirt" — color-specific queries are inconsistent
"text says 'hello'" — can’t read specific text
"exactly 3 people" — can’t count reliably
"logo in top-right corner" — spatial positioning is weak

Combining with transcript search

For the most comprehensive results, use both search types. The unified search service fuses transcript and visual scores (0.6 transcript + 0.4 visual) to find videos that match on both content and visuals.

Error responses

Status	Code	Meaning
404	Batch not found	Batch doesn’t exist or doesn’t belong to your API key
409	Batch not completed	Batch is still processing

Get Started

Understanding video defects

Guides

How it works

Code examples

Example response

Response fields

Example queries

Works well

Less reliable

Combining with transcript search

Error responses

​How it works

​Code examples

​Example response

​Response fields

​Example queries

​Works well

​Less reliable

​Combining with transcript search

​Error responses

How it works

Code examples

Example response

Response fields

Example queries

Works well

Less reliable

Combining with transcript search

Error responses