Insight Engine Documentation

Getting started guides and API reference

Why use the Insight Engine

The Insight Engine creates visual object detectors from only a few images which means you can get up and running with your project without the need to train your own vision model.

Given a few example images, you can quickly create a detector and have it ready within minutes, giving you the ability to leverage vision AI on your specialized task right away.

Use cases

The Insight Engine is a flexible tool at your disposal to solve your visual detection needs. It's specially useful if you face:

rapidly changing detection demands
need to detect a specialized or uncommon object
want to rapidly prototype

Common use cases from our partners include:

Rapid prototyping of a new vision capability
Data annotation, object segmentaiton
Orthophoto search for objects of interest

Heads up!

If you have used the Insight Engine for another use case, we'd like to hear from you!

Getting Started

Authentication

To start using the InsightEngine, you'll first have to create an account and then login. Once logged in, you can generate an API key from your dashboard by clicking the "create API key" button.

Important Security Notice

It is important to keep your API key secure and NEVER share it. Leaked API keys may compromise your account!

To use your API key, you must supply it in the "x-api-key" header when making requests to our REST API.

Creating Your First Detector

To create your first detector, visit the detectors page and click on the "create detector" button. Alternatively, you can make use of our API endpoint (see below). We recommend you use the dashboard.

To create a detector, you will need:

A name for your detector
A few example images of what you'd like to detect
Masks for your positive examples. Note that this is seamless on our dashboard GUI using our built-in segmentation tool.

Creating a detector starts training it. Its status is "Training" while the model is being prepared and "Ready" when it can run inference. If training is unsuccessful, its status changes to "Failed" and the dashboard shows the reason reported by the service.

Heads up!

You can also edit your detector by uploading/removing examples or by tweaking its properties. Saving those changes starts a new training run, so the detector must return to "Ready" before it can run inference again.

Running Object Detection

You can use your detectors for inference through the dashboard or through our REST API using the "/detectors/{detector_id}/inference" endpoint. We recommend testing the detector using the dashboard but running large jobs on many images using the API.

Orthophoto (GIS) Search

You can use detectors to search for instances of objects on large scale aerial photography (orthophotos) such as satellite or drone imagery. We currently support common image formats such as (.jpeg and .png) as well as geotiffs (.tiff).

REST API Reference

Browse all available REST endpoints.

Detectors

7 operations

List Detectors

List detector metadata owned by the authenticated principal. Args: principal (Principal): Authenticated owner. service (DetectorService): Metadata service. Returns: list[DetectorResponse]: Detector metadata without model artifacts.

GET

/detectors

List Detectors

Responses

Successful Response

application/json

array

Fields

Field	Type	Required	Description	Notes
items	object	Required	Detector metadata and the latest training state. Training parameters describe the last successful model, or the initial request if no training has succeeded. Pending overrides are not exposed.	—
id	string (uuid)	Required	—	—
name	string	Required	—	—
dataset_name	string	Required	—	—
resolution	string	Required	—	enum: "coarse", "medium", "fine"
epochs	integer	Required	—	—
state	string	Required	Client-visible state of the most recent detector training attempt.	enum: "training", "ready", "failed"
failure_reason	anyOf	Required	—	—
anyOf[1]	string	Optional	—	—
anyOf[2]	null	Optional	—	—

Create Detector

Create a detector and schedule its initial training. Args: body (DetectorRequest): Initial name and training specification. principal (Principal): Authenticated owner. service (TrainingService): Background training service. Returns: DetectorResponse: Accepted detector in the training state.

POST

/detectors

Create Detector

Request body

Schema type: object

Fields

Field	Type	Required	Description	Notes
name	string	Required	—	—
dataset_name	string	Required	—	—
resolution	string	Required	—	enum: "coarse", "medium", "fine"
epochs	integer	Required	—	—

Responses

Successful Response

application/json

object

Fields

Field	Type	Required	Description	Notes
id	string (uuid)	Required	—	—
name	string	Required	—	—
dataset_name	string	Required	—	—
resolution	string	Required	—	enum: "coarse", "medium", "fine"
epochs	integer	Required	—	—
state	string	Required	Client-visible state of the most recent detector training attempt.	enum: "training", "ready", "failed"
failure_reason	anyOf	Required	—	—
anyOf[1]	string	Optional	—	—
anyOf[2]	null	Optional	—	—

Get Detector

Read one detector for display or lifecycle polling. Args: detector_id (UUID): Detector to read. principal (Principal): Authenticated owner. service (DetectorService): Metadata service. Returns: DetectorResponse: Stored parameters and current lifecycle state.

GET

/detectors/{detector_id}

Get Detector

Parameters

Name	In	Type	Required	Description
detector_id	path	string	Required	—

Responses

Successful Response

application/json

object

Fields

Field	Type	Required	Description	Notes
id	string (uuid)	Required	—	—
name	string	Required	—	—
dataset_name	string	Required	—	—
resolution	string	Required	—	enum: "coarse", "medium", "fine"
epochs	integer	Required	—	—
state	string	Required	Client-visible state of the most recent detector training attempt.	enum: "training", "ready", "failed"
failure_reason	anyOf	Required	—	—
anyOf[1]	string	Optional	—	—
anyOf[2]	null	Optional	—	—

Update Detector

Rename a detector without initiating training. Args: detector_id (UUID): Detector to rename. body (RenameDetectorRequest): Updated display name. principal (Principal): Authenticated owner. service (DetectorService): Metadata service. Returns: DetectorResponse: Updated detector metadata.

PATCH

/detectors/{detector_id}

Update Detector

Parameters

Name	In	Type	Required	Description
detector_id	path	string	Required	—

Request body

Schema type: object

Fields

Field	Type	Required	Description	Notes
name	string	Required	—	—

Responses

Successful Response

application/json

object

Fields

Field	Type	Required	Description	Notes
id	string (uuid)	Required	—	—
name	string	Required	—	—
dataset_name	string	Required	—	—
resolution	string	Required	—	enum: "coarse", "medium", "fine"
epochs	integer	Required	—	—
state	string	Required	Client-visible state of the most recent detector training attempt.	enum: "training", "ready", "failed"
failure_reason	anyOf	Required	—	—
anyOf[1]	string	Optional	—	—
anyOf[2]	null	Optional	—	—

Delete Detector

Delete a detector when no conflicting operation is active. Args: detector_id (UUID): Detector to delete. principal (Principal): Authenticated owner. service (DetectorService): Metadata service. Returns: Response: Empty success response.

DELETE

/detectors/{detector_id}

Delete Detector

Parameters

Name	In	Type	Required	Description
detector_id	path	string	Required	—

Responses

Successful Response

Infer

Run a ready detector against a base64-encoded image. Args: detector_id (UUID): Detector to run. body (InferenceRequest): Image and score threshold. principal (Principal): Authenticated owner. service (InferenceService): Inference service. Returns: InferenceResponse: Thresholded boxes and unthresholded patch scores. Raises: HTTPException: If the detector is busy, unavailable, or the image is invalid.

POST

/detectors/{detector_id}/inference

Infer

Parameters

Name	In	Type	Required	Description
detector_id	path	string	Required	—

Request body

Schema type: object

Fields

Field	Type	Required	Description	Notes
image	string (base64)	Required	Base64-encoded image file.	—
threshold	number	Required	—	—

Responses

Successful Response

application/json

object

Fields

Field	Type	Required	Description	Notes
bounding_boxes	array	Required	—	—
items	array	Required	—	—
patch_probability_grid	object	Required	Unthresholded patch scores positioned over the source image.	—
scores	array	Required	Patch probabilities indexed by row, then column.	—
items	array	Required	—	—
items	number	Required	—	—
image_size	object	Required	Image dimensions in source pixels.	—
width	integer	Required	—	—
height	integer	Required	—	—
origin	object	Required	Top-left grid origin in source-image coordinates.	—
x	number	Required	—	—
y	number	Required	—	—
stride	object	Required	Distance between adjacent patch boundaries in source pixels.	—
x	number	Required	—	—
y	number	Required	—	—

Retrain Detector

Request training with optional parameter overrides. Args: detector_id (UUID): Detector to retrain. body (RetrainingRequest): Overrides; an empty object reuses stored parameters. principal (Principal): Authenticated owner. service (TrainingService): Background training service. Returns: DetectorResponse: Existing parameters with the training state. Raises: HTTPException: If an operation is active or a referenced resource is missing.

POST

/detectors/{detector_id}/training

Retrain Detector

Parameters

Name	In	Type	Required	Description
detector_id	path	string	Required	—

Request body

Schema type: object

Fields

Field	Type	Required	Description	Notes
dataset_name	anyOf	Optional	—	—
anyOf[1]	string	Optional	—	—
anyOf[2]	null	Optional	—	—
resolution	anyOf	Optional	—	—
anyOf[1]	string	Optional	—	enum: "coarse", "medium", "fine"
anyOf[2]	null	Optional	—	—
epochs	anyOf	Optional	—	—
anyOf[1]	integer	Optional	—	—
anyOf[2]	null	Optional	—	—

Responses

Successful Response

application/json

object

Fields

Field	Type	Required	Description	Notes
id	string (uuid)	Required	—	—
name	string	Required	—	—
dataset_name	string	Required	—	—
resolution	string	Required	—	enum: "coarse", "medium", "fine"
epochs	integer	Required	—	—
state	string	Required	Client-visible state of the most recent detector training attempt.	enum: "training", "ready", "failed"
failure_reason	anyOf	Required	—	—
anyOf[1]	string	Optional	—	—
anyOf[2]	null	Optional	—	—

Datasets

9 operations

List Datasets

List the caller's datasets with their currently usable examples. Args: principal (Principal): Authenticated owner. repository (DatasetRepository): Dataset persistence. Returns: list[Dataset]: Datasets sorted by name.

GET

/datasets

List Datasets

Responses

Successful Response

application/json

array

Fields

Field	Type	Required	Description	Notes
items	object	Required	—	—
name	string	Required	—	—
positives	array	Optional	—	default: []
items	object	Required	—	—
image_key	string	Required	—	—
mask_key	string	Required	—	—
negatives	array	Optional	—	default: []
items	string	Required	—	—

Create Dataset

Create an empty dataset before uploading files. Args: body (CreateDatasetRequest): Dataset name. principal (Principal): Authenticated owner. repository (DatasetRepository): Dataset persistence. Returns: Dataset: Newly created empty dataset.

POST

/datasets

Create Dataset

Request body

Schema type: object

Fields

Field	Type	Required	Description	Notes
name	string	Required	—	—

Responses

Successful Response

application/json

object

Fields

Field	Type	Required	Description	Notes
name	string	Required	—	—
positives	array	Optional	—	default: []
items	object	Required	—	—
image_key	string	Required	—	—
mask_key	string	Required	—	—
negatives	array	Optional	—	default: []
items	string	Required	—	—

Get Dataset

Read a dataset's complete, non-conflicting examples. Args: name (str): Dataset name. principal (Principal): Authenticated owner. repository (DatasetRepository): Dataset persistence. Returns: Dataset: Currently usable positive pairs and negative images.

GET

/datasets/{name}

Get Dataset

Parameters

Name	In	Type	Required	Description
name	path	string	Required	—

Responses

Successful Response

application/json

object

Fields

Field	Type	Required	Description	Notes
name	string	Required	—	—
positives	array	Optional	—	default: []
items	object	Required	—	—
image_key	string	Required	—	—
mask_key	string	Required	—	—
negatives	array	Optional	—	default: []
items	string	Required	—	—

Delete Dataset

Delete a dataset and all its files; repeated deletion is harmless. Args: name (str): Dataset name. principal (Principal): Authenticated owner. repository (DatasetRepository): Dataset persistence. Returns: Response: Empty success response.

DELETE

/datasets/{name}

Delete Dataset

Parameters

Name	In	Type	Required	Description
name	path	string	Required	—

Responses

Successful Response

Prepare Upload

Prepare a PUT upload, which may replace an existing file. Upload the bytes to the returned URL before registering membership. Preparing an upload does not add a positive or negative example. Args: name (str): Existing dataset name. key (str): Dataset-local file key. principal (Principal): Authenticated owner. repository (DatasetRepository): Dataset persistence. Returns: PresignedUrlResponse: URL accepting the file bytes via PUT.

POST

/datasets/{name}/files/{key}/upload-url

Prepare Upload

Parameters

Name	In	Type	Required	Description
name	path	string	Required	—
key	path	string	Required	—

Responses

Successful Response

application/json

object

Fields

Field	Type	Required	Description	Notes
url	string	Required	—	—

Prepare Download

Prepare a GET download for an existing file. Args: name (str): Dataset name. key (str): Dataset-local file key. principal (Principal): Authenticated owner. repository (DatasetRepository): Dataset persistence. Returns: PresignedUrlResponse: URL returning file bytes via GET.

POST

/datasets/{name}/files/{key}/download-url

Prepare Download

Parameters

Name	In	Type	Required	Description
name	path	string	Required	—
key	path	string	Required	—

Responses

Successful Response

application/json

object

Fields

Field	Type	Required	Description	Notes
url	string	Required	—	—

Add Positive

Register an uploaded image-mask pair; repeating the same pair is harmless. Existing assignments must be deleted before files can be reassigned. Args: name (str): Dataset name. image_key (str): Uploaded positive image key. body (PositivePairRequest): Uploaded mask key. principal (Principal): Authenticated owner. repository (DatasetRepository): Dataset persistence. Returns: Response: Empty success response.

PUT

/datasets/{name}/positives/{image_key}

Add Positive

Parameters

Name	In	Type	Required	Description
name	path	string	Required	—
image_key	path	string	Required	—

Request body

Schema type: object

Fields

Field	Type	Required	Description	Notes
mask_key	string	Required	—	—

Responses

Successful Response

Add Negative

Register an uploaded negative image; repeating the same addition is harmless. Args: name (str): Dataset name. image_key (str): Uploaded negative image key. principal (Principal): Authenticated owner. repository (DatasetRepository): Dataset persistence. Returns: Response: Empty success response.

PUT

/datasets/{name}/negatives/{image_key}

Add Negative

Parameters

Name	In	Type	Required	Description
name	path	string	Required	—
image_key	path	string	Required	—

Responses

Successful Response

Delete Image

Delete a file and its membership, including its partner in a positive pair. Deleting either a positive image or its mask deletes both files. Args: name (str): Dataset name. key (str): Dataset-local file key. principal (Principal): Authenticated owner. repository (DatasetRepository): Dataset persistence. Returns: Response: Empty success response.

DELETE

/datasets/{name}/files/{key}

Delete Image

Parameters

Name	In	Type	Required	Description
name	path	string	Required	—
key	path	string	Required	—

Responses

Successful Response

Segmentation

1 operation

Segment From Points

Segment an image using positive and negative points. Args: body (SegmentationRequest): Image bytes and point prompts. service (SegmentationService): Application-owned segmentation service. Returns: Response: Highest-scoring mask as a grayscale PNG.

POST

/segment/from_points

Segment From Points

Request body

Schema type: object

Fields

Field	Type	Required	Description	Notes
image	string (base64)	Required	Base64-encoded image file.	—
positives	array	Required	—	—
items	object	Required	A pixel coordinate in the source image.	—
x	integer	Required	—	—
y	integer	Required	—	—
negatives	array	Optional	—	—
items	object	Required	A pixel coordinate in the source image.	—
x	integer	Required	—	—
y	integer	Required	—	—

Responses

PNG image containing the highest-scoring mask.

image/png

string

Fields

Field	Type	Required	Description	Notes
value	string (binary)	Required	—	—