Create

post

https://api.humanloop.com/v4/projects/{project_id}/evaluations

Create an evaluation.

Path Params

project_id

string

required

String ID of project. Starts with pr_.

Body Params

config_id

string

required

ID of the config to evaluate. Starts with config_.

evaluator_ids

array of strings

required

IDs of evaluators to run on the dataset. IDs start with evfn_

Evaluators*

dataset_id

string

required

ID of the dataset to use in this evaluation. Starts with evts_.

provider_api_keys

object

API keys required by each provider to make API calls. The API keys provided here are not stored by Humanloop. If not specified here, Humanloop will fall back to the key saved to your organization. Ensure you provide an API key for the provider for the model config you are evaluating, or have one saved to your organization.

max_concurrency

integer

Defaults to 5

The maximum number of concurrent generations to run. A higher value will result in faster completion of the evaluation but may place higher load on your provider rate-limits.

hl_generated

boolean

Defaults to true

Whether the log generations for this evaluation should be performed by Humanloop. If False, the log generations should be submitted by the user via the API.

Responses

201Successful Response

422Validation Error