Instances
Last updated
Last updated
This endpoint retrieves a list of all instances. The request requires a Bearer token in the header.
GET /api/v1/instances/ HTTP/1.1
Host: cortecs.ai
Authorization: Bearer JWT
Accept: */*
{
"instances": [
{
"instance_id": "instance-12345",
"base_url": "https://abcd-123456.eu/v1",
"instance_args": {
"model_id": "meta-llama--Meta-Llama-3.1-8B-Instruct",
"hardware_type_id": "NVIDIA_H100_1",
"context_length": 8192,
"billing_interval": "per_minute"
},
"instance_status": {
"started_at": "2024-08-07T08:53:13Z",
"stopped_at": null,
"status": "running"
},
"worker_statuses": [
{
"init_progress": {
"current_step": 5,
"num_steps": 5,
"description": "Running"
}
}
]
}
]
}This endpoint returns detailed information of the passed instance.
The unique identifier of the instance.
instance-12345GET /api/v1/instances/{instance_id} HTTP/1.1
Host: cortecs.ai
Authorization: Bearer JWT
Accept: */*
{
"instance": {
"instance_id": "instance-12345",
"base_url": "https://abcd-123456.eu/v1",
"instance_args": {
"model_id": "meta-llama--Meta-Llama-3.1-8B-Instruct",
"hardware_type_id": "NVIDIA_H100_1",
"context_length": 8192,
"billing_interval": "per_minute"
},
"instance_status": {
"started_at": "2024-08-07T08:53:13Z",
"stopped_at": null,
"status": "running"
},
"worker_statuses": [
{
"init_progress": {
"current_step": 5,
"num_steps": 5,
"description": "Running"
}
}
]
}
}This endpoint stops all running instances. The request requires a Bearer token in the header.
POST /api/v1/instances/stop-all HTTP/1.1
Host: cortecs.ai
Authorization: Bearer JWT
Accept: */*
{
"message": "All instances stopped successfully.",
"instance_ids": [
"instance-12345",
"instance-67890"
]
}This endpoint deletes an instance based on the provided instance ID. The instance must first be stopped before it can be deleted. To stop the instance, use the /instances/stop endpoint. The request requires a Bearer token in the header.
The ID of the instance to delete.
instance-12345DELETE /api/v1/instances/{instance_id} HTTP/1.1
Host: cortecs.ai
Authorization: Bearer JWT
Accept: */*
{
"message": "Instance deleted successfully.",
"instance_id": "instance-12345"
}This endpoint deletes all instances. If the force_deletion parameter is set to true, all instances will be deleted without stopping them first. Otherwise, only the stopped instances will be deleted. The request requires a Bearer token in the header.
Whether to force delete all instances without stopping them first.
trueDELETE /api/v1/instances/delete-all HTTP/1.1
Host: cortecs.ai
Authorization: Bearer JWT
Accept: */*
{
"message": "All instances deleted successfully.",
"instance_ids": [
"instance-12345",
"instance-67890"
]
}This endpoint starts an instance based on the provided instance arguments. The full instance is not available immediately after starting, but the status can be checked using the /instances/{instance_id} endpoint. The request requires a Bearer token in the header.
The id of the model to start.
meta-llama--Meta-Llama-3.1-8B-InstructThe id of hardware type to run the model on. Default is recommended_config of the model.
NVIDIA_H100_1The billing interval.
per_minutePossible values: Context length can be reduced if the full context length is not needed for the task. Decreasing the maximum context length increases the throughput of the model. Default is min(32000, max_context_length).
8192The number of workers to run the model on. Default is 1.
1POST /api/v1/instances/start HTTP/1.1
Host: cortecs.ai
Authorization: Bearer JWT
Content-Type: application/json
Accept: */*
Content-Length: 158
{
"model_id": "meta-llama--Meta-Llama-3.1-8B-Instruct",
"hardware_type_id": "NVIDIA_H100_1",
"billing_interval": "per_minute",
"context_length": 8192,
"num_workers": 1
}{
"message": "Instance is starting.",
"instance": {
"instance_id": "instance-12345",
"base_url": "https://abcd-123456.eu/v1",
"instance_args": {
"model_id": "meta-llama--Meta-Llama-3.1-8B-Instruct",
"hardware_type_id": "NVIDIA_H100_1",
"context_length": 8192,
"billing_interval": "per_minute"
},
"instance_status": {
"started_at": "2024-08-07T08:53:13Z",
"stopped_at": null,
"status": "running"
},
"worker_statuses": [
{
"init_progress": {
"current_step": 5,
"num_steps": 5,
"description": "Running"
}
}
]
}
}This endpoint restarts an instance based on the provided instance ID. The full instance is not available immediately after restarting, but the status can be checked using the /instances/{instance_id} endpoint. The request requires a Bearer token in the header.
The ID of the instance to restart.
instance-12345POST /api/v1/instances/restart HTTP/1.1
Host: cortecs.ai
Authorization: Bearer JWT
Content-Type: application/json
Accept: */*
Content-Length: 32
{
"instance_id": "instance-12345"
}{
"message": "Instance is starting.",
"instance": {
"instance_id": "instance-12345",
"base_url": "https://abcd-123456.eu/v1",
"instance_args": {
"model_id": "meta-llama--Meta-Llama-3.1-8B-Instruct",
"hardware_type_id": "NVIDIA_H100_1",
"context_length": 8192,
"billing_interval": "per_minute"
},
"instance_status": {
"started_at": "2024-08-07T08:53:13Z",
"stopped_at": null,
"status": "running"
},
"worker_statuses": [
{
"init_progress": {
"current_step": 5,
"num_steps": 5,
"description": "Running"
}
}
]
}
}This endpoint stops an instance based on the provided instance ID. The request requires a Bearer token in the header.
The ID of the instance to stop.
instance-12345POST /api/v1/instances/stop HTTP/1.1
Host: cortecs.ai
Authorization: Bearer JWT
Content-Type: application/json
Accept: */*
Content-Length: 32
{
"instance_id": "instance-12345"
}{
"message": "Instance stopped successfully",
"instance": {
"instance_id": "instance-12345",
"base_url": "https://abcd-123456.eu/v1",
"instance_args": {
"model_id": "meta-llama--Meta-Llama-3.1-8B-Instruct",
"hardware_type_id": "NVIDIA_H100_1",
"context_length": 8192,
"billing_interval": "per_minute"
},
"instance_status": {
"started_at": "2024-08-07T08:53:13Z",
"stopped_at": null,
"status": "running"
},
"worker_statuses": [
{
"init_progress": {
"current_step": 5,
"num_steps": 5,
"description": "Running"
}
}
]
}
}