Instances

PreviousUser NextModels

Last updated 22 days ago

Get a list of all instances

This endpoint retrieves a list of all instances. The request requires a Bearer token in the header.

GEThttps://cortecs.ai/api/v1/instances/

Authorization

Response

Successfully retrieved all instances

Body

instancesarray of object

Request

const response = await fetch('https://cortecs.ai/api/v1/instances/', {
    method: 'GET',
    headers: {
      "Authorization": "Bearer JWT"
    },
});
const data = await response.json();

Response

{
  "instances": [
    {
      "instance_id": "instance-12345",
      "base_url": "https://abcd-123456.eu/v1",
      "instance_args": {
        "model_id": "meta-llama--Meta-Llama-3.1-8B-Instruct",
        "hardware_type_id": "NVIDIA_H100_1",
        "context_length": 8192,
        "billing_interval": "per_minute"
      },
      "instance_status": {
        "started_at": "2024-08-07T08:53:13Z",
        "stopped_at": null,
        "status": "running",
        "init_progress": {
          "current_step": 5,
          "num_steps": 5,
          "description": "Model running"
        }
      }
    }
  ]
}

Get detailed information about an instance

This endpoint returns detailed information of the passed instance.

GEThttps://cortecs.ai/api/v1/instances/{instance_id}

Authorization

Path parameters

instance_id*string

The unique identifier of the instance.

Example: "instance-12345"

Response

Successfully returned information about a specific instance.

Body

instanceobject

Request

const response = await fetch('https://cortecs.ai/api/v1/instances/{instance_id}', {
    method: 'GET',
    headers: {
      "Authorization": "Bearer JWT"
    },
});
const data = await response.json();

Response

{
  "instance": {
    "instance_id": "instance-12345",
    "base_url": "https://abcd-123456.eu/v1",
    "instance_args": {
      "model_id": "meta-llama--Meta-Llama-3.1-8B-Instruct",
      "hardware_type_id": "NVIDIA_H100_1",
      "context_length": 8192,
      "billing_interval": "per_minute"
    },
    "instance_status": {
      "started_at": "2024-08-07T08:53:13Z",
      "stopped_at": null,
      "status": "running",
      "init_progress": {
        "current_step": 5,
        "num_steps": 5,
        "description": "Model running"
      }
    }
  }
}

Start an instance

This endpoint starts an instance based on the provided instance arguments. The full instance is not available immediately after starting, but the status can be checked using the /instances/{instance_id} endpoint. The request requires a Bearer token in the header.

POSThttps://cortecs.ai/api/v1/instances/start

Authorization

Body

model_id*string

The id of the model to start.

Example: "meta-llama--Meta-Llama-3.1-8B-Instruct"

hardware_type_idstring

The id of hardware type to run the model on. Default is recommended_config of the model.

Example: "NVIDIA_H100_1"

billing_intervalenum

The billing interval.

Example: "per_minute"

per_minuteper_hour

context_lengthnumber

Context length can be reduced if the full context length is not needed for the task. Decreasing the maximum context length increases the throughput of the model. Default is min(32000, max_context_length).

Example: 8192

Response

The instance is starting.

Body

messagestring

Status message indicating that the instance is starting.

Example: "Instance is starting."

instanceobject

Request

const response = await fetch('https://cortecs.ai/api/v1/instances/start', {
    method: 'POST',
    headers: {
      "Authorization": "Bearer JWT",
      "Content-Type": "application/json"
    },
    body: JSON.stringify({
      "model_id": "meta-llama--Meta-Llama-3.1-8B-Instruct",
      "hardware_type_id": "NVIDIA_H100_1",
      "billing_interval": "per_minute",
      "context_length": 8192
    }),
});
const data = await response.json();

Response

{
  "message": "Instance is starting.",
  "instance": {
    "instance_id": "instance-12345",
    "base_url": "https://abcd-123456.eu/v1",
    "instance_args": {
      "model_id": "meta-llama--Meta-Llama-3.1-8B-Instruct",
      "hardware_type_id": "NVIDIA_H100_1",
      "context_length": 8192,
      "billing_interval": "per_minute"
    },
    "instance_status": {
      "started_at": "2024-08-07T08:53:13Z",
      "stopped_at": null,
      "status": "running",
      "init_progress": {
        "current_step": 5,
        "num_steps": 5,
        "description": "Model running"
      }
    }
  }
}

Restart an instance

This endpoint restarts an instance based on the provided instance ID. The full instance is not available immediately after restarting, but the status can be checked using the /instances/{instance_id} endpoint. The request requires a Bearer token in the header.

POSThttps://cortecs.ai/api/v1/instances/restart

Authorization

Body

instance_id*string

The ID of the instance to restart.

Example: "instance-12345"

Response

Successfully restarted the instance.

Body

messagestring

Status message indicating that the instance is starting.

Example: "Instance is starting."

instanceobject

Request

const response = await fetch('https://cortecs.ai/api/v1/instances/restart', {
    method: 'POST',
    headers: {
      "Authorization": "Bearer JWT",
      "Content-Type": "application/json"
    },
    body: JSON.stringify({
      "instance_id": "instance-12345"
    }),
});
const data = await response.json();

Response

{
  "message": "Instance is starting.",
  "instance": {
    "instance_id": "instance-12345",
    "base_url": "https://abcd-123456.eu/v1",
    "instance_args": {
      "model_id": "meta-llama--Meta-Llama-3.1-8B-Instruct",
      "hardware_type_id": "NVIDIA_H100_1",
      "context_length": 8192,
      "billing_interval": "per_minute"
    },
    "instance_status": {
      "started_at": "2024-08-07T08:53:13Z",
      "stopped_at": null,
      "status": "running",
      "init_progress": {
        "current_step": 5,
        "num_steps": 5,
        "description": "Model running"
      }
    }
  }
}

Stop an instance

This endpoint stops an instance based on the provided instance ID. The request requires a Bearer token in the header.

POSThttps://cortecs.ai/api/v1/instances/stop

Authorization

Body

instance_id*string

The ID of the instance to stop.

Example: "instance-12345"

Response

Successfully stopped the instance.

Body

messagestring

Status message indicating that the instance was stopped.

Example: "Instance stopped successfully"

instanceobject

Request

const response = await fetch('https://cortecs.ai/api/v1/instances/stop', {
    method: 'POST',
    headers: {
      "Authorization": "Bearer JWT",
      "Content-Type": "application/json"
    },
    body: JSON.stringify({
      "instance_id": "instance-12345"
    }),
});
const data = await response.json();

Response

{
  "message": "Instance stopped successfully",
  "instance": {
    "instance_id": "instance-12345",
    "base_url": "https://abcd-123456.eu/v1",
    "instance_args": {
      "model_id": "meta-llama--Meta-Llama-3.1-8B-Instruct",
      "hardware_type_id": "NVIDIA_H100_1",
      "context_length": 8192,
      "billing_interval": "per_minute"
    },
    "instance_status": {
      "started_at": "2024-08-07T08:53:13Z",
      "stopped_at": null,
      "status": "running",
      "init_progress": {
        "current_step": 5,
        "num_steps": 5,
        "description": "Model running"
      }
    }
  }
}

Stop all instances

This endpoint stops all running instances. The request requires a Bearer token in the header.

POSThttps://cortecs.ai/api/v1/instances/stop-all

Authorization

Response

Successfully stopped all instances.

Body

messagestring

Status message indicating that all instances were stopped.

Example: "All instances stopped successfully."

instance_idsarray of string

The IDs of the stopped instances.

Example: ["instance-12345","instance-67890"]

Request

const response = await fetch('https://cortecs.ai/api/v1/instances/stop-all', {
    method: 'POST',
    headers: {
      "Authorization": "Bearer JWT"
    },
});
const data = await response.json();

Response

{
  "message": "All instances stopped successfully.",
  "instance_ids": [
    "instance-12345",
    "instance-67890"
  ]
}

Delete an instance

This endpoint deletes an instance based on the provided instance ID. The instance must first be stopped before it can be deleted. To stop the instance, use the /instances/stop endpoint. The request requires a Bearer token in the header.

DELETEhttps://cortecs.ai/api/v1/instances/{instance_id}

Authorization

Path parameters

instance_id*string

The ID of the instance to delete.

Example: "instance-12345"

Response

Successfully deleted the model instance.

Body

messagestring

Status message indicating that the instance was deleted.

Example: "Instance deleted successfully."

instance_idstring

The ID of the deleted instance.

Example: "instance-12345"

Request

const response = await fetch('https://cortecs.ai/api/v1/instances/{instance_id}', {
    method: 'DELETE',
    headers: {
      "Authorization": "Bearer JWT"
    },
});
const data = await response.json();

Response

{
  "message": "Instance deleted successfully.",
  "instance_id": "instance-12345"
}

Delete all instances

This endpoint deletes all instances. If the force_deletion parameter is set to true, all instances will be deleted without stopping them first. Otherwise, only the stopped instances will be deleted. The request requires a Bearer token in the header.

DELETEhttps://cortecs.ai/api/v1/instances/delete-all

Authorization

Query parameters

Response

Successfully deleted all instances.

Body

messagestring

Status message indicating that all instances were deleted.

Example: "All instances deleted successfully."

instance_idsarray of string

The IDs of the deleted instances.

Example: ["instance-12345","instance-67890"]

Request

const response = await fetch('https://cortecs.ai/api/v1/instances/delete-all', {
    method: 'DELETE',
    headers: {
      "Authorization": "Bearer JWT"
    },
});
const data = await response.json();

Response

{
  "message": "All instances deleted successfully.",
  "instance_ids": [
    "instance-12345",
    "instance-67890"
  ]
}