Instances

PreviousUser NextModels

Last updated 2 months ago

Get a list of all instances

get

This endpoint retrieves a list of all instances. The request requires a Bearer token in the header.

Authorizations

Responses

200

Successfully retrieved all instances

application/json

401

Unauthorized - Bearer token missing or invalid.

500

Internal server error.

get

GET /api/v1/instances/ HTTP/1.1
Host: cortecs.ai
Authorization: Bearer JWT
Accept: */*

{
  "instances": [
    {
      "instance_id": "instance-12345",
      "base_url": "https://abcd-123456.eu/v1",
      "instance_args": {
        "model_id": "meta-llama--Meta-Llama-3.1-8B-Instruct",
        "hardware_type_id": "NVIDIA_H100_1",
        "context_length": 8192,
        "billing_interval": "per_minute"
      },
      "instance_status": {
        "started_at": "2024-08-07T08:53:13Z",
        "stopped_at": null,
        "status": "running"
      },
      "worker_statuses": [
        {
          "init_progress": {
            "current_step": 5,
            "num_steps": 5,
            "description": "Running"
          }
        }
      ]
    }
  ]
}

Get detailed information about an instance

get

This endpoint returns detailed information of the passed instance.

Authorizations

Path parameters

instance_idstringRequired

The unique identifier of the instance.

Example: instance-12345

Responses

200

Successfully returned information about a specific instance.

application/json

400

Invalid request - Missing or invalid parameters.

401

Unauthorized - Bearer token missing or invalid.

500

Internal server error.

get

GET /api/v1/instances/{instance_id} HTTP/1.1
Host: cortecs.ai
Authorization: Bearer JWT
Accept: */*

{
  "instance": {
    "instance_id": "instance-12345",
    "base_url": "https://abcd-123456.eu/v1",
    "instance_args": {
      "model_id": "meta-llama--Meta-Llama-3.1-8B-Instruct",
      "hardware_type_id": "NVIDIA_H100_1",
      "context_length": 8192,
      "billing_interval": "per_minute"
    },
    "instance_status": {
      "started_at": "2024-08-07T08:53:13Z",
      "stopped_at": null,
      "status": "running"
    },
    "worker_statuses": [
      {
        "init_progress": {
          "current_step": 5,
          "num_steps": 5,
          "description": "Running"
        }
      }
    ]
  }
}

Stop all instances

post

This endpoint stops all running instances. The request requires a Bearer token in the header.

Authorizations

Responses

200

Successfully stopped all instances.

application/json

401

Unauthorized - Bearer token missing or invalid.

500

Internal server error.

post

POST /api/v1/instances/stop-all HTTP/1.1
Host: cortecs.ai
Authorization: Bearer JWT
Accept: */*

{
  "message": "All instances stopped successfully.",
  "instance_ids": [
    "instance-12345",
    "instance-67890"
  ]
}

Delete an instance

delete

This endpoint deletes an instance based on the provided instance ID. The instance must first be stopped before it can be deleted. To stop the instance, use the /instances/stop endpoint. The request requires a Bearer token in the header.

Authorizations

Path parameters

instance_idstringRequired

The ID of the instance to delete.

Example: instance-12345

Responses

200

Successfully deleted the model instance.

application/json

400

Invalid request - Missing or invalid parameters.

401

Unauthorized - Bearer token missing or invalid.

500

Internal server error.

delete

DELETE /api/v1/instances/{instance_id} HTTP/1.1
Host: cortecs.ai
Authorization: Bearer JWT
Accept: */*

{
  "message": "Instance deleted successfully.",
  "instance_id": "instance-12345"
}

Delete all instances

delete

This endpoint deletes all instances. If the force_deletion parameter is set to true, all instances will be deleted without stopping them first. Otherwise, only the stopped instances will be deleted. The request requires a Bearer token in the header.

Authorizations

Query parameters

force_deletionbooleanOptional

Whether to force delete all instances without stopping them first.

Example: true

Responses

200

Successfully deleted all instances.

application/json

401

Unauthorized - Bearer token missing or invalid.

500

Internal server error.

delete

DELETE /api/v1/instances/delete-all HTTP/1.1
Host: cortecs.ai
Authorization: Bearer JWT
Accept: */*

{
  "message": "All instances deleted successfully.",
  "instance_ids": [
    "instance-12345",
    "instance-67890"
  ]
}

Start an instance

post

This endpoint starts an instance based on the provided instance arguments. The full instance is not available immediately after starting, but the status can be checked using the /instances/{instance_id} endpoint. The request requires a Bearer token in the header.

Authorizations

Body

model_idstringRequired

The id of the model to start.

Example: meta-llama--Meta-Llama-3.1-8B-Instruct

hardware_type_idstringOptional

The id of hardware type to run the model on. Default is recommended_config of the model.

Example: NVIDIA_H100_1

billing_intervalstring · enumOptional

The billing interval.

Example: per_minutePossible values:

context_lengthnumberOptional

Context length can be reduced if the full context length is not needed for the task. Decreasing the maximum context length increases the throughput of the model. Default is min(32000, max_context_length).

Example: 8192

num_workersintegerOptional

The number of workers to run the model on. Default is 1.

Example: 1

Responses

202

The instance is starting.

application/json

400

Invalid request - Missing or invalid parameters.

401

Unauthorized - Bearer token missing or invalid.

500

Internal server error.

post

POST /api/v1/instances/start HTTP/1.1
Host: cortecs.ai
Authorization: Bearer JWT
Content-Type: application/json
Accept: */*
Content-Length: 158

{
  "model_id": "meta-llama--Meta-Llama-3.1-8B-Instruct",
  "hardware_type_id": "NVIDIA_H100_1",
  "billing_interval": "per_minute",
  "context_length": 8192,
  "num_workers": 1
}

{
  "message": "Instance is starting.",
  "instance": {
    "instance_id": "instance-12345",
    "base_url": "https://abcd-123456.eu/v1",
    "instance_args": {
      "model_id": "meta-llama--Meta-Llama-3.1-8B-Instruct",
      "hardware_type_id": "NVIDIA_H100_1",
      "context_length": 8192,
      "billing_interval": "per_minute"
    },
    "instance_status": {
      "started_at": "2024-08-07T08:53:13Z",
      "stopped_at": null,
      "status": "running"
    },
    "worker_statuses": [
      {
        "init_progress": {
          "current_step": 5,
          "num_steps": 5,
          "description": "Running"
        }
      }
    ]
  }
}

Restart an instance

post

This endpoint restarts an instance based on the provided instance ID. The full instance is not available immediately after restarting, but the status can be checked using the /instances/{instance_id} endpoint. The request requires a Bearer token in the header.

Authorizations

Body

instance_idstringRequired

The ID of the instance to restart.

Example: instance-12345

Responses

202

Successfully restarted the instance.

application/json

400

Invalid request - Missing or invalid parameters

401

Unauthorized - Bearer token missing or invalid

500

Internal server error

post

POST /api/v1/instances/restart HTTP/1.1
Host: cortecs.ai
Authorization: Bearer JWT
Content-Type: application/json
Accept: */*
Content-Length: 32

{
  "instance_id": "instance-12345"
}

{
  "message": "Instance is starting.",
  "instance": {
    "instance_id": "instance-12345",
    "base_url": "https://abcd-123456.eu/v1",
    "instance_args": {
      "model_id": "meta-llama--Meta-Llama-3.1-8B-Instruct",
      "hardware_type_id": "NVIDIA_H100_1",
      "context_length": 8192,
      "billing_interval": "per_minute"
    },
    "instance_status": {
      "started_at": "2024-08-07T08:53:13Z",
      "stopped_at": null,
      "status": "running"
    },
    "worker_statuses": [
      {
        "init_progress": {
          "current_step": 5,
          "num_steps": 5,
          "description": "Running"
        }
      }
    ]
  }
}

Stop an instance

post

This endpoint stops an instance based on the provided instance ID. The request requires a Bearer token in the header.

Authorizations

Body

instance_idstringRequired

The ID of the instance to stop.

Example: instance-12345

Responses

200

Successfully stopped the instance.

application/json

400

Invalid request - Missing or invalid parameters.

401

Unauthorized - Bearer token missing or invalid.

500

Internal server error.

post

POST /api/v1/instances/stop HTTP/1.1
Host: cortecs.ai
Authorization: Bearer JWT
Content-Type: application/json
Accept: */*
Content-Length: 32

{
  "instance_id": "instance-12345"
}

{
  "message": "Instance stopped successfully",
  "instance": {
    "instance_id": "instance-12345",
    "base_url": "https://abcd-123456.eu/v1",
    "instance_args": {
      "model_id": "meta-llama--Meta-Llama-3.1-8B-Instruct",
      "hardware_type_id": "NVIDIA_H100_1",
      "context_length": 8192,
      "billing_interval": "per_minute"
    },
    "instance_status": {
      "started_at": "2024-08-07T08:53:13Z",
      "stopped_at": null,
      "status": "running"
    },
    "worker_statuses": [
      {
        "init_progress": {
          "current_step": 5,
          "num_steps": 5,
          "description": "Running"
        }
      }
    ]
  }
}