Instances

Get a list of all instances

get

This endpoint retrieves a list of all instances. The request requires a Bearer token in the header.

Authorizations

bearerAuth

AuthorizationstringRequired

Bearer authentication header of the form Bearer <token>.

Responses

200

Successfully retrieved all instances

application/json

401

Unauthorized - Bearer token missing or invalid.

500

Internal server error.

get

/instances/

HTTPcURLJavaScriptPython

GET /api/v1/instances/ HTTP/1.1
Host: cortecs.ai
Authorization: Bearer YOUR_SECRET_TOKEN
Accept: */*

{
  "instances": [
    {
      "instance_id": "instance-12345",
      "base_url": "https://abcd-123456.eu/v1",
      "instance_args": {
        "model_id": "meta-llama--Meta-Llama-3.1-8B-Instruct",
        "hardware_type_id": "NVIDIA_H100_1",
        "context_length": 8192,
        "billing_interval": "per_minute"
      },
      "instance_status": {
        "started_at": "2024-08-07T08:53:13Z",
        "stopped_at": null,
        "status": "running"
      },
      "worker_statuses": [
        {
          "init_progress": {
            "current_step": 5,
            "num_steps": 5,
            "description": "Running"
          }
        }
      ]
    }
  ]
}

Get detailed information about an instance

get

This endpoint returns detailed information of the passed instance.

Authorizations

bearerAuth

AuthorizationstringRequired

Bearer authentication header of the form Bearer <token>.

Path parameters

instance_idstringRequired

The unique identifier of the instance.

Example: instance-12345

Responses

200

Successfully returned information about a specific instance.

application/json

400

Invalid request - Missing or invalid parameters.

401

Unauthorized - Bearer token missing or invalid.

500

Internal server error.

get

/instances/{instance_id}

HTTPcURLJavaScriptPython

GET /api/v1/instances/{instance_id} HTTP/1.1
Host: cortecs.ai
Authorization: Bearer YOUR_SECRET_TOKEN
Accept: */*

{
  "instance": {
    "instance_id": "instance-12345",
    "base_url": "https://abcd-123456.eu/v1",
    "instance_args": {
      "model_id": "meta-llama--Meta-Llama-3.1-8B-Instruct",
      "hardware_type_id": "NVIDIA_H100_1",
      "context_length": 8192,
      "billing_interval": "per_minute"
    },
    "instance_status": {
      "started_at": "2024-08-07T08:53:13Z",
      "stopped_at": null,
      "status": "running"
    },
    "worker_statuses": [
      {
        "init_progress": {
          "current_step": 5,
          "num_steps": 5,
          "description": "Running"
        }
      }
    ]
  }
}

Start an instance

post

This endpoint starts an instance based on the provided instance arguments. The full instance is not available immediately after starting, but the status can be checked using the /instances/{instance_id} endpoint. The request requires a Bearer token in the header.

Authorizations

bearerAuth

AuthorizationstringRequired

Bearer authentication header of the form Bearer <token>.

Body

application/json

model_idstringRequired

The id of the model to start.

Example: meta-llama--Meta-Llama-3.1-8B-Instruct

hardware_type_idstringOptional

The id of hardware type to run the model on. Default is recommended_config of the model.

Example: NVIDIA_H100_1

billing_intervalstring · enumOptional

The billing interval.

Example: per_minutePossible values:

context_lengthnumberOptional

Context length can be reduced if the full context length is not needed for the task. Decreasing the maximum context length increases the throughput of the model. Default is min(32000, max_context_length).

Example: 8192

num_workersintegerOptional

The number of workers to run the model on. Default is 1.

Example: 1

Responses

202

The instance is starting.

application/json

400

Invalid request - Missing or invalid parameters.

401

Unauthorized - Bearer token missing or invalid.

500

Internal server error.

post

/instances/start

HTTPcURLJavaScriptPython

POST /api/v1/instances/start HTTP/1.1
Host: cortecs.ai
Authorization: Bearer YOUR_SECRET_TOKEN
Content-Type: application/json
Accept: */*
Content-Length: 158

{
  "model_id": "meta-llama--Meta-Llama-3.1-8B-Instruct",
  "hardware_type_id": "NVIDIA_H100_1",
  "billing_interval": "per_minute",
  "context_length": 8192,
  "num_workers": 1
}

{
  "message": "Instance is starting.",
  "instance": {
    "instance_id": "instance-12345",
    "base_url": "https://abcd-123456.eu/v1",
    "instance_args": {
      "model_id": "meta-llama--Meta-Llama-3.1-8B-Instruct",
      "hardware_type_id": "NVIDIA_H100_1",
      "context_length": 8192,
      "billing_interval": "per_minute"
    },
    "instance_status": {
      "started_at": "2024-08-07T08:53:13Z",
      "stopped_at": null,
      "status": "running"
    },
    "worker_statuses": [
      {
        "init_progress": {
          "current_step": 5,
          "num_steps": 5,
          "description": "Running"
        }
      }
    ]
  }
}

Restart an instance

post

This endpoint restarts an instance based on the provided instance ID. The full instance is not available immediately after restarting, but the status can be checked using the /instances/{instance_id} endpoint. The request requires a Bearer token in the header.

Authorizations

bearerAuth

AuthorizationstringRequired

Bearer authentication header of the form Bearer <token>.

Body

application/json

instance_idstringRequired

The ID of the instance to restart.

Example: instance-12345

Responses

202

Successfully restarted the instance.

application/json

400

Invalid request - Missing or invalid parameters

401

Unauthorized - Bearer token missing or invalid

500

Internal server error

post

/instances/restart

HTTPcURLJavaScriptPython

POST /api/v1/instances/restart HTTP/1.1
Host: cortecs.ai
Authorization: Bearer YOUR_SECRET_TOKEN
Content-Type: application/json
Accept: */*
Content-Length: 32

{
  "instance_id": "instance-12345"
}

{
  "message": "Instance is starting.",
  "instance": {
    "instance_id": "instance-12345",
    "base_url": "https://abcd-123456.eu/v1",
    "instance_args": {
      "model_id": "meta-llama--Meta-Llama-3.1-8B-Instruct",
      "hardware_type_id": "NVIDIA_H100_1",
      "context_length": 8192,
      "billing_interval": "per_minute"
    },
    "instance_status": {
      "started_at": "2024-08-07T08:53:13Z",
      "stopped_at": null,
      "status": "running"
    },
    "worker_statuses": [
      {
        "init_progress": {
          "current_step": 5,
          "num_steps": 5,
          "description": "Running"
        }
      }
    ]
  }
}

Stop an instance

post

This endpoint stops an instance based on the provided instance ID. The request requires a Bearer token in the header.

Authorizations

bearerAuth

AuthorizationstringRequired

Bearer authentication header of the form Bearer <token>.

Body

application/json

instance_idstringRequired

The ID of the instance to stop.

Example: instance-12345

Responses

200

Successfully stopped the instance.

application/json

400

Invalid request - Missing or invalid parameters.

401

Unauthorized - Bearer token missing or invalid.

500

Internal server error.

post

/instances/stop

HTTPcURLJavaScriptPython

POST /api/v1/instances/stop HTTP/1.1
Host: cortecs.ai
Authorization: Bearer YOUR_SECRET_TOKEN
Content-Type: application/json
Accept: */*
Content-Length: 32

{
  "instance_id": "instance-12345"
}

{
  "message": "Instance stopped successfully",
  "instance": {
    "instance_id": "instance-12345",
    "base_url": "https://abcd-123456.eu/v1",
    "instance_args": {
      "model_id": "meta-llama--Meta-Llama-3.1-8B-Instruct",
      "hardware_type_id": "NVIDIA_H100_1",
      "context_length": 8192,
      "billing_interval": "per_minute"
    },
    "instance_status": {
      "started_at": "2024-08-07T08:53:13Z",
      "stopped_at": null,
      "status": "running"
    },
    "worker_statuses": [
      {
        "init_progress": {
          "current_step": 5,
          "num_steps": 5,
          "description": "Running"
        }
      }
    ]
  }
}

Stop all instances

post

This endpoint stops all running instances. The request requires a Bearer token in the header.

Authorizations

bearerAuth

AuthorizationstringRequired

Bearer authentication header of the form Bearer <token>.

Responses

200

Successfully stopped all instances.

application/json

401

Unauthorized - Bearer token missing or invalid.

500

Internal server error.

post

/instances/stop-all

HTTPcURLJavaScriptPython

POST /api/v1/instances/stop-all HTTP/1.1
Host: cortecs.ai
Authorization: Bearer YOUR_SECRET_TOKEN
Accept: */*

{
  "message": "All instances stopped successfully.",
  "instance_ids": [
    "instance-12345",
    "instance-67890"
  ]
}

Delete an instance

delete

This endpoint deletes an instance based on the provided instance ID. The instance must first be stopped before it can be deleted. To stop the instance, use the /instances/stop endpoint. The request requires a Bearer token in the header.

Authorizations

bearerAuth

AuthorizationstringRequired

Bearer authentication header of the form Bearer <token>.

Path parameters

instance_idstringRequired

The ID of the instance to delete.

Example: instance-12345

Responses

200

Successfully deleted the model instance.

application/json

400

Invalid request - Missing or invalid parameters.

401

Unauthorized - Bearer token missing or invalid.

500

Internal server error.

delete

/instances/{instance_id}

HTTPcURLJavaScriptPython

DELETE /api/v1/instances/{instance_id} HTTP/1.1
Host: cortecs.ai
Authorization: Bearer YOUR_SECRET_TOKEN
Accept: */*

{
  "message": "Instance deleted successfully.",
  "instance_id": "instance-12345"
}

Delete all instances

delete

This endpoint deletes all instances. If the force_deletion parameter is set to true, all instances will be deleted without stopping them first. Otherwise, only the stopped instances will be deleted. The request requires a Bearer token in the header.

Authorizations

bearerAuth

AuthorizationstringRequired

Bearer authentication header of the form Bearer <token>.

Query parameters

force_deletionbooleanOptional

Whether to force delete all instances without stopping them first.

Example: true

Responses

200

Successfully deleted all instances.

application/json

401

Unauthorized - Bearer token missing or invalid.

500

Internal server error.

delete

/instances/delete-all

HTTPcURLJavaScriptPython

DELETE /api/v1/instances/delete-all HTTP/1.1
Host: cortecs.ai
Authorization: Bearer YOUR_SECRET_TOKEN
Accept: */*

{
  "message": "All instances deleted successfully.",
  "instance_ids": [
    "instance-12345",
    "instance-67890"
  ]
}