Models

Retrieve all available models

This endpoint retrieves information about all available models. The request requires a Bearer token in the header.

GEThttps://cortecs.ai/api/v1/models/availableModels

Response

200401500

A list of available models

Body

application/json

modelsarray of object

Request

JavaScriptCurlPython

const response = await fetch('https://cortecs.ai/api/v1/models/availableModels', {
    method: 'GET',
    headers: {},
});
const data = await response.json();

Response

200

{
  "models": [
    {
      "screen_name": "Llama 3.1 70B",
      "model_name": "meta-llama--Meta-Llama-3.1-70B-Instruct",
      "hf_name": "meta-llama/Meta-Llama-3.1-70B-Instruct",
      "size": 70600000000,
      "context_length": 16000,
      "creator": {
        "name": "Meta",
        "url": "https://ai.meta.com"
      },
      "description": "The Llama 3.1 instruction tuned text only models...",
      "prompt_example": "<|begin_of_text|><|start_header_id|>user<|end_header_id|> Tell me a joke.<|eot_id|><|start_header_id|>assistant<|end_header_id|>",
      "hardware_info": {
        "comment": "The Llama-3.1 model, boasting 70 billion parameters...",
        "required_VRAM_GB": 141.11,
        "recommended_config": "NVIDIA_H100_2",
        "hardware_configs": [
          "NVIDIA_H100_2"
        ]
      }
    }
  ]
}

Get detailed information about a model

This endpoint returns detailed information of the passed model including evaluations and hardware configs.

GEThttps://cortecs.ai/api/v1/models/model

Query parameters

Response

200

Successfully returned information about a model

Body

application/json

created_atstring

screen_namestring

model_nameinteger

image_namestring

hf_namestring

licensestring

sizeinteger

context_lengthinteger

creatorobject

quantizationstring

descriptionstring

tagsarray of string

recommended_promptstring

prompt_examplestring

evaluationobject

bitsinteger

hardware_infoobject

required_VRAM_GBnumber

recommended_configstring

hardware_configsobject

Request

JavaScriptCurlPython

const response = await fetch('https://cortecs.ai/api/v1/models/model?modelName=text', {
    method: 'GET',
    headers: {},
});
const data = await response.json();

Response

200

[
  {
    "created_at": "text",
    "screen_name": "text",
    "image_name": "text",
    "hf_name": "text",
    "license": "text",
    "quantization": "text",
    "description": "text",
    "tags": [
      "text"
    ],
    "recommended_prompt": "text",
    "prompt_example": "text",
    "required_VRAM_GB": 0,
    "recommended_config": "text"
  }
]

Start a model instance

This endpoint starts a model instance based on the provided model name and instance type. The request requires a Bearer token in the header.

POSThttps://cortecs.ai/api/v1/models/startModel

Authorization

bearerAuth

Body

application/json

model_name*string

The name of the model to start.

Example: "meta-llama--Meta-Llama-3.1-70B-Instruct"

instance_namestring

The type of instance to run the model on. Default is recommended instance type of the model.

Example: "NVIDIA_H100_2"

billing_intervalstring

per_minute (default) or per_hour

Example: "per_minute"

context_lengthnumber

Context length can be reduced if full context length is not needed for the current task. Default is max_context_length of model

Example: 4096

Response

200400401500

Successfully started the model instance

Body

application/json

messagestring

Status message indicating that the instance is starting.

Example: "Instance is starting"

idstring

The ID of the started model instance.

Example: "instance-12345"

Request

JavaScriptCurlPython

const response = await fetch('https://cortecs.ai/api/v1/models/startModel', {
    method: 'POST',
    headers: {
      "Authorization": "Bearer JWT",
      "Content-Type": "application/json"
    },
    body: JSON.stringify({
      "model_name": "meta-llama--Meta-Llama-3.1-70B-Instruct"
    }),
});
const data = await response.json();

Response

200

{
  "message": "Instance is starting",
  "id": "instance-12345"
}

Stop a model instance

This endpoint stops a model instance based on the provided instance ID. The request requires a Bearer token in the header.

POSThttps://cortecs.ai/api/v1/models/stopModel

Authorization

bearerAuth

Body

application/json

idstring

The ID of the model instance to stop.

Example: "instance-12345"

Response

200400401500

Successfully stopped the model instance

Body

application/json

messagestring

Status message indicating that the instance was stopped.

Example: "Instance stopped successfully"

instanceStoppedboolean

Indicates whether the instance was successfully stopped.

Example: true

Request

JavaScriptCurlPython

const response = await fetch('https://cortecs.ai/api/v1/models/stopModel', {
    method: 'POST',
    headers: {
      "Authorization": "Bearer JWT",
      "Content-Type": "application/json"
    },
    body: JSON.stringify({}),
});
const data = await response.json();

Response

200

{
  "message": "Instance stopped successfully",
  "instanceStopped": true
}

Delete a model instance

This endpoint deletes a model instance based on the provided instance ID. The request requires a Bearer token in the header.

POSThttps://cortecs.ai/api/v1/models/deleteModel

Authorization

bearerAuth

Body

application/json

idstring

The ID of the model instance to delete.

Example: "instance-12345"

Response

200400401500

Successfully deleted the model instance

Body

application/json

messagestring

Status message indicating that the instance was deleted.

Example: "Model deleted successfully"

modelDeletedboolean

Indicates whether the model instance was successfully deleted.

Example: true

Request

JavaScriptCurlPython

const response = await fetch('https://cortecs.ai/api/v1/models/deleteModel', {
    method: 'POST',
    headers: {
      "Authorization": "Bearer JWT",
      "Content-Type": "application/json"
    },
    body: JSON.stringify({}),
});
const data = await response.json();

Response

200

{
  "message": "Model deleted successfully",
  "modelDeleted": true
}

Restart a model instance

This endpoint restarts a model instance based on the provided instance ID. The request requires a Bearer token in the header.

POSThttps://cortecs.ai/api/v1/models/restartModel

Authorization

bearerAuth

Body

application/json

idstring

The ID of the model instance to restart.

Example: "instance-12345"

Response

200400401500

Successfully restarted the model instance

Body

application/json

messagestring

Status message indicating that the instance is starting.

Example: "Instance is starting"

idstring

The ID of the model instance that is restarting.

Example: "instance-12345"

Request

JavaScriptCurlPython

const response = await fetch('https://cortecs.ai/api/v1/models/restartModel', {
    method: 'POST',
    headers: {
      "Authorization": "Bearer JWT",
      "Content-Type": "application/json"
    },
    body: JSON.stringify({}),
});
const data = await response.json();

Response

200

{
  "message": "Instance is starting",
  "id": "instance-12345"
}

Get status of model instances

This endpoint returns the status of all model instances (started and stopped) associated with the authenticated user. The request requires a Bearer token in the header.

GEThttps://cortecs.ai/api/v1/models/myModels

Authorization

bearerAuth

Response

200401500

Successfully retrieved model instance statuses

Body

application/json

instanceStatusarray of object

Request

JavaScriptCurlPython

const response = await fetch('https://cortecs.ai/api/v1/models/myModels', {
    method: 'GET',
    headers: {
      "Authorization": "Bearer JWT"
    },
});
const data = await response.json();

Response

200

{
  "instanceStatus": [
    {
      "id": "eacb7263-e037-425d-892d-a81dddab1ed8",
      "started_at": "2024-08-21T13:01:10.638Z",
      "stopped_at": "2024-08-22T04:33:11.807Z",
      "model_id": "meta-llama--Meta-Llama-3.1-8B-Instruct",
      "model_screen_name": "Llama 3.1 8B",
      "model_status": "stopped",
      "instance_name": "NVIDIA_L4_1",
      "instance_screen_name": "NVIDIA L4 x 1",
      "init_status": "init_status: {status: 3, description: download model, num_steps: 4}",
      "domain": "abcd-123456.cortecs.ai:8081",
      "billing_interval": "per_minute",
      "context_length": 4096
    }
  ]
}