Sky inference
cortecs.aiDedicated ModelsServerless ModelsLogin
  • Introduction
  • DEDICATED INFERENCE
    • Quickstart
    • Provisioning API
      • Authentication
      • User
      • Instances
      • Models
      • Hardware Types
    • Python client
      • Objects
      • Langchain integration
    • Examples
      • Batch jobs
      • Realtime streams
  • SERVERLESS INFERENCE
    • Quickstart
    • About Serverless Routing
    • API
      • Chat Completions
      • Models
  • Discord
Powered by GitBook
On this page
  1. Getting started

LLM Workers

Dedicated infrastructure on the fly

Last updated 4 days ago

Accessing the API requires API credentials, which you get on your. Once we have a key we'll want to set our environment variables by running:

export CORTECS_CLIENT_ID="<YOUR_CLIENT_ID>"
export CORTECS_CLIENT_SECRET="<YOUR_CLIENT_SECRET>"
export OPENAI_API_KEY="<YOUR_CORTECS_API_KEY>"

Alternatively you can pass them directly to the :

from cortecs_py import Cortecs

client = Cortecs(client_id="<YOUR_CLIENT_ID>", client_secret="<YOUR_CLIENT_ID>") 
...

Example

from openai import OpenAI
from cortecs_py import Cortecs

cortecs = Cortecs()
my_model = 'cortecs/phi-4-FP8-Dynamic'

# Start a new instance
my_instance = cortecs.ensure_instance(my_model)
client = OpenAI(base_url=my_instance.base_url)

completion = client.chat.completions.create(
  model=my_model,
  messages=[
    {"role": "user", "content": "Write a joke about LLMs."}
  ]
)
print(completion.choices[0].message.content)

cortecs.stop(my_instance.instance_id)
profile page
client