Docs
cortecs.aiDedicated ModelsServerless ModelsLogin
  • Introduction
  • DEDICATED INFERENCE
    • Quickstart
    • Provisioning API
      • Authentication
      • User
      • Instances
      • Models
      • Hardware Types
    • Python client
      • Objects
      • Langchain integration
    • Examples
      • Batch jobs
      • Realtime streams
  • SERVERLESS INFERENCE
    • Quickstart
    • About Serverless Routing
    • API
      • Chat Completions
      • Models
  • Discord
Powered by GitBook
On this page
  1. DEDICATED INFERENCE
  2. Python client

Langchain integration

Dynamic provisioning out-of-the-box

You can use the with statement to use a dedicated LLM. The DedicatedLLM class which automatically shuts down and deletes your instance as soon as the with block is left.

from cortecs_py import Cortecs
from cortecs_py.integrations.langchain import DedicatedLLM

cortecs = Cortecs()

with DedicatedLLM(cortecs, 'cortecs/phi-4-FP8-Dynamic') as llm:
    essay = llm.invoke('Write an essay about dynamic provisioning')
    print(essay.content)
PreviousObjectsNextExamples

Last updated 11 days ago