Integrations

Dynamic provisioning out-of-the-box

LangChain

You can use the with statement to use a dedicated LLM. The DedicatedLLM class which automatically shuts down and deletes your instance as soon as the with block is left.

from cortecs_py import Cortecs
from cortecs_py.integrations.langchain import DedicatedLLM

cortecs = Cortecs()

with DedicatedLLM(cortecs, 'cortecs/phi-4-FP8-Dynamic') as llm:
    essay = llm.invoke('Write an essay about dynamic provisioning')
    print(essay.content)