Langchain integration
Dynamic provisioning out-of-the-box
You can use the with
statement to use a dedicated LLM. The DedicatedLLM
class which automatically shuts down and deletes your instance as soon as the with
block is left.
from cortecs_py import Cortecs
from cortecs_py.integrations.langchain import DedicatedLLM
cortecs = Cortecs()
with DedicatedLLM(cortecs, 'cortecs/phi-4-FP8-Dynamic') as llm:
essay = llm.invoke('Write an essay about dynamic provisioning')
print(essay.content)
Last updated