Langfuse

Open Source LLM Engineering Platform

Langfuse is an open-source LLM engineering platform that helps teams collaboratively debug, analyze, and iterate on their LLM applications.

All platform features are natively integrated to accelerate the development workflow. Langfuse is open, self-hostable, and extensible.

This guide will walk you through the setup.

Before you begin: Make sure you have have generated your Cortecs API key. If not, check out our QuickStart guide.

1. Deploy Langfuse

There are multiple ways to deploy Langfuse.

To get started quickly, choose your preferred deployment strategy and follow the instructions in the documentation.

Once deployment is complete, you’ll need to configure an external provider.

2. Connect to cortecs

Cortecs provides OpenAI-compatible API endpoints, making it easy to integrate with Langfuse and many other tools.

2.1. Prompt Management

One of the core features of Langfuse is prompt management, which includes storing, versioning, and retrieving your prompts. To experiment with prompts, Langfuse needs to be connected to an LLM provider such as Cortecs.

To set up the connection, follow these steps in the administration console:

1. Navigate to Settings -> LLM-Connections

2. Configure the connection with the following settings: – LLM adapter: openai – API Key: Your API Key – API Base URL: https://api.cortecs.ai/v1 – Enable default models: Disabled – Add custom model name: gpt-5-mini (feel free to pick another one from the catalogue)

Once the LLM connection is created, you can experiment with prompts, store versions, and later retrieve and use them directly in your code.

from langfuse._client.get_client import get_client

langfuse = get_client()
prompt = langfuse.get_prompt("Friend")
compiled_prompt = prompt.compile(name="Freddy")

2.2. Tracing

Langfuse provides tracing capabilities to monitor and analyze the calls made to an LLM. This feature allows you to gain visibility into model interactions and performance. To integrate tracing, Langfuse offers multiple options, including drop-in replacements and flexible connection methods, making it easy to incorporate into your existing codebase.

from langfuse.openai import openai

client = openai.OpenAI(
    api_key="eyJhbG***",
    base_url="https://api.cortecs.ai/v1"
)

completion = client.chat.completions.create(
    model="gpt-5-mini",
    messages=[
        {"role": "system", "content": "You are a professional comedian."},
        {"role": "user", "content": "Tell me a joke."}],
    stream=True,
    extra_body={"preference": "balanced"}
)

for chunk in completion:
  print(chunk.choices[0].delta.content)

Enjoy your privacy-preserving LLM engineering platform with the power of Cortecs and Langfuse. Explore other models, tweak the setup to your needs, and join the conversation on our Discord to share feedback 👩‍💻👨‍💻