How do I set token usage details using low level SDK?

[alanmun]

Here is what my code does:

def update_langfuse_trace(...):
    IDX_OF_ONLY_GENERATION = 0

    langfuse.trace(id=trace_id).update(metadata=usage)  # This merges with existing metadata

    # * For some reason, fetch_observation doesn't work, developers recommend using this way instead
    observations_result = langfuse.fetch_observations(trace_id=trace_id, type="GENERATION").data
    generation_id = observations_result[IDX_OF_ONLY_GENERATION].id

    langfuse.generation(
        id=generation_id,
        trace_id=trace_id,
        output=output,
        usage_details=usage,
    )

If I run the above code, I can see that the overall trace's metadata looks like:

{
    "input": 16329,
    "total": 16758,
    "output": 429,
    "model_id": "us.anthropic.claude-3-5-sonnet", <-was already in metadata
    "provider": "bedrock" <-was already in metadata
}

From the documentation, it looks like I made the usage_details object as desired. Above shows that my usage dict was:

{
    "input": 16329,
    "total": 16758,
    "output": 429,
}

How do I get these three to actually show up for overall traces? Right now I just see this in the traces page:

[dosubot[bot]]

To set token usage details using the low-level SDK so that the input, total, and output values show up for overall traces, you can use the usage_details parameter in the generation function. Here's an example of how to do it:

langfuse.generation(
    name="my-generation",
    usage_details={
        "input": 50,
        "output": 49,
        "total": 99,
    }
)

This will set the input, output, and total token usage details for the generation, which will be reflected in the overall traces ^[1].

Additionally, ensure that you are using the @observe(as_type="generation") decorator explicitly for generation spans. This configuration, along with updating the observation with additional context such as input, model, and usage details, ensures that the usage_details are recorded correctly in the trace metadata ^[2].

To continue talking to Dosu, mention @dosu.

---

^{Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Bug Report | Other}

[alanmun]

That is what I am doing, but its not working for langfuse 2.95.0, I am using low level sdk and not using any decorators.