Token Count to variable

[Namec999]

Self Checks

• I have searched for existing issues search for existing issues, including closed ones.
• I confirm that I am using English to submit report (我已阅读并同意 Language Policy).
• [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:）
• Please do not modify this template :) and fill in all the required fields.

Provide a description of requested docs changes

Hello,

i am using an api response payload from an http request to feed my next llm node context.

some time, the payload is too large for the llm context.
is there a value returned by the http request or something elese, to check the size in tokens or in plain characaters. and decide if it' need to be sent to the llm or not.

best

[dosubot[bot]]

To check the size of an API response payload in tokens or characters, you can use the methods get_num_tokens and get_num_characters provided in the OCILargeLanguageModel class. These methods calculate the number of tokens and characters for given prompt messages, respectively.

Here's how you can use these methods:

# Assuming you have an instance of OCILargeLanguageModel and the necessary parameters
model = "your_model_name"
credentials = {"your_credentials_key": "your_credentials_value"}
prompt_messages = [PromptMessage(content="Your API response content")]

# To get the number of tokens
num_tokens = oci_large_language_model_instance.get_num_tokens(
    model=model,
    credentials=credentials,
    prompt_messages=prompt_messages
)

# To get the number of characters
num_characters = oci_large_language_model_instance.get_num_characters(
    model=model,
    credentials=credentials,
    prompt_messages=prompt_messages
)

print(f"Number of tokens: {num_tokens}")
print(f"Number of characters: {num_characters}")

These methods will help you determine if the payload should be sent to the LLM context based on its size in tokens or characters ^[1].

To continue talking to Dosu, mention @dosu.

---

^{Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Other}