Skip to content

Commit

Permalink
llama : document logits_all deprecation (ggerganov#4418)
Browse files Browse the repository at this point in the history
llama_context_params.logits_all is a parameter for controlling
llama_eval. This documents that logits_all should not be used with
llama_decode and llama_batch.
  • Loading branch information
crasm authored and teleprint-me committed Dec 21, 2023
1 parent 78149af commit 192a329
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion llama.h
Original file line number Diff line number Diff line change
Expand Up @@ -216,7 +216,7 @@ extern "C" {

// Keep the booleans together to avoid misalignment during copy-by-value.
bool mul_mat_q; // if true, use experimental mul_mat_q kernels (DEPRECATED - always true)
bool logits_all; // the llama_eval() call computes all logits, not just the last one
bool logits_all; // the llama_eval() call computes all logits, not just the last one (DEPRECATED - set llama_batch.logits instead)
bool embedding; // embedding mode only
bool offload_kqv; // whether to offload the KQV ops (including the KV cache) to GPU
};
Expand Down

0 comments on commit 192a329

Please sign in to comment.