Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[llava][4/N] Allow llama_transformer take embedding as optional argument #4257

Closed
wants to merge 7 commits into from
Closed

[llava][4/N] Allow llama_transformer take embedding as optional argument #4257

Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
cb7ef7a
[llava][4/N] Allow llama_transformer take embedding as optional argument
larryliu0820 Jul 12, 2024
5137f56
Update on "[llava][4/N] Allow llama_transformer take embedding as opt…
larryliu0820 Jul 12, 2024
e40b026
Update on "[llava][4/N] Allow llama_transformer take embedding as opt…
larryliu0820 Jul 12, 2024
a79994e
Update on "[llava][4/N] Allow llama_transformer take embedding as opt…
larryliu0820 Jul 16, 2024
59ab0df
Update on "[llava][4/N] Allow llama_transformer take embedding as opt…
larryliu0820 Jul 16, 2024
5c282c9
Update on "[llava][4/N] Allow llama_transformer take embedding as opt…
larryliu0820 Jul 16, 2024
1e9ad23
Update on "[llava][4/N] Allow llama_transformer take embedding as opt…
larryliu0820 Jul 16, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 10 additions & 4 deletions examples/models/llama2/llama_transformer.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -501,13 +501,19 @@ def __init__(self, params: ModelArgs):

def forward(
self,
tokens: torch.Tensor,
tokens: Optional[torch.LongTensor] = None, # tokens
input_pos: Optional[
torch.Tensor
torch.LongTensor
] = None, # Scalar tensor indicating size of window of the caches
h: Optional[torch.FloatTensor] = None, # embeddings
) -> torch.Tensor:
_bsz, seqlen = tokens.shape
h = self.tok_embeddings(tokens)
if (tokens is None) ^ (h is not None):
raise ValueError(
"You cannot specify both tokens and h at the same time, and must specify either one"
)
if tokens is not None and h is None:
h = self.tok_embeddings(tokens)
seqlen = h.shape[1]

if self.use_kv_cache:
assert (
Expand Down
Loading