-
Notifications
You must be signed in to change notification settings - Fork 312
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[llava][4/N] Allow llama_transformer take embedding as optional argument #4257
Conversation
Summary: Llava run token embedding separately and combine that with image embeddings as input to text transformer. This PR adds support for that. Test Plan: Reviewers: Subscribers: Tasks: Tags: [ghstack-poisoned]
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/4257
Note: Links to docs will display an error until the docs builds have been completed. ✅ No FailuresAs of commit 1e9ad23 with merge base fbe0af1 (): This comment was automatically generated by Dr. CI and updates every 15 minutes. |
Summary: Llava run token embedding separately and combine that with image embeddings as input to text transformer. This PR adds support for that. Test Plan: Reviewers: Subscribers: Tasks: Tags: ghstack-source-id: 398c39d3b398d417e215095982c8e0dfd5fa8090 Pull Request resolved: #4257
…ional argument" Summary: Llava run token embedding separately and combine that with image embeddings as input to text transformer. This PR adds support for that. Test Plan: Reviewers: Subscribers: Tasks: Tags: [ghstack-poisoned]
Summary: Llava run token embedding separately and combine that with image embeddings as input to text transformer. This PR adds support for that. Test Plan: Reviewers: Subscribers: Tasks: Tags: ghstack-source-id: 9cf46d027d5502ec03f65c597c2769b9cb2e6506 Pull Request resolved: #4257
…ional argument" Summary: Llava run token embedding separately and combine that with image embeddings as input to text transformer. This PR adds support for that. Test Plan: Reviewers: Subscribers: Tasks: Tags: [ghstack-poisoned]
@larryliu0820 has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
…ional argument" Summary: Llava run token embedding separately and combine that with image embeddings as input to text transformer. This PR adds support for that. Test Plan: Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D59759977](https://our.internmc.facebook.com/intern/diff/D59759977) [ghstack-poisoned]
@larryliu0820 has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
…ional argument" Summary: Llava run token embedding separately and combine that with image embeddings as input to text transformer. This PR adds support for that. Test Plan: Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D59759977](https://our.internmc.facebook.com/intern/diff/D59759977) [ghstack-poisoned]
@larryliu0820 has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
…ional argument" Summary: Llava run token embedding separately and combine that with image embeddings as input to text transformer. This PR adds support for that. Test Plan: Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D59759977](https://our.internmc.facebook.com/intern/diff/D59759977) [ghstack-poisoned]
@larryliu0820 has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
…ional argument" Summary: Llava run token embedding separately and combine that with image embeddings as input to text transformer. This PR adds support for that. Test Plan: Reviewers: Subscribers: Tasks: Tags: Differential Revision: [D59759977](https://our.internmc.facebook.com/intern/diff/D59759977) [ghstack-poisoned]
@larryliu0820 has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator. |
This pull request has been merged in 8a1589d. |
Stack from ghstack (oldest at bottom):
Summary: Llava run token embedding separately and combine that with
image embeddings as input to text transformer. This PR adds support for
that.
Test Plan:
Reviewers:
Subscribers:
Tasks:
Tags:
Differential Revision: D59759977