Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Spec infer demo #724

Merged
merged 63 commits into from
May 16, 2023
Merged

Spec infer demo #724

merged 63 commits into from
May 16, 2023

Conversation

jiazhihao
Copy link
Collaborator

Description of changes:

Preparing a demo for SpecInfer

Related Issues:

Linked Issues:

  • Issue #

Issues closed by this PR:

  • Closes #

Before merging:

  • Did you update the flexflow-third-party repo, if modifying any of the Cmake files, the build configs, or the submodules?

jiazhihao and others added 18 commits April 29, 2023 00:21
@jiazhihao
Support multiple FFModels in a single top_level_task
5c2e3e7
@jiazhihao
resolve merge conflict
f31d2a7
@jiazhihao
Merge branch 'inference' of https://github.com/flexflow/FlexFlow into…
6de473b 
… inference
@jiazhihao
[TreeVerifyMHA] bug fixes
7ca3c3f
@jiazhihao
bug fixes
dcf2101
@jiazhihao
Merge branch 'inference' of https://github.com/flexflow/FlexFlow into…
940ca61 
… tree_verify_bug_fix
@jiazhihao
TreeIncMHA and SpecIncMHA bug fixes
2ce12c6
@xinhaoc
fomat.
869bb91
@jiazhihao
.
a7abb1f
@jiazhihao
resolve conflict
d9e977d
@jiazhihao
add sentence piece tokenizer
545154d
@jiazhihao
Merge branch 'tree_verify_bug_fix' of https://github.com/flexflow/Fle…
6a5d6be 
…xFlow into tree_verify_bug_fix
@jiazhihao
format
a6063f3
@jiazhihao
Merge branch 'inference' of https://github.com/flexflow/FlexFlow into…
198f68b 
… tree_verify_bug_fix
@jiazhihao
prepare spec_infer demo
12fc1b0
@jiazhihao
prettier prints
65b18ae
@jiazhihao
make the llama model work
7531b70
@xinhaoc
add small model config
7a570c6
@jiazhihao jiazhihao mentioned this pull request May 14, 2023
Closed
1 task
jiazhihao and others added 29 commits May 15, 2023 20:30
@jiazhihao
format
a0d897f
@jiazhihao
move files
e41ccbf
@jiazhihao
Update README.md
27da033
@jiazhihao
add an overview figure
fd86d3b
@jiazhihao
Merge branch 'spec_infer_demo' of https://github.com/flexflow/FlexFlow …
bc6f173 
…into spec_infer_demo
@jiazhihao
update images
51c34de
@jiazhihao
Update README.md
c0d12f7
@jiazhihao
Update README.md
93e6c23
@jiazhihao
Update README.md
e396f39
@jiazhihao
Update README.md
6d4860e
@jiazhihao
Update README.md
cac16b5
@xinhaoc
add tokenizer in readme
363f1f7
@xinhaoc
fix
a355f4b
@xinhaoc
fix
093fe87
@xinhaoc
fix
c7ca42e
@jiazhihao
Update README.md
c2cac57
@jiazhihao
Update README.md
3c2af1a
@jiazhihao
add gif
d96285e
@jiazhihao
Merge branch 'spec_infer_demo' of https://github.com/flexflow/FlexFlow …
52efadd 
…into spec_infer_demo
@xinhaoc
add weights to readme, clean some print
969d919
@jiazhihao
Update README.md
88edcb0
@jiazhihao
update demo
666fd00
@jiazhihao
Merge branch 'spec_infer_demo' of https://github.com/flexflow/FlexFlow …
abad659 
…into spec_infer_demo
@jiazhihao
Update README.md
457e8f8
@xinhaoc
Update README.md
dcd799c
@xinhaoc
remove outdate file
a070549
@jiazhihao
Update README.md
a992c64
@jiazhihao
Update README.md
5cf8600
@jiazhihao
.
3dd5a7d
@jiazhihao jiazhihao merged commit 555aa33 into inference May 16, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@jiazhihao @xinhaoc @goliaro