Code

BLOOM #17474

Code

Merged

younesbelkada merged 195 commits intohuggingface:mainhuggingface/transformers:mainfrom younesbelkada:bigscience176byounesbelkada/transformers:bigscience176bCopy head branch name to clipboardJun 9, 2022

+2,582-3

Commits

Commits on May 12, 2022

• adding template

  

  thomwolf

  authored and

  younesbelkada

  committed
  c3a209e

• update model

  

  thomwolf

  authored and

  younesbelkada

  committed
  c08d29f

• model update

  

  thomwolf

  authored and

  younesbelkada

  committed
  95e2500

• update conf for debug model

  

  thomwolf

  authored and

  younesbelkada

  committed
  90232f9

• update conversion

  

  thomwolf

  authored and

  younesbelkada

  committed
  a439f1a

• update conversion script

  

  thomwolf

  authored and

  younesbelkada

  committed
  73cfd5f

• update conversion script

  

  thomwolf

  authored and

  younesbelkada

  committed
  c66f8c0

• fix missing keys check

  

  thomwolf

  authored and

  younesbelkada

  committed
  780e7c7

• add tests to test the tokenizer in the local machine

  younesbelkada

  committed
  7c30689

• Change variable name

  younesbelkada

  committed
  a9851a5

• add tests on xnli dataset

  younesbelkada

  committed
  e40ddf2

• add more description

  younesbelkada

  committed
  43173bd

• add descriptions + clearer code

  younesbelkada

  committed
  919839d

• clearer code

  younesbelkada

  committed
  e8c1e82

• adding new tests + skipping few tests because of env problems

  younesbelkada

  committed
  69c7928

• change comment

  younesbelkada

  committed
  267faac

• add dtype on the configuration

  younesbelkada

  committed
  e9ab687

• add test embeddings

  younesbelkada

  committed
  9f30687

• add hardcoded test

  younesbelkada

  committed
  16e018e

• fix dtype issue

  younesbelkada

  committed
  6923335

• adding torch.float16 to config

  younesbelkada

  committed
  4db67cf

• adding more metrics (min, max, mean)

  younesbelkada

  committed
  4a13037

• add sum

  younesbelkada

  committed
  32b77a4

• now the test passes with almost equal

  younesbelkada

  committed
  cf02880

• add files for conversion - test passes on cpu gpu

  younesbelkada

  committed
  9172116

• add final changes

  younesbelkada

  committed
  32ca7bc

• cleaning code

  younesbelkada

  committed
  a47a4e6

• add new args in the docstring

  younesbelkada

  committed
  f50d041

• fix one liner function

  younesbelkada

  committed
  c719264

• remove macros

  younesbelkada

  committed
  203b17b

• remove forward attention

  younesbelkada

  committed
  3c843ac

• clean up init funtion

  younesbelkada

  committed
  e2d845e

• add comments on the issue

  younesbelkada

  committed
  a2d3c0d

• rm scale mask softmax

  younesbelkada

  committed
  71240cd

• do make style

  younesbelkada

  committed
  ccf61a8

• fix dtype in init

  younesbelkada

  committed
  e6c802d

• fixing for loop on att probs

  younesbelkada

  committed
  4c3944a

• fix style with black

  younesbelkada

  committed
  44afc62

• fix style + doc error

  younesbelkada

  committed
  675c9ad

• fix and debug CI errors (docs + style)

  younesbelkada

  committed
  4fb6453

• some updates

  Show description for 69e0e8f
  younesbelkada

  committed
  69e0e8f

• make use cache working

  younesbelkada

  committed
  a0ece0f

• add changes

  Show description for c366f27
  younesbelkada

  committed
  c366f27

• add changes

  Show description for 975d3d6
  younesbelkada

  committed
  975d3d6

• test commit

  Show description for a6d9231
  

  younesbelkada

  and

  thomasw21

  committed
  a6d9231

• final changes

  Show description for e3437fb
  

  younesbelkada

  and

  thomasw21

  committed
  e3437fb

• changes - model + conversion

  younesbelkada

  committed
  3a710f4

• move to correct dir

  younesbelkada

  committed
  cc38b40

• put ,

  younesbelkada

  committed
  bef7ff9

• fex fixes

  younesbelkada

  committed
  417c77c

• fix tokenizer autodoc

  younesbelkada

  committed
  4bf01e9

• fix minor CI issues

  younesbelkada

  committed
  b79967c

• fix minor CI issues

  younesbelkada

  committed
  943e820

• fix minor CI issues

  younesbelkada

  committed
  9c8f385

• Merge branch 'main' into bigscience176b

  younesbelkada

  authored
  8400b48

• fix style issue

  younesbelkada

  committed
  489cb22

• fix minor import issues

  younesbelkada

  committed
  3168103

• fix few issues

  younesbelkada

  committed
  bf936c7

• remove def main on the test

  younesbelkada

  committed
  4d64adc

• add require torch

  younesbelkada

  committed
  8ba0768

• replace decorator with 'with'

  younesbelkada

  committed
  69a4675

• fix style

  younesbelkada

  committed
  c210216

Commits on May 13, 2022

• change to bloom

  younesbelkada

  committed
  fadd48c

• add quick fix tokenizer

  younesbelkada

  committed
  cc8457c

• fix tokenizer file

  younesbelkada

  committed
  afa817d

• fix tokenizer

  Show description for e580cb5
  younesbelkada

  committed
  e580cb5

• fix import issue

  younesbelkada

  committed
  19be820

• add bloom to readme

  younesbelkada

  committed
  4661d71

• fix consistency

  younesbelkada

  committed
  9ea7654

• Update docs/source/en/model_doc/bloom.mdx

  Show description for 0c693f3
  

  younesbelkada

  and

  sgugger

  authored
  0c693f3

• Apply suggestions from code review

  Show description for 56e5aab
  

  younesbelkada

  and

  sgugger

  authored
  56e5aab

• fix doc issue

  younesbelkada

  committed
  7109db1

• small fix - modeling test

  younesbelkada

  committed
  c9e7dd8

Commits on May 15, 2022

• some changes

  Show description for 1ac3e5e
  younesbelkada

  committed
  1ac3e5e

• remove useless division

  younesbelkada

  committed
  7f80ef3

Commits on May 16, 2022

• more tests should pass

  younesbelkada

  committed
  b748343

• more tests should pass

  younesbelkada

  committed
  886737b

Commits on May 17, 2022

• more tests should pass

  younesbelkada

  committed
  5f601a4

• let's try this one

  Show description for 40a529b
  younesbelkada

  committed
  40a529b

• Merge branch 'main' into bigscience176b

  younesbelkada

  authored
  3474a10

• refactor

  Show description for 21e3cb8
  younesbelkada

  committed
  21e3cb8

• major changes

  Show description for d784580
  younesbelkada

  committed
  d784580

• modify readme

  younesbelkada

  committed
  3687e53

• small fixes

  younesbelkada

  committed
  274b975

• small fix

  Show description for 181d011
  younesbelkada

  committed
  181d011

• remove old test file from fetcher

  younesbelkada

  committed
  9ccd19c

• fix small typo

  younesbelkada

  committed
  a5691f1

• major change

  Show description for 82022d4
  younesbelkada

  committed
  82022d4

Commits on May 18, 2022

• remove onnx config

  younesbelkada

  committed
  5d8da14

• major changes

  Show description for 0668055
  younesbelkada

  committed
  0668055

• make style

  younesbelkada

  committed
  967b6f2

• small change

  younesbelkada

  committed
  cdf41e8

• adding a slow test + commenting old ones for now

  younesbelkada

  committed
  20b4b32

• make style

  younesbelkada

  committed
  2861f30

• Apply suggestions from code review

  Show description for 4cfa5c7
  

  younesbelkada

  and

  sgugger

  authored
  4cfa5c7

• make style

  younesbelkada

  committed
  7f08d87

Commits on May 19, 2022

• fix duplicates

  younesbelkada

  committed
  8d49b08

• cleaning comments on config

  younesbelkada

  committed
  15e8509

• clean a bit conversion file

  younesbelkada

  committed
  79a8611

• refacor a bit modeling file

  younesbelkada

  committed
  9dea87f

• refactor tokenizer file

  younesbelkada

  committed
  d5fc5c1

• fix tokenization test issue

  younesbelkada

  committed
  1c5b48d

• fix tokenization issue #2

  younesbelkada

  committed
  5f91e98

• fix tokenization issue second try

  younesbelkada

  committed
  6305189

• Merge branch 'bigscience176b' of https://github.com/younesbelkada/transformers into bigscience176b

  younesbelkada

  committed
  734dc43

• fix test issue

  younesbelkada

  committed
  7d66d6f

• make style + add suggestions

  younesbelkada

  committed
  6fa2fc1

• change test fetcher

  younesbelkada

  committed
  49be5c8

Commits on May 20, 2022

• try this one

  Show description for c5d67c9
  younesbelkada

  committed
  c5d67c9

• possible final changes

  younesbelkada

  committed
  3d990f1

• make style

  younesbelkada

  committed
  8a98470

• try fix padding side issue

  younesbelkada

  committed
  5535e34

• fix side

  younesbelkada

  committed
  f0b5f5e

• fix padding issue

  younesbelkada

  committed
  4284e5e

• fix ko-readme

  younesbelkada

  committed
  de1f55c

• fix config auto

  younesbelkada

  committed
  531824a

• cleaning modeling file

  younesbelkada

  committed
  540f579

• keep bloom in caps in ko

  younesbelkada

  committed
  f5dcc7b

• update config docs

  younesbelkada

  committed
  9a017ff

• remove pretraining_pp

  younesbelkada

  committed
  40649de

• remove model parallel

  younesbelkada

  committed
  743631b

Commits on May 23, 2022

• update config

  Show description for 2706bf6
  younesbelkada

  committed
  2706bf6

• fix duplicates

  younesbelkada

  committed
  5b9a94b

• fix fetcher

  younesbelkada

  committed
  8a0999b

• fix refactor issue

  Show description for b4fd549
  younesbelkada

  committed
  b4fd549

Commits on May 24, 2022

• try to remove alibi

  younesbelkada

  committed
  17638f8

• small fixes

  Show description for 7db6afe
  younesbelkada

  committed
  7db6afe

• put correct values

  Show description for 13a565b
  younesbelkada

  committed
  13a565b

• fix attention mask loop

  Show description for 1450762
  

  younesbelkada

  and

  thomasw21

  committed
  1450762

• small fixes:

  Show description for 76be6d1
  younesbelkada

  committed
  76be6d1

Commits on May 25, 2022

• small fixes

  Show description for ad3b871
  younesbelkada

  committed
  ad3b871

• small changes

  Show description for 0ee5e5b
  younesbelkada

  committed
  0ee5e5b

• small changes

  Show description for a9e14b5
  younesbelkada

  committed
  a9e14b5

• small fixes

  Show description for ba04fec
  younesbelkada

  committed
  ba04fec

• major changes

  Show description for a643689
  younesbelkada

  committed
  a643689

• fix readmes

  younesbelkada

  committed
  69904bf

• major changes

  Show description for 3232a28
  younesbelkada

  committed
  3232a28

• refactor a bit

  younesbelkada

  committed
  a967a8a

• refactor a bit

  younesbelkada

  committed
  77eabf1

• put correct name on test

  younesbelkada

  committed
  8767fe8

• change docstring

  younesbelkada

  committed
  307cd5f

• small changes

  Show description for 690323a
  younesbelkada

  committed
  690323a

• fix small nit

  Show description for cce9498
  younesbelkada

  committed
  cce9498

Commits on May 26, 2022

• minor fix

  Show description for 88acef4
  younesbelkada

  committed
  88acef4

• minor fix

  Show description for b85e01a
  younesbelkada

  committed
  b85e01a

• forward contrib credits from PR14084

  

  sIncerass

  authored and

  younesbelkada

  committed
  e1db789

Commits on May 27, 2022

• Apply suggestions from code review

  Show description for 401ca5e
  

  younesbelkada

  and

  stas00

  authored
  401ca5e

• apply modifications

  Show description for bf219d2
  

  younesbelkada

  and

  stas00

  committed
  bf219d2

• resolve softmax upcast

  younesbelkada

  committed
  e768ab6

Commits on May 28, 2022

• Apply suggestions from code review

  Show description for ffd36c7
  

  younesbelkada

  and

  stas00

  authored
  ffd36c7

• Update src/transformers/models/bloom/modeling_bloom.py

  Show description for 733d7f8
  

  younesbelkada

  and

  Muennighoff

  authored
  733d7f8

• final changes modeling

  Show description for a3741d4
  

  younesbelkada

  and

  stas00

  committed
  a3741d4

Commits on May 30, 2022

• Merge commit 'd156898f3b9b2c990e5963f5030a7143d57921a2'

  younesbelkada

  committed
  06d98db

• merge commit

  younesbelkada

  committed
  6844146

• Apply suggestions from code review

  Show description for 1883390
  

  younesbelkada

  and

  stas00

  authored
  1883390

• apply suggestions

  Show description for bd3dbfa
  younesbelkada

  committed
  bd3dbfa

• Merge branch 'main' into bigscience176b

  younesbelkada

  committed
  ea1ab4e

Commits on May 31, 2022

• Fix gradient checkpointing

  Show description for 2969d59
  

  younesbelkada

  and

  stas00

  committed
  2969d59

• add slow but exact

  younesbelkada

  committed
  04efa88

• add accelerate compatibility

  Show description for 8662ebf
  

  younesbelkada

  and

  Narsil

  committed
  8662ebf

• forward contrib credits

  Show description for 1302638
  

  6 peoplecommitted
  1302638

• Apply suggestions from code review

  Show description for fe672cc
  

  younesbelkada

  and

  patrickvonplaten

  authored
  fe672cc

• fix torch device on tests

  younesbelkada

  committed
  3db958b

• make style

  younesbelkada

  committed
  d998c8c

• Apply suggestions from code review

  Show description for 47e5722
  

  younesbelkada

  and

  patrickvonplaten

  authored
  47e5722

• fix nits

  Show description for cd1f5c1
  younesbelkada

  committed
  cd1f5c1

• remove final nits

  younesbelkada

  committed
  1d45173

• fix doc

  Show description for 41d2573
  younesbelkada

  committed
  41d2573

• Update src/transformers/__init__.py

  Show description for a6e0b42
  

  younesbelkada

  and

  sgugger

  authored
  a6e0b42

• Update src/transformers/models/bloom/modeling_bloom.py

  Show description for 56e2928
  

  younesbelkada

  and

  sgugger

  authored
  56e2928

• apply suggestions

  Show description for 5003f50
  

  younesbelkada

  and

  sgugger

  committed
  5003f50

• put test torchscript to false

  younesbelkada

  committed
  8f3bffe

Commits on Jun 1, 2022

• Update src/transformers/models/bloom/modeling_bloom.py

  Show description for ba1d9fc
  

  younesbelkada

  and

  justheuristic

  authored
  ba1d9fc

• fix alibi

  Show description for 95f8dc7
  younesbelkada

  committed
  95f8dc7

• add small doc

  younesbelkada

  committed
  948e0fb

• make quality

  younesbelkada

  committed
  694ee1c

Commits on Jun 2, 2022

• replace torch.nn

  younesbelkada

  committed
  7d026b4

Commits on Jun 3, 2022

• remove token type emb

  younesbelkada

  committed
  9c2f84a

• fix fused op + output bias

  younesbelkada

  committed
  1ca7c1a

• add fused op

  Show description for fab4f62
  younesbelkada

  committed
  fab4f62

• remove fused op

  younesbelkada

  committed
  bd0bd2f

• make quality

  younesbelkada

  committed
  8b481ce

Commits on Jun 5, 2022

• small changes

  Show description for 2907274
  younesbelkada

  committed
  2907274

• Update src/transformers/models/bloom/modeling_bloom.py

  younesbelkada

  authored
  bce5132

• fix slow

  younesbelkada

  committed
  5eecde7

• make style

  younesbelkada

  committed
  4d54e7f

Commits on Jun 6, 2022

• add accelerate support

  younesbelkada

  committed
  a24ae86

• add bloom to deepspeed tests

  stas00

  committed
  f723b3e

Commits on Jun 8, 2022

• minor changes

  younesbelkada

  committed
  fadb367

• Apply suggestions from code review

  Show description for 8ef8f6d
  

  younesbelkada

  and

  patrickvonplaten

  authored
  8ef8f6d

• minor change

  younesbelkada

  committed
  1d63415

• slow tests pass

  younesbelkada

  committed
  24c1df8

• Apply suggestions from code review

  Show description for de5b1c2
  

  younesbelkada

  and

  sgugger

  authored
  de5b1c2

• Update docs/source/en/model_doc/bloom.mdx

  Show description for 6262d77
  

  younesbelkada

  and

  sgugger

  authored
  6262d77

• minor changes:

  Show description for 3a59eb3
  younesbelkada

  committed
  3a59eb3