Code

[Release] lmms-eval v0.3.0 release#428

Code

Merged

kcz358 merged 26 commits intomainEvolvingLMMs-Lab/lmms-eval:mainfrom feat-devEvolvingLMMs-Lab/lmms-eval:feat-devCopy head branch name to clipboardNov 27, 2024

+9,828-18

Commits

Commits on Nov 27, 2024

• [Feat] Add qwen2_audio model support and Automatic speech recognition task with LibriSpeech dataset (#289)

  Show description for 3e53152
  

  2 peopleauthored and

  kcz358

  committed
  3e53152

  Copy full SHA for 3e53152

• add clotho_aqa task

  

  pbcong

  authored and

  kcz358

  committed
  6c17c18

  Copy full SHA for 6c17c18

• Apply black formatting

  

  pbcong

  authored and

  kcz358

  committed
  9b32985

  Copy full SHA for 9b32985

• formatting

  

  pbcong

  authored and

  kcz358

  committed
  396c89a

  Copy full SHA for 396c89a

• excluding xl due to downloading issue.

  

  Luodian

  authored and

  kcz358

  committed
  8f03304

  Copy full SHA for 8f03304

• [Feat] add audiobench version of clothoaqa (#302)

  Show description for 7fa93c9
  

  pbcong

  authored and

  kcz358

  committed
  7fa93c9

  Copy full SHA for 7fa93c9

• Add AIR_bench task (#315)

  Show description for e1deb38
  

  pbcong

  authored and

  kcz358

  committed
  e1deb38

  Copy full SHA for e1deb38

• add common_voice_15 and people_speech tasks (#316)

  Show description for 763d768
  

  2 peopleauthored and

  kcz358

  committed
  763d768

  Copy full SHA for 763d768

• add indent to yaml

  

  Yingluo-momo

  authored and

  kcz358

  committed
  e8cc550

  Copy full SHA for e8cc550

• Add openhermes task (#323)

  Show description for 6634344
  

  pbcong

  authored and

  kcz358

  committed
  6634344

  Copy full SHA for 6634344

• [Refactor] Fixing doc to audio return type, qwen_audio revise (#329)

  Show description for 6d3392a
  kcz358

  committed
  6d3392a

  Copy full SHA for 6d3392a

• add muchomusic and vocalsound task (#331)

  

  pbcong

  authored and

  kcz358

  committed
  d7d5c69

  Copy full SHA for d7d5c69

• add alpaca audio task (#333)

  

  pbcong

  authored and

  kcz358

  committed
  016eeef

  Copy full SHA for 016eeef

• [feat] added gigaspeech config (#334)

  Show description for c0996fa
  

  2 peopleauthored and

  kcz358

  committed
  c0996fa

  Copy full SHA for c0996fa

• add tedlium_long_form and tedlium_dev_test tasks (#345)

  Show description for b154837
  

  2 peopleauthored and

  kcz358

  committed
  b154837

  Copy full SHA for b154837

• [Feat] add-wavcaps (#349)

  Show description for 8c1618c
  

  2 peopleauthored and

  kcz358

  committed
  8c1618c

  Copy full SHA for 8c1618c

• Update dep and fix log samples for audio (#355)

  Show description for 9e4fc32
  kcz358

  committed
  9e4fc32

  Copy full SHA for 9e4fc32

• fix vocalsound (#362)

  

  pbcong

  authored and

  kcz358

  committed
  0f94b16

  Copy full SHA for 0f94b16

• Add using simple prompt for Qwen2 Audio to align (#360)

  kcz358

  committed
  b422011

  Copy full SHA for b422011

• Add retry for gpt api call and improve air_bench aggregation function (#376)

  Show description for a7f339c
  

  pbcong

  authored and

  kcz358

  committed
  a7f339c

  Copy full SHA for a7f339c

• [Feat] Add mix_evals audio2text (#420)

  Show description for f2cbf0c
  kcz358

  committed
  f2cbf0c

  Copy full SHA for f2cbf0c

• Gemini Audio (#421)

  Show description for aabb021
  

  pufanyi

  authored and

  kcz358

  committed
  aabb021

  Copy full SHA for aabb021

• Revise prompt

  kcz358

  committed
  30a47b3

  Copy full SHA for 30a47b3

• delete redundant tasks in gigaspeech

  

  KairuiHu

  authored and

  kcz358

  committed
  8647c9c

  Copy full SHA for 8647c9c

• Fix wavcaps bugs

  kcz358

  committed
  fcb9464

  Copy full SHA for fcb9464

• Add lmms-eval-0.3 docs

  Show description for 59e81b0
  

  kcz358

  and

  KairuiHu

  committed
  59e81b0

  Copy full SHA for 59e81b0