Skip to content

v0.2.0
Compare
Choose a tag to compare
@yifanmai yifanmai released this 11 Jan 20:15
· 1063 commits to main since this release
v0.2.0
0f1433d
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.

Models

  • Added Aeph Alpha's Luminous models (#1215)
  • Added AI21's J1-Grande v2 beta model (#1177)
  • Added OpenAI's ChatGPT model (#1231)
  • Added OpenAI's text-davinci-003 model (#1200)

Scenarios

  • Added filtering by subject and level for MATHScenario (#1137)

Frontend

  • Reduced frontend JSON file sizes (#1185)
  • Added table sorting in frontend (#832)
  • Fixed frontend bugs for certain adapter methods (#1236, #1237)
  • Fixed frontend bugs for runs with multiple trials (#1211)

Adaptation

  • Improved sampling of in-context examples (#1172)
  • Internal refactor (#1280)

Result summarization

  • Added average win-rate computation for model-v-scenario tables (#1240)
  • Added additional calibration metrics as a "Targeted evaluation" (#1247)

Misc

  • Added documentation to Read the Docs (#1159, #1164)
  • Breaking schema change: input of Instance and output of Reference are now objects (#1280)
Assets 2
Loading