Code

GSM8K Guidance Benchmark#50

Code

Open

riedgar-ms wants to merge 37 commits intomainmicrosoft/promptbase:mainfrom riedgar-ms/gsm8k-benchmarkmicrosoft/promptbase:riedgar-ms/gsm8k-benchmarkCopy head branch name to clipboard

+1,088

Commits

Commits on Apr 9, 2024

• Starting to figure out data

  riedgar-ms

  committed
  1d8ec5b

• Make sure we can process data

  riedgar-ms

  committed
  cc7fcc1

• Add component definition YAML

  riedgar-ms

  committed
  b6f44fb

• Trying to get to first submission

  riedgar-ms

  committed
  5605b2f

• Give up on locale :-/

  riedgar-ms

  committed
  18bbb22

• Merge remote-tracking branch 'origin/main' into riedgar-ms/gsm8k-benchmark

  riedgar-ms

  committed
  5be4bed

• Drafting a component

  riedgar-ms

  committed
  6fa1f93

• Roughing out the mistral7b component

  riedgar-ms

  committed
  98405aa

• Roughing out more changes

  riedgar-ms

  committed
  1e932cd

• Better name

  riedgar-ms

  committed
  2935835

Commits on Apr 10, 2024

• Hack things into working

  riedgar-ms

  committed
  d16df86

• Rename things to match switch to LlamaCpp

  riedgar-ms

  committed
  d3b2a4e

• Add a scoring component

  riedgar-ms

  committed
  033c407

• Start expanding on the prompts

  riedgar-ms

  committed
  cb56759

• Unsuccessful tweak

  riedgar-ms

  committed
  ff305b2

• Add some random examples

  riedgar-ms

  committed
  b8d25bf

• Thinking about few shots a bit

  riedgar-ms

  committed
  ac49c39

• Doing some renaming

  riedgar-ms

  committed
  6f8ef9d

• Tweaking

  riedgar-ms

  committed
  2076b53

• Some token limits

  riedgar-ms

  committed
  d60345c

Commits on Apr 11, 2024

• More involved JSON

  riedgar-ms

  committed
  1cfefad

Commits on Apr 13, 2024

• Make sure pydantic and jsonschema are available

  riedgar-ms

  committed
  0714d88

Commits on Apr 15, 2024

• Fix regex for a number

  riedgar-ms

  committed
  6f54a51

• Increase context window for model

  riedgar-ms

  committed
  1bbc53d

Commits on Apr 17, 2024

• Create a JSONL sampling component

  riedgar-ms

  committed
  88ef9ef

• Hook new component into pipeline

  riedgar-ms

  committed
  85657f3

• Try tweaking the prompts (and saving)

  riedgar-ms

  committed
  53d4fe5

• Fix division by zero

  riedgar-ms

  committed
  4ec2b82

• Want mlflow available

  riedgar-ms

  committed
  bf85df8

• Try some more logging

  riedgar-ms

  committed
  ade5579

• Tweaking guidance programs

  riedgar-ms

  committed
  542b7c8

Commits on Apr 18, 2024

• Try some extra new lines

  riedgar-ms

  committed
  8a2afa5

• Small tweaks and linting

  riedgar-ms

  committed
  e51a025

• Bad stop for reasons

  riedgar-ms

  committed
  622be28

• Dumb mistakes in json formatted response

  riedgar-ms

  committed
  a90380e

Commits on Apr 19, 2024

• Another silly mistake

  riedgar-ms

  committed
  db91f01

Commits on Apr 21, 2024

• Need Rust

  riedgar-ms

  committed
  94c465c