Skip to content

Benchmarking LLMs (large language models) on leetcode algorithmic challenges

Notifications You must be signed in to change notification settings

whisk/leetgptsolver

Repository files navigation

leetgptsolver

Research to identify the capabilities of LLMs in solving coding interview problems.

TODO

  • Make command-line help self-explanatory
  • Improve reporting to generate short ready-to-publish reports
  • Make logging/output more informative, especially when downloading questions
  • Add version information
  • Detect a question creation time
  • Implement a simple custom downloading queue
  • Add jq filtering for reporting
  • Replace JSON files with a local database for questions
  • Implement a real rate limiter instead of SimpleThrottler

Usage

To interact with leetcode.com, please sign up and sign in with your Leetcode account using the Chrome browser.

Misc

"2603. Collect Coins in a Tree" - probably the last from March 2023 "2663. Lexicographically Smallest Beautiful String" - probably the first from May 2023

About

Benchmarking LLMs (large language models) on leetcode algorithmic challenges

Topics

Resources

Stars

Watchers

Forks

Languages