mshumer/gpt-prompt-engineer - GitHub vs ZenPrompts

mshumer/gpt-prompt-engineer is an open source prompt engineering toolkit that generates, tests, and ranks candidate prompts for a task you define. You describe the use case, supply test cases, and the notebooks create multiple prompt variants, run them against every test case, and sort results with an ELO rating system starting at 1200. It ships as Jupyter notebooks you can run in Google Colab or locally.

Most prompt tools help you write one prompt at a time. gpt-prompt-engineer treats prompt selection like a tournament: dozens of candidates compete on your test cases, and the highest ELO scores surface the winners. Separate notebooks cover classification tasks, Claude 3 Opus with auto-generated test cases, and Opus-to-Haiku conversion for cheaper inference. Optional Weights & Biases and Portkey logging trace each run.

ML engineers, prompt engineers, and AI developers use it when they need reproducible prompt tuning instead of manual trial and error. The repo has 9.7k GitHub stars and runs on your own OpenAI or Anthropic API keys. It is free under the MIT license.

ZenPrompts is a cutting-edge platform designed to empower prompt engineers in the creation, refinement, testing, and sharing of sophisticated prompts for various OpenAI models. This user-friendly platform features a powerful prompt editor that makes it simple to compare prompts across multiple AI models, ensuring the selection of the most suitable one based on quality, cost, and performance. The service, which is free to use during its beta release, provides valuable tools for users to develop and showcase their prompt portfolio with a minimalist design that emphasizes their creativity.

With ZenPrompts, users can experiment with prompts without the concern of losing previous work, share their inventions with a broader community, and adopt the DRY (Don't Repeat Yourself) method for efficient prompt creation using dynamic variables. Additionally, users have the option to document prompts with comments for better organization and preservation of developmental insights. The platform's goal is to enhance the capabilities of prompt engineers and to display their skills in the era of Large Language Models and AI.