OpenHermes-13B vs mshumer/gpt-prompt-engineer - GitHub

OpenHermes-13B is a fine-tuned language model built on a fully open-source dataset of 242,000 entries, primarily generated by GPT-4 and curated from various open AI datasets. It was developed to improve text generation by removing AI disclaimers and refusals, aiming for clearer and more direct communication outputs. The model incorporates data from multiple contributors, including Teknium, WizardLM Team, Microsoft, and others, ensuring a diverse and comprehensive training base.

This model is designed for developers and researchers who want an open-source alternative for advanced text generation tasks without the typical AI safety disclaimers. It supports integration with popular machine learning libraries like Transformers and can be deployed locally or via Docker, making it accessible for experimentation and production use.

OpenHermes-13B offers transparency in its training process, with public access to its WANDB project logs and detailed benchmark results. It shows competitive performance on benchmarks like GPT4All and BigBench, with slight improvements over similar models, although it has some trade-offs in AGI-Eval scores.

Technically, it was trained using multi-GPU setups with Adam optimizer and cosine learning rate scheduling, emphasizing reproducibility and open collaboration. The model is compatible with various inference tools and quantizations, allowing use in environments like llama.cpp and LM Studio.

Overall, OpenHermes-13B provides a valuable resource for those seeking a powerful, open-source language model fine-tuned for instruction following and text generation without restrictive AI disclaimers, suitable for research, development, and deployment in diverse AI applications.

mshumer/gpt-prompt-engineer is an open source prompt engineering toolkit that generates, tests, and ranks candidate prompts for a task you define. You describe the use case, supply test cases, and the notebooks create multiple prompt variants, run them against every test case, and sort results with an ELO rating system starting at 1200. It ships as Jupyter notebooks you can run in Google Colab or locally.

Most prompt tools help you write one prompt at a time. gpt-prompt-engineer treats prompt selection like a tournament: dozens of candidates compete on your test cases, and the highest ELO scores surface the winners. Separate notebooks cover classification tasks, Claude 3 Opus with auto-generated test cases, and Opus-to-Haiku conversion for cheaper inference. Optional Weights & Biases and Portkey logging trace each run.

ML engineers, prompt engineers, and AI developers use it when they need reproducible prompt tuning instead of manual trial and error. The repo has 9.7k GitHub stars and runs on your own OpenAI or Anthropic API keys. It is free under the MIT license.