Prompt management and evaluation
PromptHub
PromptHub provides a collaborative platform for developing, testing and deploying AI prompts. Individuals and teams can share public prompt libraries or work privately, compare model outputs side by side, run evaluations against test cases and chain prompts without writing code. Git-based versioning organizes changes, while APIs, shareable forms and a Zapier integration connect prompts to applications and workflows.
Founded by Dan Cleary and Dalton Pierce, PromptHub applies software development practices to prompt engineering. Its Pipelines feature runs evaluations automatically on commits, merge requests and API calls, with datasets for repeatable benchmarks and configurable pass/fail rules. Branch-based deployment lets applications retrieve or run a selected branch’s latest prompt version. Cleary subsequently announced that he was joining Anthropic’s Applied AI team as a Member of Technical Staff.
PromptHub’s business combines free public collaboration with paid plans for private prompts, team permissions and enterprise requirements. Users bring their own model-provider API keys and pay token costs separately. A 2024 customer case study describes ChainDefender using PromptHub for its AI smart contract auditor; PromptHub reported that the team tested hundreds of prompts and variants and implemented its API during the first week.
2 talks
Newest first
1 speaker at AIE
Affiliations reflect their AIE appearances, not necessarily current employment.
Start here
- Prompt Engineering Tactics
Start here for concrete prompt modifications, including an according-to clause that names a source and emotional stimuli appended to a request.
Dan ClearyAI Engineer Summit 2023
- The Model Isn’t Wrong—You’re Just Bad at Prompting
Continue here for examples involving DeepSeek-R1, OpenAI o1, and MedPrompt that illustrate why prompting choices depend on the model.
Dan ClearyAI Engineer Summit 2025
Messages from the stage
Grounding and collaboration as prompt tactics
The 2023 talk presents explicit source references and multi-persona collaboration as responses to unreliable outputs. It also introduces EmotionPrompt and connects emotional stimuli in prompts to reported benchmark improvements.
More prompt content can hurt
The 2025 talk discusses why extended reasoning may help reasoning models while unnecessary few-shot examples or excessive context can reduce performance. It places these tradeoffs alongside automatically generated reasoning chains and provider-specific meta-prompting.
Affiliations reflect each recorded session, not necessarily current employment.