LaunchDarkly Config Online Evaluations Skill¶
An Agent Skill for attaching judges to config variations for automatic LLM-as-a-judge evaluation.
Overview¶
This skill teaches agents how to: - Create custom judge configs with evaluation criteria - Attach judges to config variations via API - Configure sampling rates for cost control - Monitor evaluation results in the dashboard
Installation (Local)¶
Copy skills/agentcontrol/online-evals/ into your agent client's skills path.
Prerequisites¶
- The LaunchDarkly MCP server, connected as a member who can change AgentControl configs
- Existing config with variations (use
launchdarkly-configs-createskill) - For custom judges: understanding of LLM-as-a-judge methodology
Usage¶
Structure¶
Related¶
- config Create - Create configs first
- config Targeting - Enable targeting on judges
- config Variations - Manage variations
- Online Evaluations Docs
License¶
Apache-2.0