Skip to content

LaunchDarkly Config Online Evaluations Skill

View on GitHub

Chat tasks

An Agent Skill for attaching judges to config variations for automatic LLM-as-a-judge evaluation.

Overview

This skill teaches agents how to: - Create custom judge configs with evaluation criteria - Attach judges to config variations via API - Configure sampling rates for cost control - Monitor evaluation results in the dashboard

Installation (Local)

Copy skills/agentcontrol/online-evals/ into your agent client's skills path.

Prerequisites

  • The LaunchDarkly MCP server, connected as a member who can change AgentControl configs
  • Existing config with variations (use launchdarkly-configs-create skill)
  • For custom judges: understanding of LLM-as-a-judge methodology

Usage

Attach security and API contract judges to the model-selector config at 100% sampling
Create a custom judge that checks for scope creep in code changes

Structure

launchdarkly-online-evals/
├── SKILL.md
└── README.md

License

Apache-2.0