langsmith-evaluator
langchain-ai/langsmith-skills · skills.sh
Open source Repository Open in the app JSON README (API)
About
Skill publicada por langchain-ai/langsmith-skills no skills.sh. Instale com: npx skills add langchain-ai/langsmith-skills@langsmith-evaluator
Details
- Kind
- Agent skills
- Topic
- No topic detected
- Publisher
- langchain-ai
- Origin
- skillssh
- Category
- ferramentas
- Stars
- 153
- Forks
- 16
- Open pull requests
- 2
- Last push
- 2026-08-17T22:09:21Z
- Repository state
- ativo
- Language
- Shell
- Added
- 2026-08-30 15:21:39
- Updated
- 2026-09-08 15:03:24
- Origin id
langchain-ai/langsmith-skills/langsmith-evaluator
README
# LangSmith Skills > **⚠️** — This project is in early development. APIs and skill content may change. Agent skills for observing and evaluating LLM applications with [LangSmith](https://smith.langchain.com/). Query traces, build evaluation datasets, and create custom evaluators — all from your coding agent. > Looking for skills to **build and improve** agents with LangChain, LangGraph, or Deep Agents? See [langchain-skills](https://github.com/langchain-ai/langchain-skills). ## Supported Coding Agents These skills can be installed for any agent supported by [`skills.sh`](https://skills.sh), including Claude Code, [Deep Agents CLI](https://docs.langchain.com/oss/python/deepagents/cli/overview), Cursor, Windsurf, Goose, and [many more](https://github.com/vercel-labs/skills). ## Prerequisites - A [LangSmith API key](https://smith.langchain.com/) ## Installation ### Quick Install Using [`npx skills`](https://github.com/vercel-labs/skills): **Local** (current project): ```bash npx skills add langchain-ai/langsmith-skills --skill '*' --yes ``` **Global** (all projects): ```bash npx skills add langchain-ai/langsmith-skills --skill '*' --yes --global ``` To link skills to a specific agent (e.g. Claude Code): ```bash npx skills add langchain-ai/langsmith-skills --agent claude-code --skill '*' --yes --global ``` --- ### Claude Code Plugin Install directly as a [Claude Code plugin](https://code.claude.com/docs/en/plugins): ```bash /plugin marketplace add langchain-ai/langsmith-skills /plugin install langsmith-skills@langsmith-skills ``` --- ### Install Script (Claude Code & Deep Agents CLI only) Alternatively, clone the repo and use the install script: ```bash # Install for Claude Code in current directory (default) ./install.sh # Install for Claude Code in a specific project directory ./install.sh ~/my-project # Install for Claude Code globally ./install.sh --global # Install for DeepAgents CLI in a specific project directory ./install.sh --deepagents ~/my-project # Install for DeepAgents CLI globally (includes agent persona) ./install.sh --deepagents --global ``` | Flag / Argument | Description | |------|-------------| | `DIRECTORY` | Target project directory (default: current directory, ignored with `--global`) | | `--claude` | Install for Claude Code (default) | | `--deepagents` | Install for DeepAgents CLI | | `--global`, `-g` | Install globally instead of current directory | | `--force`, `-f` | Overwrite skills with same names as this package | | `--yes`, `-y` | Skip confirmation prompts | ## Usage After installation, set your API keys: ```bash export LANGSMITH_API_KEY=<your-key> export OPENAI_API_KEY=<your-key> # For OpenAI models export ANTHROPIC_API_KEY=<your-key> # For Anthropic models ``` Then run your coding agent from the directory where you installed (for local installs) or from anywhere (for global installs). ## Available Skills (3) ### LangSmith - **langsmith-trace** - Query and export traces (includes helper scripts) - **langsmith-dataset** - Generate evaluation datasets from traces (includes helper scripts) - **langsmith-evaluator** - Create custom evaluators (includes helper scripts) **Note:** All skills include Python and TypeScript helper scripts for common operations. ## Development Agent configuration lives in `config/`. To update an existing installation: ```bash ./install.sh --force ```