AI agents: choose a framework for the job

Write the acceptance test first

Describe one input, the expected output and the actions the agent may take. For document triage, prepare a small set of fictional tickets with known destinations. Include an ambiguous ticket and one asking the agent to ignore its instructions. Decide when it should ask a person for help. Keep the examples unchanged while comparing projects so an attractive demo cannot quietly redefine success.

Find a manageable shortlist

Start with a capability such as workflow, retrieval or evaluation. Add the language filter if your team must maintain the integration in Python or TypeScript. Open a few repositories and locate their minimal example, dependency requirements and tracing documentation. Anthropic distinguishes fixed workflows from agents that choose their own steps; that distinction helps you decide whether you need model-directed tool use at all.

Run one task under a fixed budget

In a disposable test environment, compare the same inputs with the same model where possible. Record task success, tool calls, elapsed time and reported token use. Set a stopping condition before the run. Try a tool timeout, an invalid response and a request outside the allowed scope. Inspect whether a retry repeats a write or creates duplicate work. A framework that exposes these failures clearly may be easier to operate than one with more integrations.

Understand what the listing leaves out

This index searches selected repository metadata, not source code or runtime behavior. A category match does not establish that a project supports your model, deployment environment or security requirements. Stars and a recent push help with discovery but cannot replace a test. Check the original repository for current instructions, release changes and licence terms. Hosted services and model APIs can have separate charges even when framework code is publicly available.

Do I need multiple agents?

Begin with one bounded workflow and add another agent only when it solves a measured problem. For example, compare a separate review step against a simple validation function on the same saved examples. Extra coordination adds more behavior to inspect and more opportunities for a task to stall.

What should I save from an evaluation?

Keep the project revision, model identifier, configuration, test inputs and expected answers together. Record failures as well as successful outputs. This gives you a repeatable comparison after a dependency or prompt changes, instead of relying on a memorable conversation.