AI tools for ai infrastructure & developer apis

Groq provides inference APIs for supported text and speech models.

Best for: Fast hosted inference for supported models

An open-source Python framework for retrieval pipelines, document indexing, and tool-using AI agents.

Best for: Haystack suits developers assembling a document-grounded application from explicit components.

Voice-AI evaluation tools for test suites, audio analysis, and diagnosing failures in conversational voice systems.

Best for: Hume AI's current focus is evaluation infrastructure for voice systems.

Jan is an open-source desktop assistant that can run downloaded models locally or connect to online providers.

Best for: Offline desktop conversations with local models

Visual Python framework for building, testing, and serving AI agents and retrieval workflows with reusable components.

Best for: Python developers building visual agent and retrieval workflows

Langfuse records LLM traces, manages prompt versions and supports evaluations.

Best for: Open-source LLM observability and prompt versions

LangSmith helps developers inspect LLM runs and evaluate application behaviour.

Best for: Tracing and evaluating LLM applications

Build stateful AI agents that retain memory across sessions, with a web app, CLI workflow, and developer API.

Best for: Letta suits builders who need an agent to maintain context beyond one conversation.

Document-AI tools for parsing, structured extraction, indexing, and building source-based agent workflows.

Best for: LlamaIndex suits developers turning documents into usable context or structured records.

LM Studio runs downloaded models on your desktop and offers the Bionic agent for work with local files.

Best for: Desktop local AI and model experiments

A TypeScript framework for agents, tools, workflows, memory, and evaluation, with a hosted platform option.

Best for: Mastra suits TypeScript developers building a repeatable agent application.

Voice AI platform for voiceovers, text-to-speech APIs, and conversational agents for business workflows.

Best for: Business voiceovers and separately priced speech or agent services

Ollama provides a local runtime for supported language models and an API for applications.

Best for: Local model APIs and developer prototypes

OpenRouter routes API requests to a model and provider from a shared catalog.

Best for: Comparing model APIs with provider routing

Pipedream Connect supplies app connectors, managed authentication and tools for developer-built agents.

Best for: App connectors and auth for agent developers

A Python AI SDK for typed agent outputs, model connections, tool calls, dependency injection, and evaluation workflows.

Best for: Pydantic AI suits Python developers building agents inside an application with explicit input and output contracts.

RAGFlow is an open-source retrieval-augmented generation engine combining document parsing, retrieval and agent workflows.

Best for: Document retrieval pipelines with source references

Replicate hosts models with versioned inputs and APIs for generating or processing media and text.

Best for: Running hosted AI models through an API