PromptLayer is an AI engineering platform for prompt management, LLM evaluation, and agent observability. It moves prompts out of application code into a version-controlled registry shared by engineers and non-technical staff. Updating a prompt often means an engineer changes code and redeploys the application. PromptLayer removes that step, so marketers, lawyers, educators, and product managers […]
AI Infrastructure & Tooling
Evaluation frameworks, prompt management, observability, fine-tuning, RAG, vector
databases, MLOps
12 tools selected.
Portkey is an AI gateway and LLMOps platform for routing, monitoring, and securing LLM requests in production applications. Teams connect through an SDK, and the vendor says setup takes three lines of code. Requests then pass through the gateway, where rules such as PII redaction and load balancing apply. Palo Alto Networks completed its acquisition […]
Lakera is an AI security platform built for prompt injection protection and runtime guardrails in generative AI. It is a Check Point company. The platform covers AI applications, autonomous agents and employee use of external AI tools. Legacy data loss prevention tools and web gateways cannot interpret natural language prompts or intent. Lakera targets that […]
Modal is a serverless cloud infrastructure platform built by Modal Labs Inc. It lets developers run Python code in the cloud without managing servers, Dockerfiles, or Kubernetes clusters. It is aimed at data engineering, generative AI, and machine learning teams that need to run compute-heavy workloads without operating their own infrastructure. Deploying models or running […]
Patronus AI Platform is an automated evaluation, testing, and security tool for applications built on large language models. It is aimed at AI and ML engineers, enterprise security teams, and compliance officers who need to check model outputs for hallucinations, safety violations, and policy breaches before and after deployment. The company was founded in 2023 […]
Vellum is an AI assistant platform built around a persistent digital identity rather than a stateless chat window. Each assistant gets its own memory, filesystem, and email address, and it can run background tasks between conversations instead of waiting for a new prompt. The company originally launched in 2023 as an enterprise LLM development platform, […]
Braintrust is an observability, evaluation, and discovery platform built for AI agents and LLM applications. Instead of tracking uptime and latency like traditional monitoring tools, it focuses on how AI output quality changes over time. It targets engineering, product, and data teams that ship LLM-based features and need to know when a model drifts or […]
Weaviate is an open-source, AI-native vector database built by Weaviate B.V. It stores and searches unstructured data such as text, images, audio, and video using vector embeddings alongside structured attributes. Teams use it to build semantic search, retrieval-augmented generation, and stateful AI agent applications. The system converts unstructured objects into dense vector representations, either through […]
Qdrant is an open source vector similarity search engine and vector database written in Rust. It stores, manages, and queries high-dimensional vector embeddings produced by machine learning models. Teams building retrieval-augmented generation, semantic search, or recommendation systems use it as core infrastructure. The database pairs dense or sparse vector representations with JSON metadata payloads. It […]
Ragas is an open-source framework built by Exploding Gradients for evaluating and monitoring Retrieval Augmented Generation (RAG) pipelines. It gives developers automated metrics to assess retrieval quality and generation quality without relying solely on manual review. The tool addresses a common problem in RAG evaluation. When a chatbot gives a wrong answer, teams often cannot […]