Overview
Fulcrum is an AI-powered advisory platform built on Mastra that helps users navigate high-stakes decisions through structured reasoning and a curated library of 734 mental models. It offers two modes - Decision Mode for fast, actionable recommendations and Strategy Mode for comprehensive multi-perspective analysis - all personalized to the user's role, industry, and context.
The Challenge
Generic AI chatbots give generic advice. High-stakes decisions - career moves, business strategy, risk assessment - require structured thinking grounded in proven frameworks, not just fluent text generation. The system needed to retrieve relevant mental models via semantic search, weave them into natural advice, adapt depth to question complexity, and guard against off-topic or harmful inputs - all while streaming responses in real time.
Approach
Built a Next.js 16 app with Mastra agents backed by Google Gemini models. Knowledge objects are embedded with pgvector and retrieved via weighted cosine similarity (top-6, min-score 0.5). An LLM-based input guardrail classifies intent before routing to the advisor or meta-responder agent. The advisor's system prompt dynamically adjusts based on mode and user profile. A custom evaluation framework with 23 golden test cases and 11 scorers validates retrieval quality, formatting compliance, personalization, and mode transitions.
Outcome
A production multi-user SaaS with Clerk authentication, per-user conversation threads, feedback collection, and a knowledge preference system. The evaluation suite covers retrieval NDCG, precision/recall, citation accuracy, and mode-specific compliance. Observability is handled through Langfuse tracing with per-scorer sampling strategies.