The core problem: you can't manually QA an AI agent. When you ship a new prompt, swap a model, or add a tool, how do you know the agent still behaves correctly across the thousands of ways users might interact with it?\nMost teams resort to manual spot-checking (doesn't scale), waiting for users to complain (too late), or brittle scripted tests.
Our answer is simulation: synthetic users interact with your agent the way real users do, and LLM-based judges evaluate whether it responded correctly - across the full conversational arc, not just single turns.\nThree things make this actually work:\nScenario generation + real conversation import - Our scenario generation agent bootstraps your test suite from a description of your agent. But real users find paths no generator anticipates, so we also ingest your production conversations and automatically extract test cases from them. Your coverage evolves as your users do.
Mock tool platform - Agents call tools. Running simulations against real APIs is slow and flaky. Our mock tool platform lets you define tool schemas, behavior, and return values so simulations exercise tool selection and decision-making without touching production systems.
Deterministic, structured test cases - LLMs are stochastic. A CI test that passes "most of the time" is useless. Rather than free-form prompts, our evaluators are defined as structured conditional action trees: explicit conditions that trigger specific responses, with support for fixed messages when word-for-word precision matters. This means the synthetic user behaves consistently across runs - same branching logic, same inputs - so a failure is a real regression, not noise.
Cekura also monitors your live agent traffic. The obvious alternative here is a tracing platform like Langfuse or LangSmith - and they're great tools for debugging individual LLM calls. But conversational agents have a different failure mode: the bug isn't in any single turn, it's in how turns relate to each other. Take a verification flow that requires name, date of birth, and phone number before proceeding - if the agent skips asking for DOB and moves on anyway, every individual turn looks fine in isolation. The failure only becomes visible when you evaluate the full session as a unit. Cekura is built around this from the ground up.\nWhere tracing platforms evaluate turn by turn, Cekura evaluates the full session. Imagine a banking agent where the user fails verification in step 1, but the agent hallucinates and proceeds anyway. A turn-based evaluator sees step 3 (address confirmation) and marks it green - the right question was asked. Cekura's judge sees the full transcript and flags the session as failed because verification never succeeded.
Try us out at https://www.cekura.ai - 7-day free trial, no credit card required. Paid plans from $30/month.
We also put together a product video if you'd like to see it in action: https://www.youtube.com/watch?v=n8FFKv1-nMw. The first minute dives into quick onboarding - and if you want to jump straight to the results, skip to 8:40.
Curious what the HN community is doing - how are you testing behavioral regressions in your agents? What failure modes have hurt you most? Happy to dig in below!","offTopic":true},{"id":"abee0091-8cec-476b-98eb-196be1d81be7","excerpt":"Show HN: Opencom, full Intercom alternative, open-source, Convex Backend — Hi HN,
I’ve been building Opencom, an open-source customer messaging platform - basically an Intercom alternative you can run yourself with minimal setup or infrastructure to manage. Repo + demo here:
I’ve been building Opencom, an open-source customer messaging platform - basically an Intercom alternative you can run yourself with minimal setup or infrastructure to manage. Repo + demo here: https://github.com/opencom-org/opencom I started working on it after years of frustration using Intercom. I love the selection of tools they offer, but not their complex and expensive pricing model or their frequent layout and UX changes. I wanted an open source alternative that was simple and cheap to run, making it accessible for projects of any size. \nI wanted something that was as close to Intercom's feature-set as possible, and then some. You can self host as much or as little as you like, from everything to nothing, using just the hosted frontends and backend. What it includes - Real-time chat via an embeddable widget, connected to a shared inbox\n- Tickets, campaigns/series, surveys (NPS/CSAT), segments, reports\n- Product tours/tooltips/checklists\n- Knowledge base + an AI agent with human handoff\n- Email channel via Resend integration\n- React Native SDK - with native iOS and Android SDKs planned Hosting / deployment model Opencom is set up so you can keep the architecture fairly simple:\nBackend: everything runs on Convex (the repo’s deploy guide treats Convex as the canonical backend), and email features use Resend (e.g., RESEND_API_KEY, webhook verification, etc.). \nFrontend: the web dashboard/widget/landing are standard Next.js/Vite apps, so you can host them on Vercel if you like (Convex has first-class docs/integration for deploying alongside Vercel). Mix-and-match: there’s an explicit “hosted apps + your own Convex backend” profile, as well as “self-host web + your own backend” and “full self-host”. In other words, you can either host everything yourself, or use the hosted web + mobile apps while pointing them at your Convex backend. React Native SDK If you want Intercom-style in-app messaging inside your own mobile app, Opencom includes a React Native SDK package in the monorepo, alongside native iOS/Android SDKs planned. About the hosted offering At the moment, the hosted setup is intended as a demo (so people can kick the tyres without doing a full deploy). If there’s enough demand for a reasonably priced, direct Intercom alternative, I could shift it into a proper hosted service. Feedback I’d really appreciate - Does the Convex-backend + Vercel-frontend split feel sensible for teams who’d otherwise buy Intercom?\n- Anything missing that’s a hard requirement for switching?\n- Thoughts on licensing / pricing expectations if this became a straightforward hosted option (no enterprise sales theatrics)?\n- Any other thoughts - this is my first open sourced project, and it is built heavily with AI, so I expect there to be issues / silly mistakes, and I know there are areas that need work. My hope is that others want a product like this and would be open to chipping in / fleshing out features they want or need. Thanks for checking it out!","offTopic":true},{"id":"ec6bd13f-2ee0-4117-84c8-9d22a5771b70","excerpt":"Cloudflare, AWS, Firebase and Supabase all shipped their own \"Ask AI\" button. I built one any SaaS can drop in. — **I'm building the future of modern SaaS UX** \n\nIf you've used those dashboards recently you've seen it. A little Ask AI in the corner. Each of them took a dedicated team and months of work to build, which ","url":"https://www.reddit.com/r/SaaS/comments/1vttio5/cloudflare_aws_firebase_and_supabase_all_shipped/","role":"pain","weight":0.6749731,"occurredAt":"2026-08-20T19:32:02.000Z","sourceKey":"reddit","sourceName":"Reddit","credibility":0.62,"venue":"SaaS","intent":"other","painScore":0.4,"sentiment":-1,"confidence":0.4821236,"matchedPatterns":[],"statement":"Cloudflare, AWS, Firebase and Supabase all shipped their own \"Ask AI\" button.","title":"Cloudflare, AWS, Firebase and Supabase all shipped their own \"Ask AI\" button. I built one any SaaS can drop in.","body":"**I'm building the future of modern SaaS UX** \n\nIf you've used those dashboards recently you've seen it. A little Ask AI in the corner. Each of them took a dedicated team and months of work to build, which is fine when you're Cloudflare and impossible when you're four people.\n\n**So I built the version any SaaS can add to their own product.**\n\nThe important part, and the reason I didn't just build another chatbot: it doesn't read your docs and point at them. It reads the real data in your product and takes the real action, scoped to whoever is logged in. It can't see or do anything that user couldn't already do themselves.\n\n\"Give Priya the same access as Jamie, minus production.\" That's ten clicks across three screens, or a support ticket. It's one sentence now, and the assistant checks the existing state, tells you what it's about to do, and waits for you to confirm before anything changes.\n\nThings I'll say up front because you'd find out anyway:\n\n* Read-only questions work in about five minutes. Actions that write take longer, because those run against your own backend and you have to wire that up.\n* It can't see or generate images yet. It tells you the manual steps instead of pretending.\n* Everything starts in a sandbox where nothing can touch real data.\n* Every action is logged, with whether it actually worked.\n\nBeta is out and it's free. **Reply here or DM me and I'll send the link.**\n\n**This is built to work with any tech stack and can be set up in under 5 min**\n\nSame guy built TradePilot, a trading copilot with 170+ tools in production, so agents people rely on for real decisions isn't new ground for me + you don't need to worry about tool selection memory layer or write a complete new infra yet make your SaaS has a 24/7 actual AI agent helping users.\n\nGenuinely want to hear where you think this breaks.","offTopic":true},{"id":"1c1f61a3-be40-4aff-a76c-3fd0097a4951","excerpt":"Show HN: Syntro.io — Hey HN, I’m Mauricio, cofounder of DailyBot (YC S21). TLDR: Some time ago we started providing Slack connect support to our own customers and we ended up building a simple Slack app for this. The goal was to have an AI embedded in the Slack connect channels to help us answer questions and solve s","url":"https://news.ycombinator.com/item?id=45129043","role":"pain","weight":0.94836414,"occurredAt":"2025-09-04T16:28:54.000Z","sourceKey":"hackernews","sourceName":"Hacker News","credibility":0.7,"venue":"news","intent":"feature_request","painScore":0.46526316,"sentiment":-0.2631579,"confidence":0.6472313,"matchedPatterns":["missing_feature"],"statement":"Show HN: Syntro.io.","title":"Show HN: Syntro.io","body":"Hey HN, I’m Mauricio, cofounder of DailyBot (YC S21). TLDR: Some time ago we started providing Slack connect support to our own customers and we ended up building a simple Slack app for this. The goal was to have an AI embedded in the Slack connect channels to help us answer questions and solve support tickets. We then built it as a separate product namedSyntro (https://syntro.io). The background: we and many of our customers noticed the same pattern of support requests increasingly coming in through Slack Connect channels. It’s faster and more natural for customers than opening a Zendesk or Email ticket. But it creates visible problems such as: Having no structure - so there are messages scattered across threads and DMs.\nManual overhead - like copying into another ticketing/product feedback system. We wanted to turn Slack itself into a support platform, and we found some alternatives in the market but they were a bit expensive and over complicated - so, knowing that at DailyBot we’re quite expert on Slack integrations - why not build it out? Syntro does five things very well: * Detects new requests in Slack by turning threads into tickets automatically.\n* Maintains a board (kanban/list) for prioritization and tracking via web app.\n* Suggests replies from your knowledge base or past tickets (you edit before sending).\n* Keeps Slack and the ticket board in sync, so agents can work in either.\n* Broadcast feature for announcements to multiple Slack Connect channels. Technically, the tricky parts were: 1. Thread detection & context sync: Slack messages aren’t structured like tickets. We had to build logic to consolidate messages into coherent “requests.”\n2. Two-way sync: Ensuring replies in Slack update the ticket system (and vice versa) without duplication.\n3. Knowledge base ingestion: We support file uploads + URLs, then use embeddings for AI reply suggestions.\n4. Slack Connect quirks. Permissions and visibility differ across shared channels; making the app feel native took time. Current stack: Django backend, Next.js frontend, Postgres (with pgvector) for embeddings, Redis for state, LangChain + OpenAI/Claude for LLMs, and a Python bot for Slack integration. Limitations / things we’re still working on:\n- Better SLA tracking and reminders.\n- More granular analytics for managers.\n- Expanding to email perhaps. We may open source it in the future - we have to clean up code first and decouple it from some DailyBot underlying infrastructure. Product + demo here: https://syntro.io (completely free for launch) We’d love your feedback - especially from folks who’ve hacked together their own Slack support workflows. What’s missing? What’s overkill? What do you wish someone had built instead? Thanks HN,\nMauricio","offTopic":true},{"id":"2c0b4739-ea74-42e6-b4fb-009910cb3cac","excerpt":" — Hi HN,\nI’ve been building Opencom, an open-source customer messaging platform - basically an Intercom alternative you can run yourself with minimal setup or infrastructure to manage. Repo + demo here: https://github.com&#","url":"https://news.ycombinator.com/item?id=47194407","role":"request","weight":0.91618884,"occurredAt":"2026-02-28T12:19:42.000Z","sourceKey":"hackernews","sourceName":"Hacker News","credibility":0.7,"venue":"news","intent":"feature_request","painScore":0.27543697,"sentiment":0.125,"confidence":0.7183333,"matchedPatterns":["free_tier","missing_feature"],"statement":"- Anything missing that’s a hard requirement for switching?","title":null,"body":"Hi HN,\nI’ve been building Opencom, an open-source customer messaging platform - basically an Intercom alternative you can run yourself with minimal setup or infrastructure to manage. Repo + demo here: https://github.com/opencom-org/opencom I started working on it after years of frustration using Intercom. I love the selection of tools they offer, but not their complex and expensive pricing model or their frequent layout and UX changes. I wanted an open source alternative that was simple and cheap to run, making it accessible for projects of any size. I wanted something that was as close to Intercom's feature-set as possible, and then some. You can self host as much or as little as you like, from everything to nothing, using just the hosted frontends and backend. What it includes - Real-time chat via an embeddable widget, connected to a shared inbox - Tickets, campaigns/series, surveys (NPS/CSAT), segments, reports - Product tours/tooltips/checklists - Knowledge base + an AI agent with human handoff - Email channel via Resend integration - React Native SDK - with native iOS and Android SDKs planned Hosting / deployment model Opencom is set up so you can keep the architecture fairly simple: Backend: everything runs on Convex (the repo’s deploy guide treats Convex as the canonical backend), and email features use Resend (e.g., RESEND_API_KEY, webhook verification, etc.). Frontend: the web dashboard/widget/landing are standard Next.js/Vite apps, so you can host them on Vercel if you like (Convex has first-class docs/integration for deploying alongside Vercel). Mix-and-match: there’s an explicit “hosted apps + your own Convex backend” profile, as well as “self-host web + your own backend” and “full self-host”. In other words, you can either host everything yourself, or use the hosted web + mobile apps while pointing them at your Convex backend. React Native SDK If you want Intercom-style in-app messaging inside your own mobile app, Opencom includes a React Native SDK package in the monorepo, alongside native iOS/Android SDKs planned. About the hosted offering At the moment, the hosted setup is intended as a demo (so people can kick the tyres without doing a full deploy). If there’s enough demand for a reasonably priced, direct Intercom alternative, I could shift it into a proper hosted service. Feedback I’d really appreciate - Does the Convex-backend + Vercel-frontend split feel sensible for teams who’d otherwise buy Intercom? - Anything missing that’s a hard requirement for switching? - Thoughts on licensing / pricing expectations if this became a straightforward hosted option (no enterprise sales theatrics)? - Any other thoughts - this is my first open sourced project, and it is built heavily with AI, so I expect there to be issues / silly mistakes, and I know there are areas that need work. My hope is that others want a product like this and would be open to chipping in / fleshing out features they want or need. Thanks for checking it out!","offTopic":true},{"id":"71ade4f0-b56b-4087-9983-b72ed16df7fc","excerpt":" — Hi HN, I just launched a new SaaS I've been building, and I'd be incredibly grateful for your feedback. It's called AgentiveAIQ, and it's a no-code platform for building AI chatbots. I know there are a lot of "build a RAG bot" tools out there, but I was frustrated with a few things:","url":"https://news.ycombinator.com/item?id=45795140","role":"pain","weight":0.85716665,"occurredAt":"2025-11-03T01:51:12.000Z","sourceKey":"hackernews","sourceName":"Hacker News","credibility":0.7,"venue":"news","intent":"problem_report","painScore":0.48,"sentiment":0.375,"confidence":0.57916665,"matchedPatterns":["frustrating"],"statement":"I know there are a lot of build a RAG bot tools out there, but I was frustrated with a few things: Most are a black box.","title":null,"body":"Hi HN, I just launched a new SaaS I've been building, and I'd be incredibly grateful for your feedback. It's called AgentiveAIQ, and it's a no-code platform for building AI chatbots. I know there are a lot of "build a RAG bot" tools out there, but I was frustrated with a few things: Most are a "black box." You have no idea what users are really asking or if the bot is even helping, short of reading every single log. Customizing the look and feel to match a brand is often a huge pain or impossible. They're often just "talk" and can't do anything (like capture a lead or check inventory). So, I built our platform around three core ideas: 1. A WYSIWYG Editor: You can visually build your chat widget—change every color, logo, font, and border radius to make it look 100% native to your site. No CSS overrides needed. 2. A "Dual-Agent" System: This is the part I'm most excited about. The Main Agent: This is the chatbot your visitor talks to. It uses a RAG + Knowledge Graph backend to answer questions accurately. The Assistant Agent: This is a second AI that works in the background. After a conversation ends, it reads the entire transcript, performs sentiment analysis, identifies the user's goal (e.g., Sales, Support), and emails you (the site owner) a structured summary with actionable insights. You get the signal without all the noise. 3. "Agentic" Tools: The bot can actually execute tasks. It has pre-built flows to trigger webhooks, send lead info to a CRM, or query live data from Shopify/WooCommerce (like product info or order status). Under the hood, it uses a RAG + Knowledge Graph for its brain and has a fact-validation layer to try and minimize hallucinations by cross-referencing answers with the source data. I'm here for honest feedback. Is the "dual agent" concept genuinely useful, or just a gimmick? Is the WYSIWYG editor intuitive? We have a 14-day free trial on the Pro plan (no credit card required), and I'd love to hear what you think. You can check it out here: agentiveaiq.com\nThanks!","offTopic":false}],"breakdown":[{"sourceKey":"hackernews","sourceName":"Hacker News","count":5},{"sourceKey":"reddit","sourceName":"Reddit","count":1}],"total":6}}