Amsterdam, a long-established European technology centre, hosted the Chatbot Summit 2025 for the first time this year. Certus attended and after speaking with several of the 250 delegates, it was clear why the city earned that privilege.
The event really highlighted that conversational and agentic AI are no longer experiments within this community. They are shaping business operations now, and have been for some time in certain industries. With that being said, the consistent insight from both product leaders and practitioners was that value still depends on people. Human oversight, governance, and change management remain essential to innovation.
Key Insights for Business and IT Leadership
1. Agentic AI is moving to quickly into delivery in lots of organisations
Roderick Soesman, Chief Executive of Comfort Partners (a housing maintenance company in the Netherlands) presented a compelling example of applied AI. His company created a voice and chat assistant named Lotte, managing appointment scheduling and customer calls through full integration with back-office systems. Live since September 2025, it already handles thousands of requests daily.
For leadership teams, our takeaway was clear: scaling AI requires mature data, strong integration, and disciplined organisational change.
2. Conversational architecture is changing
Rutger de Ruiter, Head of Product at CM.com, offered several forward-looking observations:
Large-language-model inaccuracies (better described as confabulations) will diminish as inference models stabilise.
The notion of “building” agents ( with low/no-code at present) will give way to prompt-based building, where AI generates, develops and manages the multiple components/integrations.
Conversational front-ends will overtake traditional web or application interfaces.
Customer behaviour is evolving: algorithmic agents will act as rational buyers, selecting products by suitability, availability, and timing rather than brand identity.
In 2026, AI agents will outperform humans on several knowledge-based tasks, applying reasoning across multiple contexts before responding.
3. Guardrails and human judgement remain vital, and will for some time to come
Even the most advanced implementations acknowledged one truth: reliability depends on people guiding the system.
Effective governance requires:
Fine-tuned LLMs with explicit system instructions that define behaviour, and are regularly monitored and re-written/tweaked by humans. Every word in a system prompt leads to a variation in the LLM output.
Dedicated guardrail LLMs operate a level of conformance to observe, assess, and escalate outputs from another LLM when required (which is basically required for every message that reaches your customer via a chatbot’s response).
Continuous monitoring and observability of LLM performance, with defined metrics to support the upholding of your company’s legal, ethical, and quality standards.
These measures protect your organisation’s reputation, ensure regulatory compliance, and maintain user trust.
4. Data science and effective change management are critical success factors
Workshops led by ING’s customer-support chatbot team revealed their level of precision behind successful AI operations. Their scientific approach to data analysis, re-skilling of service desk resources, and performance monitoring was as impressive as the technology itself.
Some recent study undertaken at Certus with Stanford Graduate Business School, in Digital Transformation and AI also emphasised the same point. Technical strength alone cannot deliver value without structured organisational change, measurement of outcomes, and clearly defined roles for both humans and machines.
Where chatbots are built into ITSM tooling at present, some of these operations and insights are visible and managed by the vendor on your behalf, but the conference highlighted just how limited in visibility and oversight these solutions still are at present, with a lack of configurability thats restricted to the ITSM tooling vendor.
Implications for Certus Services Clients
The themes that emerged from our trip to Amsterdam carry clear implications for senior leaders:
Align strategy with business outcomes. AI programmes should always begin with defined goals, be them customer experience, efficiency, or cost reduction – before any technology is deployed.
Maintain human involvement. Determine which decisions require oversight, how escalation works, and how employees’ expertise evolves alongside automation.
Establish an AI Enablement Framework. Governance, risk control, and accountability must underpin every implementation.
Invest in reliable data foundations. Clean, connected data sources and transparent metrics will determine success.
Adapt operating models. As AI tools assume repetitive tasks, staff roles should shift toward analysis, training, and assurance to sustain confidence and performance.
These principles sit at the heart of Certus Services’ advisory approach: enabling clients to integrate new capabilities responsibly, while preserving ‘digital trust’ and measurable value.
Next Steps for Leadership
Forward-thinking organisations should act now:
Assess readiness. Evaluate data quality, integration maturity, and governance mechanisms.
Define measurable outcomes. Identify what improvement looks like, the real reason why you want to achieve it and if it really aligns to your company’s wider strategy, and how it will be evidenced.
Design the enablement layer. Outline the roles, escalation routes, new operational activities, and compliance responsibilities.
Select a contained pilot. Choose a service interaction where improvement can be quantified easily as this will support with more significant investment planning once the initial pilot is seen and understood by leadership.
Support people through change. Communicate purpose, skill development, and future pathways for affected teams.
Certus Services supports clients through each of these stages, ensuring that AI adoption strengthens operational resilience, without disruption.
Closing
Chatbot Summit 2025 demonstrated that progress in conversational and agentic AI is accelerating quickly, yet sustainable success still relies on leadership discipline, trustworthy governance, and thoughtful people management. Organisations that balance these factors will convert the emerging potential into their direct advantage.
References:
Chatbot Summit 2025 Agenda, Amsterdam (chatbotsummit.com)
CM.com Insights Blog, Agents of Change (cm.com)
Comfort Partners Case Study, Voice AI Assistant Lotte (chatbotsummit.com)