AI Adoption Playbook

• incident.io's Lawrence Jones on Building AI That Automatically Investigates Technical Outages

  When technical systems fail at companies like Netflix or Etsy, every minute of downtime can cost millions. That's why incident.io is building AI systems that can automatically investigate and diagnose technical problems faster than human engineers. In this episode of The AI Adoption Playbook, Lawrence Jones, Product Engineer at incident.io, tells Ravin how they're creating an automated incident investigator that can analyze logs, traces, and metrics to determine what went wrong during an outage. He shares their methodical approach to AI development, focusing on measurable progress through evaluation metrics and scorecards rather than intuitive "vibe-based" changes. Lawrence also discusses the evolution of their AI teams and roles, including their newly launched AI Engineer position designed specifically for the unique challenges of AI development, and how they use LLMs themselves to evaluate AI system performance. Topics discussed: Building an AI incident investigator that can automatically analyze logs, traces, and metrics to determine the root cause of technical outages. Creating comprehensive evaluation frameworks with scorecards and metrics to measure AI performance against historical incident data. Using LLMs as evaluators to determine if AI responses were helpful by analyzing post-incident conversations and user feedback. Developing internal tooling that enables teams to rapidly test and improve AI systems while maintaining quality standards. Evolving from individual "vibe-based" AI development to team-based systematic improvement with clear metrics for success. Structuring AI engineering roles and teams to balance product engineering skills with specialized AI development knowledge. Implementing product-focused AI features like chatbots that can help automate routine tasks during incident response. Leveraging parallel human and AI processes to collect validation data and improve AI system performance over time. Building versus buying AI evaluation tools and the advantages of custom solutions integrated with existing product data. Exploring the future of AI in technical operations and whether AI will enhance or replace human roles in incident management. Listen to more episodes:  Apple  Spotify  YouTube

  --------  

  40:33
• Shopify's Spencer Lawrence on Bridging AI Capability and Organizational Impact

  What happens when AI capabilities outpace organizational readiness? At Shopify, this tension has pushed them to develop a practical implementation approach that balances rapid experimentation with sustainable value creation.  Spencer Lawrence, Director of Data Science & Engineering, shares how they've evolved from simple text expansion experiments to sophisticated AI assistants like Help Center and Sidekick that are transforming both customer support and merchant operations. At the heart of their strategy is a barbell approach enabling self-service for small AI use cases while making targeted investments in transformative projects. Spencer also explains how their one-week sprint cycles, sophisticated evaluation frameworks, and cross-functional collaboration have helped them overcome the common challenges that prevent organizations from realizing AI's full potential. Successful AI implementation requires more than just technical solutions — it demands new organizational structures, evaluation methods, and a willingness to constantly reevaluate what knowledge work means in an AI-augmented world. Topics discussed: Shopify's evolution from early text expansion experiments to production-level AI assistants that support both customers and merchants. Creating sophisticated evaluation frameworks that combine human annotators with LLM judges to ensure quality and consistency of AI outputs. Implementing a barbell strategy that balances small self-service AI use cases with strategic investments in high-impact projects. Running one-week sprints across all AI work to maximize iteration cycles and maintain velocity even at enterprise scale. Addressing the gap between AI capabilities and real-world impact through both technological solutions and organizational change. Building feedback loops between technical teams and legal/compliance departments to create AI solutions that meet governance requirements. Fostering a culture that values experimentation while developing clear policies that give employees confidence to innovate responsibly. Exploring how AI will raise productivity expectations rather than simply reducing workloads across all roles and functions. Using AI as a strategic thought partner to generate novel ideas and help evaluate different perspectives on complex problems. Developing a forward-looking perspective on knowledge work that embraces AI augmentation while maintaining human judgment and oversight. Listen to more episodes:  Apple  Spotify  YouTube

  --------  

  45:35
• MongoDB's David Vainchenker on Shipping Fast and Learning from AI Usage Patterns

  Forget theoretical planning — MongoDB dove headfirst into AI adoption and let real-world usage guide their strategy. David Vainchenker, Sr. Director of Enterprise Initiatives & Tools at MongoDB, joins Ravin on this episode of The AI Adoption Playbook to share this practical approach and unpack their evolution from simple chatbots to sophisticated agent-based systems.  David shares their practical challenges with measuring AI's business impact, explaining why time savings metrics alone weren't convincing to leadership without translating to actual dollar savings or increased capacity. He also offers candid insights about security concerns, copyright issues with AI-generated code, and the delicate balance between innovation and governance. Topics discussed: Why shipping AI tools quickly and learning from actual usage patterns proved more effective than predicting theoretical use cases. The challenge of translating AI time savings into measurable business impact that resonates with leadership and affects the P&L. Security and compliance considerations when implementing AI at enterprise scale, including permission-aware retrieval requirements. Managing the balance between build vs. buy decisions in the fast-evolving AI landscape while ensuring business continuity. The reality of AI-assisted coding adoption rates varying significantly between junior and senior engineers in large organizations, and the copyright implications of having non-human-generated code. How MongoDB approaches vertical (specialized) vs. horizontal (platform) AI solutions for different use cases across the enterprise. The budgeting challenges created when every existing software vendor offers AI capabilities as premium add-ons. The importance of maintaining cross-system AI capabilities that match human workflows spanning multiple applications. Listen to more episodes:  Apple  Spotify  YouTube Website

  --------  

  41:03
• Lattice's Allen Jeter on Building Practical AI Assistants That Transform Enterprise Operations

  In this episode of The AI Adoption Playbook, Allen Jeter, Director of IT at Lattice, describes how his team transformed internal operations by strategically implementing AI assistants across multiple departments. Starting with a clear focus on reducing manual work and response times, Allen walks Ravin through how Lattice built their first AI solutions, from an experimental chatbot using Okta Workflows and Pinecone to production-grade systems serving their People Operations and security teams. What sets Lattice's approach apart is their pragmatic focus on solving real business problems rather than chasing AI for its own sake. By identifying specific pain points, implementing security guardrails from the beginning, and deploying AI directly within existing workflows like Slack, they've achieved impressive adoption across the organization. Allen also shares invaluable advice for IT leaders looking to implement AI, emphasizing early experimentation, stakeholder involvement, and the importance of understanding your business problems before attempting AI solutions. Topics discussed: Implementing AI assistants for People Operations that provide 24/7 support for employee questions about benefits and company policies. Building a security bot that helps sales teams respond to customer security questionnaires faster, reducing bottlenecks and accelerating sales cycles. Evaluating the crowded AI vendor landscape with specific requirements rather than getting caught up in marketing hype. The importance of integrating AI tools into existing workflows like Slack channels to maximize adoption without changing user behavior. Creating effective prompt engineering strategies to help teams customize AI responses and maintain accuracy across different domains. Implementing proper governance and permissions structures that respect existing data access controls to ensure compliance. Measuring success through concrete metrics like reduction in manual work hours and decreased time-to-answer across departments. Using AI to enrich support ticket metadata automatically, enabling better insights without manual categorization work. Balancing experimentation with security guardrails to enable innovation while protecting sensitive company and customer data. Resources Mentioned: Credal’s blog post, “The Enterprise Adoption Curve: Lessons Learned So Far”

  --------  

  49:55
• The Simple Framework That Got 100% Employee AI Adoption

  What's the secret to successful AI adoption? According to Robert Mitchell, Chief AI Officer at WSI, it's not just about choosing the right tools, but it's about mastering a delicate balance between executive vision and hands-on experimentation. After helping countless mid-sized businesses implement AI, Robert explains why these organizations need fundamentally different approaches than enterprises, focusing on quick wins and easy implementation over maximum capability. His conversation with Ravin on this episode of The AI Adoption Playbook explores WSI's unique dual-track implementation approach, combining executive planning with grassroots experimentation. Robert shares practical insights on building effective AI councils with representation across all business functions, ensuring that AI initiatives benefit from diverse perspectives and real-world operational knowledge. Robert also walks through WSI’s proven framework for balancing top-down strategy with bottom-up experimentation, why SMBs require different solutions than enterprises, and how to build truly cross-functional AI governance. Topics discussed: A proven "top-down, bottom-up" implementation framework that combines executive buy-in with identifying and empowering internal AI champions who can drive adoption through monthly AI Council meetings and team challenges Detailed ROI calculation methodology for AI initiatives, illustrated through a case study showing how 10% productivity gains on a $5M payroll can translate to $3M in additional business value at 6x EBITDA multiple Specific approach to AI governance using three core documents - policies for internal data usage, client data handling, and vendor data management - that must be established before any employee training begins Concrete example of high-ROI automation: a $20K investment to eliminate 7 days of manual accounting work monthly, improving employee satisfaction while enabling team to focus on higher-value activities Strategic methodology for creating "aha moments" by having employees first experiment with AI in their personal domain expertise before applying it to work processes, making adoption more intuitive Practical framework for quick wins: identifying 90-minute process improvements through Loom video analysis of employee pain points, then rapidly implementing targeted AI solutions

  --------  

  44:17

Plus de podcasts Technologies

Podcasts tendance de Technologies

À propos de AI Adoption Playbook