The Right Tool for the Job

When to Use This Activity

Run this activity after participants understand the basics of LLMs and have explored their capabilities. It works well as a follow-up to the "Inside the Black Box of LLMs" activity, helping participants transition from understanding what AI is to how it can be enhanced with tools.

Key Learning Outcomes

• Understand how tools transform chatbots into AI agents
• Identify specific situations where tools overcome LLM limitations
• Develop a decision framework for choosing the right tool for different tasks

Activity Overview

Cards

This activity uses input, output, capability, and risk cards:

Materials Needed

• The mat
• Input/output cards (text-input, text-output)
• Capability cards (websearch, code-sandbox, data-retrieval, image-generation)
• Risk card (hallucination)
• Post-it notes
• Markers
• Optional: Device with AI assistant for live demos

Step-by-Step Guide

Step 1: Introduction

Start with this analogy:

A Language Model is like a knowledgeable colleague who has some important limitations. They can only work with what they learned during training, and they can't take actions directly. This means they might give outdated information or talk about doing something without being able to actually do it.

Then explain how tools help address these limitations. Describe how sensing tools help verify information and action tools enable specific tasks, while noting that each has its constraints.

Demonstrate this using the mat:

1. First, place only text-input and text-output cards (basic chatbot)
2. Then add capability cards at the bottom to show how tools extend the basic model

Step 2: Tool Exploration

Begin with this framing:

Let's look at what each tool can and can't do. Understanding their limitations is just as important as knowing their capabilities.

Then introduce each capability card:

Web search Explain how AI can search the internet for current information, but remind participants that like any search, it might miss things or need multiple attempts.

Code sandbox Describe how AI can write and run code for calculations and data analysis, noting that it works well for defined tasks but may need guidance for complex operations.

Data retrieval Explain that AI can search through company documents and databases, emphasizing that effectiveness depends on data organization.

Image generation Discuss how AI can create certain types of images, being clear that results vary and often require multiple attempts.

Step 3: Limitation Matching

Present these real-world challenges:

1. "I need accurate calculations for quarterly budget projections"
2. "I need current information about competitor product launches"
3. "I need truly random customer selections for our satisfaction survey"
4. "I need to find specific clauses in our 200-page policy document"
5. "I need to visualize our new product concept for stakeholders"

As groups rotate through stations, encourage discussion with this prompt:

There might be multiple ways to solve each challenge. What matters is understanding why you'd choose one tool over another.

Step 4: Before/After Scenarios

Ask participants:

Think of a time when AI gave you an answer you couldn't trust. How might tools have helped?

Step 5: Decision Framework

Present this practical framework for tool selection:

When working with AI, consider:

• For current information: Use web search, but verify important facts
• For calculations: Use code sandbox for defined tasks
• For company information: Use data retrieval with well-organized data
• For visuals: Try image generation where appropriate

Step 6: Debrief

Guide a discussion about:

• Which tools could help with common AI frustrations
• Specific tasks where tools might be immediately useful
• How tools might affect trust in AI systems

End the discussion with this reflection:

Tools don't make AI perfect, but they do make it more reliable. The key is knowing when and how to use them.

Facilitation Tips

• Handle Expectations: When enthusiasm gets too high, remind participants that tools help but aren't magic.
• Manage Time: Set clear expectations that some demos will work better than others.
• Stay Practical: Keep focus on routine tasks where tools can provide reliable help.

What's Next

Close with this key message:

Start small. Pick one routine task where these tools might help, and experiment. Some things will work better than others - that's how we learn what's actually useful in our daily work.

The goal is to have participants find practical, reliable uses for these tools in their work.

Resources

• Why Tools Are Game-Changers for AI
• Model Context Protocol (MCP)