The Gap
What most AI research overlooks
Most research on generative AI focuses on model performance, recommendation quality, accuracy, or hallucinations. Far less attention has been paid to something simple: the tone of the AI itself.
Lab experiment · GenAI · User behavior
AI tone in itinerary planning
Large language models can deliver the same information in very different styles. Does that style change how people use an AI assistant? To find out, I built a custom AI-powered travel planning platform and ran a controlled lab experiment with 87 participants in which only the tone of the AI varied.
My role: I designed and built the platform and conducted the lab experiment, including behavioral logging and eye tracking. I am also a co-author of a related field experiment on the same question (see below).
- Generative AI
- User Engagement
- Eye Tracking
- Prompt Engineering
- Travel Planning
01 / Overview
Research motivation
The Observation
Same information, very different experiences
A travel assistant might describe a destination as "The cathedral is a major tourist attraction." — or as "The cathedral is an absolute must-see and one of the most breathtaking highlights of the city!" Both communicate the same fact, but create very different user experiences.
The Question
Does tone produce measurable effects?
A single line in the system prompt can make a model sound enthusiastic or strictly neutral. If such a small design choice changes engagement, trust, or decisions, it deserves the same attention as model quality.
02 / Platform
Building an AI travel planning platform
To study this question, I built a custom web-based travel planning system. Participants planned road trips in four European countries: Spain, Italy, Portugal, and Germany.
Module 1
Interactive itinerary planner
Users could select travel preferences, choose activities, define trip characteristics, and modify itineraries using drag-and-drop interactions.
Module 2
AI-generated travel stories
For every day of the trip, the system generated a narrative description of the planned activities and destinations — the primary vehicle for the tone manipulation.
Module 3
Conversational chatbot
Participants could ask follow-up questions, request modifications, or seek recommendations from an integrated generative AI assistant.
Module 4
Interactive map
A map visualization allowed participants to inspect and understand their travel route throughout the planning session.
Design goal
Realistic experience over artificial survey
The goal was to create a realistic travel-planning experience rather than an artificial survey task — giving us access to genuine behavioral engagement data rather than hypothetical self-reports.
03 / Experimental Manipulation
Three tone conditions
The most important aspect of the study is also the simplest. Everything remained identical across participants except for the tone of the AI assistant.
Condition A
Enthusiastic tone
The AI was instructed to communicate in an expressive and emotionally engaging manner. Descriptions emphasized excitement, anticipation, and positive emotions — destinations were presented as exciting, breathtaking, unforgettable, must-see experiences.
Condition B
Neutral tone
The AI communicated in a factual and restrained style. Responses focused on information rather than emotional framing. The assistant still provided the same recommendations but without expressive language or persuasive wording.
Condition C
Default (control)
The control condition used the model's standard behavior without explicit tone instructions. This represents how many real-world AI applications currently operate.
04 / Procedure
What participants did
Each laboratory session lasted approximately one hour. Participants completed four separate travel-planning tasks, allowing us to observe not only what users said about the system but also how they actually behaved while using it.
Per Trip
Six-step planning sequence
- Defined travel preferences.
- Reviewed the generated itinerary.
- Modified the route if desired.
- Read AI-generated travel descriptions.
- Interacted with the chatbot.
- Evaluated the final trip.
Participants
87 participants, 348 sessions
Participants were primarily university students — a population that frequently uses digital tools and regularly engages in travel planning. Financial constraints were intentionally removed from the scenario to focus attention on the AI interaction itself.
05 / Measurement
Measuring user engagement
One advantage of running the experiment on a custom platform was that every interaction could be logged automatically, providing behavioral measures beyond what surveys alone could capture.
Behavioral logging
What the platform recorded automatically
- Whether participants used the chatbot
- Number of prompts submitted
- Length of individual prompts
- Frequency of itinerary modifications
- Navigation patterns through the interface
Eye Tracking
Attention across interface elements
Eye-tracking technology allowed us to measure fixation durations, gaze patterns, transitions between interface elements, and allocation of visual attention.
Areas of interest
06 / Findings
Lab findings
Engagement
Neutral tone generated more interaction
Contrary to the intuition that an enthusiastic assistant is more engaging, the neutral assistant generated the most interaction. Participants in the neutral condition were more likely to use the chatbot, submitted more prompts, and wrote longer prompts.
Interpretation
Why neutral may invite more input
A restrained assistant may leave more room for users to articulate their own needs, whereas an enthusiastic one drives the conversation itself. Users then contribute more information rather than following the AI's lead.
Helpfulness
Chatbot use increased perceived helpfulness
Across all conditions, participants rated the platform as more helpful when they actively used the chatbot, which highlights the value of conversational interaction itself, independent of tone.
07 / Related Study
Lab vs. field
The same question was also tested in a randomized field experiment with real users of an online travel planning service, published as a preprint on which I am a co-author.
This study · Lab experiment
Controlled setting, 87 participants
- Custom platform, four planning tasks per participant
- Behavioral logging and eye tracking
- Neutral tone led to more chatbot use and longer prompts
Related study · Field experiment
Real users, real purchase decisions
- Randomized tone conditions in live itinerary planning
- Enthusiastic tone led to longer prompts
- Enthusiastic and neutral tone increased subscription purchases
Takeaway: Tone effects depend on context. What drives engagement in a controlled lab task can differ from what drives behavior when real users plan real trips. For product teams, this argues for testing tone in production, for example through A/B tests, rather than relying on lab results or intuition alone.