Communicating With Machines

For as long as machines have existed, humans have struggled to communicate effectively with them. The rise of large language models (LLMs) has transformed this dynamic, making “prompting” the bridge between our intentions and AI’s actions. By providing pre-trained models with clear instructions and context, we can ensure they understand and respond correctly. As UX practitioners, we now play a key role in facilitating this interaction, helping humans and machines truly connect.

Thank you for reading this post, don't forget to subscribe!

The UX discipline was born alongside graphical user interfaces (GUIs), offering a way for the average person to interact with computers without needing to write code. We introduced familiar concepts like desktops, trash cans, and save icons to align with users’ mental models, while complex code ran behind the scenes. Now, with the power of AI and the transformer architecture, a new form of interaction has emerged—natural language communication. This shift has changed the design landscape, moving us from pure graphical interfaces to an era where text-based interactions dominate. As designers, we must reconsider where our focus should lie in this evolving environment.

A Mental Shift

In the era of command-based design, we focused on breaking down complex user problems, mapping out customer journeys, and creating deterministic flows. Now, with AI at the forefront, our challenge is to provide models with the right context for optimal output and refine the responses through iteration.

Shifting Complexity to the Edges

Successful communication, whether with a person or a machine, hinges on context. Just as you would clearly explain your needs to a salesperson to get the right product, AI models also need clear instructions. Expecting users to input all the necessary information in their prompts won’t lead to widespread adoption of these models.

Here, UX practitioners play a critical role. We can design user experiences that integrate context—some visible to users, others hidden—shaping how AI interacts with them. This ensures that users can seamlessly communicate with machines without the burden of detailed, manual prompts.

The Craft of Prompting

As designers, our role in crafting prompts falls into three main areas:

Interface Prompting: We can guide users by offering conversation starters, suggesting follow-ups, or even re-prompting automatically with contextual hints.
System Prompting: Behind the scenes, we provide instructions to steer the model—whether regarding tone, safety, or brand values. Designers can shape how AI models interact in alignment with company goals.
Training Data Prompting: UX should be involved in refining datasets for AI training, ensuring high-quality inputs that are free from bias or conflicting information.

Even if your team isn’t building custom models, there’s still plenty of work to be done. You can help select pre-trained models that align with user goals and design a seamless experience around them.

Understanding the Context Window

A key concept for UX designers to understand is the “context window“—the information a model can process to generate an output. Think of it as the amount of memory the model retains during a conversation. Companies can use this to include hidden prompts, helping guide AI responses to align with brand values and user intent.

Context windows are measured in tokens, not time, so even if you return to a conversation weeks later, the model remembers previous interactions, provided they fit within the token limit. With innovations like Gemini’s 2-million-token context window, AI models are moving toward infinite memory, which will bring new design challenges for UX practitioners.

How to Approach Prompting

Prompting is an iterative process where you craft an instruction, test it with the model, and refine it based on the results. Some effective techniques include:

Use a framework: A structure like RACE (Role, Action, Context, Expectation) helps organize information clearly for the model.
Give examples: Providing examples shows the model how you want it to respond. This is called “Few-Shot” prompting, whereas simply giving a command is “Zero-Shot.”
Ask the model to reason: Asking for explanations (Chain of Thought) helps the model process complex tasks more effectively.
Specify what not to do: Clear guidelines, including examples of incorrect responses, help the model avoid errors.

Depending on the scenario, you’ll either use direct, simple prompts (for user-facing interactions) or broader, more structured system prompts (for behind-the-scenes guidance).

Get Organized

As prompting becomes more common, teams need a unified approach to avoid conflicting instructions. Proper documentation on system prompting is crucial, especially in larger teams. This helps prevent errors and hallucinations in model responses.

Prompt experimentation may reveal limitations in AI models, and there are several ways to address these:

Refine system prompts to reduce errors.
Fine-tune models by collaborating with engineers.
Disclaimers can inform users of potential limitations.
Model replacement may be necessary if the chosen AI fails to meet expectations.

Looking Ahead

The UX landscape is evolving rapidly. Many organizations, particularly smaller ones, have yet to realize the importance of UX in AI prompting. Others may not allocate enough resources, underestimating the complexity and importance of UX in shaping AI interactions.

As John Culkin said, “We shape our tools, and thereafter, our tools shape us.” The responsibility of integrating UX into AI development goes beyond just individual organizations—it’s shaping the future of human-computer interaction.

This is a pivotal moment for UX, and how we adapt will define the next generation of design.

Content updated October 2024.