OpenAI’s o1 model

September 26, 2024in Salesforce

The release of OpenAI’s o1 model has sparked some confusion. Unlike previous models that focused on increasing parameters and capabilities, this one takes a different approach. Let’s explore the technical distinctions first, share a real-world experience, and wrap up with some recommendations on when to use each model.

Technical Differences

The core difference is that o1 serves as an “agentic wrapper” around GPT-4 (or a similar model). This means it incorporates a layer of metacognition, or “thinking about thinking,” before addressing a query. Instead of immediately answering the question, o1 first evaluates the best strategy for tackling it by breaking it down into subtasks.

Once this analysis is complete, o1 begins executing each subtask. Depending on the answers it receives, it may adjust its approach. This method resembles the “tree of thought” strategy, allowing users to see real-time explanations of the subtasks being addressed. For a deeper dive into agentic approaches, I highly recommend Andrew Ng’s insightful letters on the topic.

However, this method comes with a cost—it’s about six times more expensive and approximately six times slower than traditional approaches. While this metacognitive process can enhance understanding, it doesn’t guarantee improved answers for straightforward factual queries or tasks like generating trivia questions, where simplicity may yield better results.

Real-World Example

To illustrate the practical implications, Tectonic began to deepen the understanding of variational autoencoders—a trend in multimodal LLMs. While we had a basic grasp of the concept, we had specific questions about their advantages over traditional autoencoders and the nuances of training them. This information isn’t easily accessible through a simple search; it’s more akin to seeking insight from a domain expert.

To enhance our comprehension, we engaged with both GPT-4 and o1. We quickly noticed that o1’s responses were more thoughtful and facilitated a meaningful dialogue. In contrast, GPT-4 tended to recycle the same information, offering limited depth—much like how some people might respond in conversation.

A particularly striking example occurred when we attempted to clarify our understanding.

Interaction with o1: It took 27 seconds to process our input before analyzing my paragraph sentence by sentence. It identified what we understood correctly and highlighted a key misunderstanding: it’s the differentiability that matters, not the multiplication.
Interaction with GPT-4: In this case, it merely repeated its earlier explanation of variational autoencoders without genuinely engaging with my input.

The difference was notable. o1 responded like a thoughtful colleague, addressing our specific points, while GPT-4 felt more like a know-it-all friend who rambled on, requiring me to sift through the information for valuable insights.

Summary and Recommendations

In essence, if we were to personify these models, GPT-4 would be the overzealous friend who dives into a stream of consciousness, while o1 would be the more attentive listener who takes a moment to reflect before delivering precise and relevant insights.

Here are some scenarios where o1 may outperform GPT-4, justifying its higher cost:

Personal Learning: Use o1 to improve your understanding of complex issues or to thoughtfully work through problems by responding to your input.
Logical Reasoning Tasks: Engage o1 for challenges where logical reasoning is paramount, such as math olympiad questions or as a virtual partner in escape room games.
Strategic Problem-Solving: Opt for o1 when it’s beneficial to consider the approach to solving a problem before jumping in, such as with puzzles or multi-stage challenges.

By leveraging these insights, you can better navigate the strengths of each model in your tasks and inquiries.