Supervised Fine-Tuning Archives - gettectonic.com
AI Market Heat

AI Market Heat

Alibaba Feels the Heat as DeepSeek Shakes Up AI Market Chinese tech giant Alibaba is under pressure following the release of an AI model by Chinese startup DeepSeek that has sparked a major reaction in the West. DeepSeek claims to have trained its model—comparable to advanced Western AI—at a fraction of the cost and with significantly fewer AI chips. In response, Alibaba launched Qwen 2.5-Max, its latest AI language model, on Tuesday—just one day before the Lunar New Year, when much of China’s economy typically slows down for a 15-day holiday. A Closer Look at Qwen 2.5-Max Qwen 2.5-Max is a Mixture of Experts (MoE) model trained on 20 trillion tokens. It has undergone supervised fine-tuning and reinforcement learning from human feedback to enhance its capabilities. MoE models function by using multiple specialized “minds,” each focused on a particular domain. When a query is received, the model dynamically routes it to the most relevant expert, improving efficiency. For instance, a coding-related question would be processed by the model’s coding expert. This MoE approach reduces computational requirements, making training more cost-effective and faster. Other AI vendors, such as France-based Mistral AI, have also embraced this technique. DeepSeek’s Disruptive Impact While Qwen 2.5-Max is not a direct competitor to DeepSeek’s R1 model—the release of which triggered a global selloff in AI stocks—it is similar to DeepSeek-V3, another MoE-based model launched earlier this month. Alibaba’s swift release underscores the competitive threat posed by DeepSeek. As the world’s fourth-largest public cloud vendor, Alibaba, along with other Chinese tech giants, has been forced to respond aggressively. In the wake of DeepSeek R1’s debut, ByteDance—the owner of TikTok—also rushed to update its AI offerings. DeepSeek has already disrupted the AI market by significantly undercutting costs. In 2023, the startup introduced V2 at just 1 yuan ($0.14) per million tokens, prompting a price war. By comparison, OpenAI’s GPT-4 starts at $10 per million tokens—a staggering difference. The timing of Alibaba and ByteDance’s latest releases suggests that DeepSeek has accelerated product development cycles across the industry, forcing competitors to move faster than planned. “Alibaba’s cloud unit has been rapidly advancing its AI technology, but the pressure from DeepSeek’s rise is immense,” said Lisa Martin, an analyst at Futurum Group. A Shifting AI Landscape DeepSeek’s rapid growth reflects a broader shift in the AI market—one driven by leaner, more powerful models that challenge conventional approaches. “The drive to build more efficient models continues,” said Gartner analyst Arun Chandrasekaran. “We’re seeing significant innovation in algorithm design and software optimization, allowing AI to run on constrained infrastructure while being more cost-competitive.” This evolution is not happening in isolation. “AI companies are learning from one another, continuously reverse-engineering techniques to create better, cheaper, and more efficient models,” Chandrasekaran added. The AI industry’s perception of cost and scalability has fundamentally changed. Sam Altman, CEO of OpenAI, previously estimated that training GPT-4 cost over $100 million—but DeepSeek claims it built R1 for just $6 million. “We’ve spent years refining how transformers function, and the efficiency gains we’re seeing now are the result,” said Omdia analyst Bradley Shimmin. “These advances challenge the idea that massive computing power is required to develop state-of-the-art AI.” Competition and Data Controversies DeepSeek’s success showcases the increasing speed at which AI innovation is happening. Its distillation technique, which trains smaller models using insights from larger ones, has allowed it to create powerful AI while keeping costs low. However, OpenAI and Microsoft are now investigating whether DeepSeek improperly used their models’ data to train its own AI—a claim that, if true, could escalate into a major dispute. Ironically, OpenAI itself has faced similar accusations, leading some enterprises to prefer using its models through Microsoft Azure, which offers additional compliance safeguards. “The future of AI development will require stronger security layers,” Shimmin noted. “Enterprises need assurances that using models like Qwen 2.5 or DeepSeek R1 won’t expose their data.” For businesses evaluating AI models, licensing terms matter. Alibaba’s Qwen 2.5 series operates under an Apache 2.0 license, while DeepSeek uses an MIT license—both highly permissive, allowing companies to scrutinize the underlying code and ensure compliance. “These licenses give businesses transparency,” Shimmin explained. “You can vet the code itself, not just the weights, to mitigate privacy and security risks.” The Road Ahead The AI arms race between DeepSeek, Alibaba, OpenAI, and other players is just beginning. As vendors push the limits of efficiency and affordability, competition will likely drive further breakthroughs—and potentially reshape the AI landscape faster than anyone anticipated. Like Related Posts Salesforce OEM AppExchange Expanding its reach beyond CRM, Salesforce.com has launched a new service called AppExchange OEM Edition, aimed at non-CRM service providers. Read more The Salesforce Story In Marc Benioff’s own words How did salesforce.com grow from a start up in a rented apartment into the world’s Read more Salesforce Jigsaw Salesforce.com, a prominent figure in cloud computing, has finalized a deal to acquire Jigsaw, a wiki-style business contact database, for Read more Service Cloud with AI-Driven Intelligence Salesforce Enhances Service Cloud with AI-Driven Intelligence Engine Data science and analytics are rapidly becoming standard features in enterprise applications, Read more

Read More
Salesforce AI Research Introduces LaTRO

Salesforce AI Research Introduces LaTRO

Salesforce AI Research Introduces LaTRO: A Breakthrough in Enhancing Reasoning for Large Language Models Large Language Models (LLMs) have revolutionized tasks such as answering questions, generating content, and assisting with workflows. However, they often struggle with advanced reasoning tasks like solving complex math problems, logical deduction, and structured data analysis. Salesforce AI Research has addressed this challenge by introducing LaTent Reasoning Optimization (LaTRO), a groundbreaking framework that enables LLMs to self-improve their reasoning capabilities during training. The Need for Advanced Reasoning in LLMs Reasoning—especially sequential, multi-step reasoning—is essential for tasks that require logical progression and problem-solving. While current models excel at simpler queries, they often fall short in tackling more complex tasks due to a reliance on external feedback mechanisms or runtime optimizations. Enhancing reasoning abilities is therefore critical to unlocking the full potential of LLMs across diverse applications, from advanced mathematics to real-time data analysis. Existing techniques like Chain-of-Thought (CoT) prompting guide models to break problems into smaller steps, while methods such as Tree-of-Thought and Program-of-Thought explore multiple reasoning pathways. Although these techniques improve runtime performance, they don’t fundamentally enhance reasoning during the model’s training phase, limiting the scope of improvement. Salesforce AI Research Introduces LaTRO: A Self-Rewarding Framework LaTRO shifts the paradigm by transforming reasoning into a training-level optimization problem. It introduces a self-rewarding mechanism that allows models to evaluate and refine their reasoning pathways without relying on external feedback or supervised fine-tuning. This intrinsic approach fosters continual improvement and empowers models to solve complex tasks more effectively. How LaTRO Works LaTRO’s methodology centers on sampling reasoning paths from a latent distribution and optimizing these paths using variational techniques. Here’s how it works: This self-rewarding cycle ensures that the model continuously refines its reasoning capabilities during training. Unlike traditional methods, LaTRO’s framework operates autonomously, without the need for external reward models or costly supervised feedback loops. Key Benefits of LaTRO Performance Highlights LaTRO’s effectiveness has been validated across various datasets and models: Applications and Implications LaTRO’s ability to foster logical coherence and structured reasoning has far-reaching applications in fields requiring robust problem-solving: By enabling LLMs to autonomously refine their reasoning processes, LaTRO brings AI closer to achieving human-like cognitive abilities. The Future of AI with LaTRO LaTRO sets a new benchmark in AI research by demonstrating that reasoning can be optimized during training, not just at runtime. This advancement by Salesforce AI Research highlights the potential for self-evolving AI models that can independently improve their problem-solving capabilities. Salesforce AI Research Introduces LaTRO As the field of AI progresses, frameworks like LaTRO pave the way for more autonomous, intelligent systems capable of navigating complex reasoning tasks across industries. LaTRO represents a significant leap forward, moving AI closer to achieving true autonomous reasoning. Like Related Posts Salesforce OEM AppExchange Expanding its reach beyond CRM, Salesforce.com has launched a new service called AppExchange OEM Edition, aimed at non-CRM service providers. Read more The Salesforce Story In Marc Benioff’s own words How did salesforce.com grow from a start up in a rented apartment into the world’s Read more Salesforce Jigsaw Salesforce.com, a prominent figure in cloud computing, has finalized a deal to acquire Jigsaw, a wiki-style business contact database, for Read more Service Cloud with AI-Driven Intelligence Salesforce Enhances Service Cloud with AI-Driven Intelligence Engine Data science and analytics are rapidly becoming standard features in enterprise applications, Read more

Read More
ChatGPT and Einstein GPT

ChatGPT and Einstein GPT

Artificial intelligence (AI) has been rapidly advancing globally, with breakthroughs captivating professionals across various sectors. One milestone that has gained significant attention is the emergence of ChatGPT, a cutting-edge language model revolutionizing the tech landscape. This development has profoundly impacted businesses relying on Salesforce for their customer relationship management (CRM) needs. In March 2023, Salesforce unveiled its latest AI innovation, Einstein GPT, promising to transform how companies engage with their clientele. In this article, we explore what Salesforce Einstein GPT entails and how it can benefit teams across diverse industries. When OpenAI introduced ChatGPT in November 2022, they didn’t expect the overwhelming response it received. Initially positioned as a “research preview,” this AI chatbot aimed to refine existing technology while soliciting feedback from users. However, ChatGPT quickly became a viral sensation, surpassing OpenAI’s expectations and prompting them to adapt to its newfound popularity. Developed on the foundation of the GPT-3.5 language model, ChatGPT was specifically tailored to facilitate engaging and accessible conversations, distinguishing it from its predecessors. Its launch attracted a diverse user base keen to explore its capabilities, prompting OpenAI to prioritize addressing potential misuse and enhancing its safety features. As ChatGPT gained traction, it caught the attention of Salesforce, a leading CRM provider. In March 2023, Salesforce unveiled Einstein GPT, its own AI innovation, poised to transform customer engagement. Built on the GPT-3 architecture and seamlessly integrated into Salesforce Clouds, Einstein GPT promised to revolutionize how businesses interact with their clientele. Einstein GPT boasts a range of features designed to personalize customer experiences and streamline workflows. From generating natural language responses to crafting personalized content and automating tasks, Einstein GPT offers versatility and value across industries. By leveraging both Einstein AI and GPT technology, businesses can unlock unprecedented efficiency and deliver superior customer experiences. Despite its success, OpenAI acknowledges the need for ongoing refinement and vigilance, emphasizing the importance of responsible deployment and transparency in the development of AI technology. Exploring Einstein GPT Salesforce presents Einstein GPT as the premier generative AI tool for CRM worldwide. Utilizing the advanced GPT-3 architecture, Einstein GPT seamlessly integrates into all Salesforce Clouds, including Tableau, MuleSoft, and Slack. This groundbreaking technology empowers users to generate natural language responses to customer inquiries, craft personalized content, and compose entire email messages on behalf of sales personnel. With its high degree of customization, Einstein GPT can be finely tuned to meet the specific needs of various industries, use cases, and customer requirements, delivering significant value to businesses of all sizes and sectors. Objectives of Salesforce AI Einstein GPT Salesforce AI Einstein GPT is designed to achieve several key objectives: Distinguishing Einstein GPT from Einstein AI Einstein GPT represents the latest evolution of Salesforce’s Einstein artificial intelligence technology. Unlike its predecessors, Einstein GPT integrates proprietary Einstein AI models with ChatGPT and other leading large language models. This integration enables users to interact with CRM data using natural language prompts, resulting in highly personalized, AI-generated content and triggering powerful automations that enhance workflows and productivity. By leveraging both Einstein AI and GPT technology, businesses can achieve unparalleled efficiency and deliver exceptional customer experiences. Features of Einstein GPT in Salesforce CRM Key features and capabilities of Salesforce Einstein chatbot GPT include: Utilizing Einstein GPT for Business Improvement Einstein GPT can be leveraged across various domains to enhance business operations: Integration with Salesforce Data Cloud Salesforce Data Cloud, a cloud-based data management system, enables real-time data aggregation from diverse sources. Einstein GPT utilizes unified customer data profiles from the Salesforce Data Cloud to personalize interactions throughout the customer journey. OpenAI on ChatGPT Methods We trained this model using Reinforcement Learning from Human Feedback (RLHF), using the same methods as InstructGPT, but with slight differences in the data collection setup. We trained an initial model using supervised fine-tuning: human AI trainers provided conversations in which they played both sides—the user and an AI assistant. We gave the trainers access to model-written suggestions to help them compose their responses. We mixed this new dialogue dataset with the InstructGPT dataset, which we transformed into a dialogue format. To create a reward model for reinforcement learning, we needed to collect comparison data, which consisted of two or more model responses ranked by quality. To collect this data, we took conversations that AI trainers had with the chatbot. We randomly selected a model-written message, sampled several alternative completions, and had AI trainers rank them. Using these reward models, we can fine-tune the model using Proximal Policy Optimization. We performed several iterations of this process. ChatGPT is fine-tuned from a model in the GPT-3.5 series, which finished training in early 2022. You can learn more about the 3.5 series here. ChatGPT and GPT-3.5 were trained on an Azure AI supercomputing infrastructure. Limitations ChatGPT and Einstein GPT Salesforce Einstein GPT signifies a significant advancement in AI technology, empowering businesses to deliver tailored customer experiences and streamline operations. With its integration into Salesforce CRM and other platforms, Einstein GPT offers unprecedented capabilities for personalized engagement and automated insights, ensuring organizations remain competitive in today’s dynamic market landscape. When OpenAI quietly launched ChatGPT in late November 2022, the San Francisco-based AI company didn’t anticipate the viral sensation it would become. Initially viewed as a “research preview,” it was meant to showcase a refined version of existing technology while gathering feedback from the public to address its flaws. However, the overwhelming success of ChatGPT caught OpenAI off guard, leading to a scramble to capitalize on its newfound popularity. ChatGPT, based on the GPT-3.5 language model, was fine-tuned to be more conversational and accessible, setting it apart from previous iterations. Its release marked a significant milestone, attracting millions of users eager to test its capabilities. OpenAI quickly realized the need to address potential misuse and improve the model’s safety features. Since its launch, ChatGPT has undergone several updates, including the implementation of adversarial training to prevent users from exploiting it (known as “jailbreaking”). This technique involves pitting multiple chatbots against each other to identify and neutralize malicious behavior. Additionally,

Read More
gettectonic.com