Large Language Model Archives - gettectonic.com - Page 7

GENAI Shows No Racial or Sexual Bias

Researchers from Mass General Brigham recently published findings in PAIN indicating that large language models (LLMs) do not exhibit race- or sex-based biases when recommending opioid treatments. The team highlighted that, while biases are prevalent in many areas of healthcare, they are particularly concerning in pain management. Studies have shown that Black patients’ pain is often underestimated and undertreated by clinicians, while white patients are more likely to be prescribed opioids than other racial and ethnic groups. These disparities raise concerns that AI tools, including LLMs, could perpetuate or exacerbate such biases in healthcare. To investigate how AI tools might either mitigate or reinforce biases, the researchers explored how LLM recommendations varied based on patients’ race, ethnicity, and sex. Using 40 real-world patient cases from the MIMIC-IV Note data set—each involving complaints of headache, abdominal, back, or musculoskeletal pain—the cases were stripped of references to sex and race. Random race categories (American Indian or Alaska Native, Asian, Black, Hispanic or Latino, Native Hawaiian or Other Pacific Islander, and white) and sex (male or female) were then assigned to each case. This process was repeated until all combinations of race and sex were generated, resulting in 480 unique cases. These cases were analyzed using GPT-4 and Gemini, both of which assigned subjective pain ratings and made treatment recommendations. The analysis found that neither model made opioid treatment recommendations that differed by race or sex. However, the tools did show some differences—GPT-4 tended to rate pain as “severe” more frequently than Gemini, which was more likely to recommend opioids. While further validation is necessary, the researchers believe the results indicate that LLMs could help address biases in healthcare. “These results are reassuring in that patient race, ethnicity, and sex do not affect recommendations, indicating that these LLMs have the potential to help address existing bias in healthcare,” said co-first authors Cameron Young and Ellie Einchen, students at Harvard Medical School, in a press release. However, the study has limitations. It categorized sex as a binary variable, omitting a broader gender spectrum, and it did not fully represent mixed-race individuals, leaving certain marginalized groups underrepresented. The team suggested future research should incorporate these factors and explore how race influences LLM recommendations in other medical specialties. Marc Succi, MD, strategic innovation leader at Mass General Brigham and corresponding author of the study, emphasized the need for caution in integrating AI into healthcare. “There are many elements to consider, such as the risks of over-prescribing or under-prescribing medications and whether patients will accept AI-influenced treatment plans,” Succi said. “Our study adds key data showing how AI has the potential to reduce bias and improve health equity.” Succi also noted the broader implications of AI in clinical decision support, suggesting that AI tools will serve as complementary aids to healthcare professionals. “In the short term, AI algorithms can act as a second set of eyes, running in parallel with medical professionals,” he said. “However, the final decision will always remain with the doctor.” These findings offer important insights into the role AI could play in reducing bias and enhancing equity in pain management and healthcare overall. Like Related Posts Who is Salesforce? Who is Salesforce? Here is their story in their own words. From our inception, we’ve proudly embraced the identity of Read more Salesforce Unites Einstein Analytics with Financial CRM Salesforce has unveiled a comprehensive analytics solution tailored for wealth managers, home office professionals, and retail bankers, merging its Financial Read more AI-Driven Propensity Scores AI plays a crucial role in propensity score estimation as it can discern underlying patterns between treatments and confounding variables Read more Tectonic’s Successful Salesforce Track Record Salesforce Technology Services Integrator – Tectonic has successfully delivered Salesforce in a variety of industries including Public Sector, Hospitality, Manufacturing, Read more

Read More
October 23, 2024in AI Tools, Data
AI That Forgets
23Oct

AI That Forgets

Salesforce has introduced a generative AI system designed to prioritize data privacy through a unique “forgetting” feature. This innovation allows the AI to process information through large language models (LLMs) without retaining the data, helping companies manage sensitive information more securely. AI That Forgets. As part of the latest wave in generative AI, Salesforce’s solution takes the form of digital “agents”—intelligent systems capable of understanding and responding to customer inquiries autonomously. CEO Marc Benioff has hailed this development as a significant breakthrough for the company, emphasizing its potential to transform customer interactions. AI That Forgets. At a recent event, Patrick Stokes, Salesforce’s EVP of Products and Industries, highlighted how this system supports organizations by reducing the costs and risks associated with building their own AI models. According to Stokes, many companies lack the resources to develop in-house AI sustainably, and Salesforce’s privacy-first approach provides an appealing alternative. Rather than focusing solely on creating the most powerful LLM, Salesforce has built AI agents that connect data and actions securely, addressing privacy concerns that have hindered AI adoption. AI That Forgets Salesforce’s approach integrates privacy-focused safeguards, which Stokes describes as a “trust layer” within the AI system. This feature verifies that data retrieved during an AI query aligns with the user’s access permissions, protecting sensitive information. Stokes notes that unlike traditional AI models that retain data, Salesforce’s LLM processes only the information required for each interaction and then “forgets” it afterward. This zero-retention approach creates a more secure environment, where companies retain governance over data usage and minimize risks associated with long-term data storage. Zahra Bahrololoumi, CEO of Salesforce UK and Ireland, also emphasized that Salesforce’s AI solutions offer users the confidence to adopt generative AI without compromising security. With over 1,000 AI agents already implemented, companies are benefiting from reduced burnout and increased productivity while maintaining data trust and integrity. Like Related Posts Who is Salesforce? Who is Salesforce? Here is their story in their own words. From our inception, we’ve proudly embraced the identity of Read more Salesforce Marketing Cloud Transactional Emails Salesforce Marketing Cloud Transactional Emails are immediate, automated, non-promotional messages crucial to business operations and customer satisfaction, such as order Read more Salesforce Unites Einstein Analytics with Financial CRM Salesforce has unveiled a comprehensive analytics solution tailored for wealth managers, home office professionals, and retail bankers, merging its Financial Read more AI-Driven Propensity Scores AI plays a crucial role in propensity score estimation as it can discern underlying patterns between treatments and confounding variables Read more

Read More
October 23, 2024in Data, Generative AI
Google on Google AI
21Oct

Google on Google AI

As a leading cloud provider, Google Cloud is also a major player in the generative AI market. Google on Google AI provides insights into this new tool. In the past two years, Google has been in a competitive battle with AWS, Microsoft, and OpenAI to gain dominance in the generative AI space. Recently, Google introduced several generative Artificial Intelligence products, including its flagship large language model, Gemini, and the Vertex AI Model Garden. Last week, it also unveiled Audio Overview, a tool that transforms documents into audio discussions. Despite these advancements, Google has faced criticism for lagging in some areas, such as issues with its initial image generation tool, like X’s Grok. However, the company remains committed to driving progress in generative AI. Google’s strategy focuses not only on delivering its proprietary models but also offering a broad selection of third-party models through its Model Garden. Google’s Thoughts on Google AI Warren Barkley, head of product for Google Cloud’s Vertex AI, GenAI, and machine learning, emphasized this approach in a recent episode of the Targeting AI podcast. He noted that a key part of Google’s ongoing effort is ensuring users can easily transition to more advanced models. “A lot of what we did in the early days, and we continue to do now, is make it easy for people to move to the next generation,” Barkley said. “The models we built 18 months ago are a shadow of what we have today. So, providing pathways for people to upgrade and stay on the cutting edge is critical.” Google is also focused on helping users select the right AI models for specific applications. With over 100 closed and open models available in the Model Garden, evaluating them can be challenging for customers. To address this, Google introduced evaluation tools that allow users to test prompts and compare model responses. In addition, Google is exploring advancements in Artificial Intelligence reasoning, which it views as crucial to driving the future of generative AI. Like Related Posts Who is Salesforce? Who is Salesforce? Here is their story in their own words. From our inception, we’ve proudly embraced the identity of Read more Salesforce Unites Einstein Analytics with Financial CRM Salesforce has unveiled a comprehensive analytics solution tailored for wealth managers, home office professionals, and retail bankers, merging its Financial Read more AI-Driven Propensity Scores AI plays a crucial role in propensity score estimation as it can discern underlying patterns between treatments and confounding variables Read more Tectonic’s Successful Salesforce Track Record Salesforce Technology Services Integrator – Tectonic has successfully delivered Salesforce in a variety of industries including Public Sector, Hospitality, Manufacturing, Read more

Read More
October 21, 2024in Generative AI, Google
Fully Formatted Facts
20Oct

Fully Formatted Facts

A recent discovery by programmer and inventor Michael Calvin Wood is addressing a persistent challenge in AI: hallucinations. These false or misleading outputs, long considered an inherent flaw in large language models (LLMs), have posed a significant issue for developers. However, Wood’s breakthrough is challenging this assumption, offering a solution that could transform how AI-powered applications are built and used. The Importance of Wood’s Discovery for Developers Wood’s findings have substantial implications for developers working with AI. By eliminating hallucinations, developers can ensure that AI-generated content is accurate and reliable, particularly in applications where precision is critical. Understanding the Root Cause of Hallucinations Contrary to popular belief, hallucinations are not primarily caused by insufficient training data or biased algorithms. Wood’s research reveals that the issue stems from how LLMs process and generate information based on “noun-phrase routes.” LLMs organize information around noun phrases, and when they encounter semantically similar phrases, they may conflate or misinterpret them, leading to incorrect outputs. How LLMs Organize Information For example: The Noun-Phrase Dominance Model Wood’s research led to the development of the Noun-Phrase Dominance Model, which posits that neural networks in LLMs self-organize around noun phrases. This model is key to understanding and eliminating hallucinations by addressing how AI processes noun-phrase conflicts. Fully-Formatted Facts (FFF): A Solution Wood’s solution involves transforming input data into Fully-Formatted Facts (FFF)—statements that are literally true, devoid of noun-phrase conflicts, and structured as simple, complete sentences. Presenting information in this format has led to significant improvements in AI accuracy, particularly in question-answering tasks. How FFF Processing Works While Wood has not provided a step-by-step guide for FFF processing, he hints that the process began with named-entity recognition using the Python SpaCy library and evolved into using an LLM to reduce ambiguity while retaining the original writing style. His company’s REST API offers a wrapper around GPT-4o and GPT-4o-mini models, transforming input text to remove ambiguity before processing it. Current Methods vs. Wood’s Approach Current approaches, like Retrieval Augmented Generation (RAG), attempt to reduce hallucinations by adding more context. However, these methods often introduce additional noun-phrase conflicts. For instance, even with RAG, ChatGPT-3.5 Turbo experienced a 23% hallucination rate when answering questions about Wikipedia articles. In contrast, Wood’s method focuses on eliminating noun-phrase conflicts entirely. Results: RAG FF (Retrieval Augmented Generation with Formatted Facts) Wood’s method has shown remarkable results, eliminating hallucinations in GPT-4 and GPT-3.5 Turbo during question-answering tasks using third-party datasets. Real-World Example: Translation Error Elimination Consider a simple translation example: This transformation eliminates hallucinations by removing the potential noun-phrase conflict. Implications for the Future of AI The Noun-Phrase Dominance Model and the use of Fully-Formatted Facts have far-reaching implications: Roadmap for Future Development Wood and his team plan to expand their approach by: Conclusion: A New Era of Reliable AI Wood’s discovery represents a significant leap forward in the pursuit of reliable AI. By aligning input data with how LLMs process information, he has unlocked the potential for accurate, trustworthy AI systems. As this technology continues to evolve, it could have profound implications for industries ranging from healthcare to legal services, where AI could become a consistent and reliable tool. While there is still work to be done in expanding this method across all AI tasks, the foundation has been laid for a revolution in AI accuracy. Future developments will likely focus on refining and expanding these capabilities, enabling AI to serve as a trusted resource across a range of applications. Experience RAGFix For those looking to explore this technology, RAGFix offers an implementation of these groundbreaking concepts. Visit their official website to access demos, explore REST API integration options, and stay updated on the latest advancements in hallucination-free AI: Visit RAGFix.ai Like Related Posts Who is Salesforce? Who is Salesforce? Here is their story in their own words. From our inception, we’ve proudly embraced the identity of Read more Salesforce Marketing Cloud Transactional Emails Salesforce Marketing Cloud Transactional Emails are immediate, automated, non-promotional messages crucial to business operations and customer satisfaction, such as order Read more Salesforce Unites Einstein Analytics with Financial CRM Salesforce has unveiled a comprehensive analytics solution tailored for wealth managers, home office professionals, and retail bankers, merging its Financial Read more AI-Driven Propensity Scores AI plays a crucial role in propensity score estimation as it can discern underlying patterns between treatments and confounding variables Read more

Read More
October 20, 2024in Salesforce
LLMs and AI
19Oct

LLMs and AI

Large Language Models (LLMs): Revolutionizing AI and Custom Solutions Large Language Models (LLMs) are transforming artificial intelligence by enabling machines to generate and comprehend human-like text, making them indispensable across numerous industries. The global LLM market is experiencing explosive growth, projected to rise from $1.59 billion in 2023 to $259.8 billion by 2030. This surge is driven by the increasing demand for automated content creation, advances in AI technology, and the need for improved human-machine communication. Several factors are propelling this growth, including advancements in AI and Natural Language Processing (NLP), large datasets, and the rising importance of seamless human-machine interaction. Additionally, private LLMs are gaining traction as businesses seek more control over their data and customization. These private models provide tailored solutions, reduce dependency on third-party providers, and enhance data privacy. This guide will walk you through building your own private LLM, offering valuable insights for both newcomers and seasoned professionals. What are Large Language Models? Large Language Models (LLMs) are advanced AI systems that generate human-like text by processing vast amounts of data using sophisticated neural networks, such as transformers. These models excel in tasks such as content creation, language translation, question answering, and conversation, making them valuable across industries, from customer service to data analysis. LLMs are generally classified into three types: LLMs learn language rules by analyzing vast text datasets, similar to how reading numerous books helps someone understand a language. Once trained, these models can generate content, answer questions, and engage in meaningful conversations. For example, an LLM can write a story about a space mission based on knowledge gained from reading space adventure stories, or it can explain photosynthesis using information drawn from biology texts. Building a Private LLM Data Curation for LLMs Recent LLMs, such as Llama 3 and GPT-4, are trained on massive datasets—Llama 3 on 15 trillion tokens and GPT-4 on 6.5 trillion tokens. These datasets are drawn from diverse sources, including social media (140 trillion tokens), academic texts, and private data, with sizes ranging from hundreds of terabytes to multiple petabytes. This breadth of training enables LLMs to develop a deep understanding of language, covering diverse patterns, vocabularies, and contexts. Common data sources for LLMs include: Data Preprocessing After data collection, the data must be cleaned and structured. Key steps include: LLM Training Loop Key training stages include: Evaluating Your LLM After training, it is crucial to assess the LLM’s performance using industry-standard benchmarks: When fine-tuning LLMs for specific applications, tailor your evaluation metrics to the task. For instance, in healthcare, matching disease descriptions with appropriate codes may be a top priority. Conclusion Building a private LLM provides unmatched customization, enhanced data privacy, and optimized performance. From data curation to model evaluation, this guide has outlined the essential steps to create an LLM tailored to your specific needs. Whether you’re just starting or seeking to refine your skills, building a private LLM can empower your organization with state-of-the-art AI capabilities. For expert guidance or to kickstart your LLM journey, feel free to contact us for a free consultation. Like Related Posts Who is Salesforce? Who is Salesforce? Here is their story in their own words. From our inception, we’ve proudly embraced the identity of Read more Salesforce Marketing Cloud Transactional Emails Salesforce Marketing Cloud Transactional Emails are immediate, automated, non-promotional messages crucial to business operations and customer satisfaction, such as order Read more Salesforce Unites Einstein Analytics with Financial CRM Salesforce has unveiled a comprehensive analytics solution tailored for wealth managers, home office professionals, and retail bankers, merging its Financial Read more AI-Driven Propensity Scores AI plays a crucial role in propensity score estimation as it can discern underlying patterns between treatments and confounding variables Read more

Read More
October 19, 2024in Artificial Intelligence, Data, Google, Salesforce, Technology
Generative AI Energy Consumption Rises
18Oct

Generative AI Energy Consumption Rises

Generative AI Energy Consumption Rises, but Impact on ROI Unclear The energy costs associated with generative AI (GenAI) are often overlooked in enterprise financial planning. However, industry experts suggest that IT leaders should account for the power consumption that comes with adopting this technology. When building a business case for generative AI, some costs are evident, like large language model (LLM) fees and SaaS subscriptions. Other costs, such as preparing data, upgrading cloud infrastructure, and managing organizational changes, are less visible but significant. Generative AI Energy Consumption Rises One often overlooked cost is the energy consumption of generative AI. Training LLMs and responding to user requests—whether answering questions or generating images—demands considerable computing power. These tasks generate heat and necessitate sophisticated cooling systems in data centers, which, in turn, consume additional energy. Despite this, most enterprises have not focused on the energy requirements of GenAI. However, the issue is gaining more attention at a broader level. The International Energy Agency (IEA), for instance, has forecasted that electricity consumption from data centers, AI, and cryptocurrency could double by 2026. By that time, data centers’ electricity use could exceed 1,000 terawatt-hours, equivalent to Japan’s total electricity consumption. Goldman Sachs also flagged the growing energy demand, attributing it partly to AI. The firm projects that global data center electricity use could more than double by 2030, fueled by AI and other factors. ROI Implications of Energy Costs The extent to which rising energy consumption will affect GenAI’s return on investment (ROI) remains unclear. For now, the perceived benefits of GenAI seem to outweigh concerns about energy costs. Most businesses have not been directly impacted, as these costs tend to affect hyperscalers more. For instance, Google reported a 13% increase in greenhouse gas emissions in 2023, largely due to AI-related energy demands in its data centers. Scott Likens, PwC’s global chief AI engineering officer, noted that while energy consumption isn’t a barrier to adoption, it should still be factored into long-term strategies. “You don’t take it for granted. There’s a cost somewhere for the enterprise,” he said. Energy Costs: Hidden but Present Although energy expenses may not appear on an enterprise’s invoice, they are still present. Generative AI’s energy consumption is tied to both model training and inference—each time a user makes a query, the system expends energy to generate a response. While the energy used for individual queries is minor, the cumulative effect across millions of users can add up. How these costs are passed to customers is somewhat opaque. Licensing fees for enterprise versions of GenAI products likely include energy costs, spread across the user base. According to PwC’s Likens, the costs associated with training models are shared among many users, reducing the burden on individual enterprises. On the inference side, GenAI vendors charge for tokens, which correspond to computational power. Although increased token usage signals higher energy consumption, the financial impact on enterprises has so far been minimal, especially as token costs have decreased. This may be similar to buying an EV to save on gas but spending hundreds and losing hours at charging stations. Energy as an Indirect Concern While energy costs haven’t been top-of-mind for GenAI adopters, they could indirectly address the issue by focusing on other deployment challenges, such as reducing latency and improving cost efficiency. Newer models, such as OpenAI’s GPT-4o mini, are more economical and have helped organizations scale GenAI without prohibitive costs. Organizations may also use smaller, fine-tuned models to decrease latency and energy consumption. By adopting multimodel approaches, enterprises can choose models based on the complexity of a task, optimizing for both speed and energy efficiency. The Data Center Dilemma As enterprises consider GenAI’s energy demands, data centers face the challenge head-on, investing in more sophisticated cooling systems to handle the heat generated by AI workloads. According to the Dell’Oro Group, the data center physical infrastructure market grew in the second quarter of 2024, signaling the start of the “AI growth cycle” for infrastructure sales, particularly thermal management systems. Liquid cooling, more efficient than air cooling, is gaining traction as a way to manage the heat from high-performance computing. This method is expected to see rapid growth in the coming years as demand for AI workloads continues to increase. Nuclear Power and AI Energy Demands To meet AI’s growing energy demands, some hyperscalers are exploring nuclear energy for their data centers. AWS, Google, and Microsoft are among the companies exploring this option, with AWS acquiring a nuclear-powered data center campus earlier this year. Nuclear power could help these tech giants keep pace with AI’s energy requirements while also meeting sustainability goals. I don’t know. It seems like if you akin AI accessibility to more nuclear power plants you would lose a lot of fans. As GenAI continues to evolve, both energy costs and efficiency are likely to play a greater role in decision-making. PwC has already begun including carbon impact as part of its GenAI value framework, which assesses the full scope of generative AI deployments. “The cost of carbon is in there, so we shouldn’t ignore it,” Likens said. Generative AI Energy Consumption Rises Like Related Posts Who is Salesforce? Who is Salesforce? Here is their story in their own words. From our inception, we’ve proudly embraced the identity of Read more Salesforce Unites Einstein Analytics with Financial CRM Salesforce has unveiled a comprehensive analytics solution tailored for wealth managers, home office professionals, and retail bankers, merging its Financial Read more AI-Driven Propensity Scores AI plays a crucial role in propensity score estimation as it can discern underlying patterns between treatments and confounding variables Read more Tectonic’s Successful Salesforce Track Record Salesforce Technology Services Integrator – Tectonic has successfully delivered Salesforce in a variety of industries including Public Sector, Hospitality, Manufacturing, Read more

Read More
October 18, 2024in Data, Enterprise, Generative AI, Google, Technology
AI Agents and Digital Transformation
18Oct

AI Agents and Digital Transformation

In the rapidly developingng world of technology, Artificial Intelligence (AI) is revolutionizing industries and reshaping how we interact with digital systems. One of the most promising advancements within AI is the development of AI agents. These intelligent entities, often powered by Large Language Models (LLMs), are driving the next wave of digital transformation by enabling automation, personalization, and enhanced decision-making across various sectors. AI Agents and digital transformation are here to stay. What is an AI Agent? An AI agent, or intelligent agent, is a software entity capable of perceiving its environment, reasoning about its actions, and autonomously working toward specific goals. These agents mimic human-like behavior using advanced algorithms, data processing, and machine-learning models to interact with users and complete tasks. LLMs to AI Agents — An Evolution The evolution of AI agents is closely tied to the rise of Large Language Models (LLMs). Models like GPT (Generative Pre-trained Transformer) have showcased remarkable abilities to understand and generate human-like text. This development has enabled AI agents to interpret complex language inputs, facilitating advanced interactions with users. Key Capabilities of LLM-Based Agents LLM-powered agents possess several key advantages: Two Major Types of LLM Agents LLM agents are classified into two main categories: Multi-Agent Systems (MAS) A Multi-Agent System (MAS) is a group of autonomous agents working together to achieve shared goals or solve complex problems. MAS applications span robotics, economics, and distributed computing, where agents interact to optimize processes. AI Agent Architecture and Key Elements AI agents generally follow a modular architecture comprising: Learning Strategies for LLM-Based Agents AI agents utilize various learning techniques, including supervised, reinforcement, and self-supervised learning, to adapt and improve their performance in dynamic environments. How Autonomous AI Agents Operate Autonomous AI agents act independently of human intervention by perceiving their surroundings, reasoning through possible actions, and making decisions autonomously to achieve set goals. AI Agents’ Transformative Power Across Industries AI agents are transforming numerous industries by automating tasks, enhancing efficiency, and providing data-driven insights. Here’s a look at some key use cases: Platforms Powering AI Agents The Benefits of AI Agents and Digital Transformation AI agents offer several advantages, including: The Future of AI Agents The potential of AI agents is immense, and as AI technology advances, we can expect more sophisticated agents capable of complex reasoning, adaptive learning, and deeper integration into everyday tasks. The future promises a world where AI agents collaborate with humans to drive innovation, enhance efficiency, and unlock new opportunities for growth in the digital age. AI Agents and Digital Transformation By partnering with AI development specialists at Tectonic, organizations can access cutting-edge solutions tailored to their needs, positioning themselves to stay ahead in the rapidly evolving AI-driven market. Agentforce Like Related Posts Who is Salesforce? Who is Salesforce? Here is their story in their own words. From our inception, we’ve proudly embraced the identity of Read more Salesforce Marketing Cloud Transactional Emails Salesforce Marketing Cloud Transactional Emails are immediate, automated, non-promotional messages crucial to business operations and customer satisfaction, such as order Read more Salesforce Unites Einstein Analytics with Financial CRM Salesforce has unveiled a comprehensive analytics solution tailored for wealth managers, home office professionals, and retail bankers, merging its Financial Read more AI-Driven Propensity Scores AI plays a crucial role in propensity score estimation as it can discern underlying patterns between treatments and confounding variables Read more

Read More
October 18, 2024in Agentforce Platform, Artificial Intelligence, Data, Enterprise, Salesforce Copilots, Technology
Latest on AI, CRM, and Data Innovations
18Oct

Latest on AI, CRM, and Data Innovations

What’s Happening at Salesforce? The Latest on AI, CRM, and Data Innovations OneMagnify and CX Today have collaborated to explore the latest advancements in AI, CRM, and data at Salesforce. The Salesforce suite is evolving rapidly, driven by the emergence of generative AI, large language models, and increasingly diverse customer demands. Discover how Salesforce is adapting to this dynamic landscape, what the future holds for the industry giant, and how business leaders can maximize the potential of the Salesforce platform. Adam MacDonald, a Salesforce Solution Engineer at OneMagnify, emphasizes, “Organizations often struggle with Salesforce implementation when they fail to align internally and address data silos as the first step in their digital transformation. Defining the solution with the end goal in mind, while allowing for quick, focused wins, is a solid strategy for securing the long-term organizational buy-in essential for successful implementation.” Like Related Posts Who is Salesforce? Who is Salesforce? Here is their story in their own words. From our inception, we’ve proudly embraced the identity of Read more Salesforce Marketing Cloud Transactional Emails Salesforce Marketing Cloud Transactional Emails are immediate, automated, non-promotional messages crucial to business operations and customer satisfaction, such as order Read more Salesforce Unites Einstein Analytics with Financial CRM Salesforce has unveiled a comprehensive analytics solution tailored for wealth managers, home office professionals, and retail bankers, merging its Financial Read more AI-Driven Propensity Scores AI plays a crucial role in propensity score estimation as it can discern underlying patterns between treatments and confounding variables Read more

Read More
October 18, 2024in Data, Generative AI, Salesforce
Recent advancements in AI
18Oct

Recent advancements in AI

Recent advancements in AI have been propelled by large language models (LLMs) containing billions to trillions of parameters. Parameters—variables used to train and fine-tune machine learning models—have played a key role in the development of generative AI. As the number of parameters grows, models like ChatGPT can generate human-like content that was unimaginable just a few years ago. Parameters are sometimes referred to as “features” or “feature counts.” While it’s tempting to equate the power of AI models with their parameter count, similar to how we think of horsepower in cars, more parameters aren’t always better. An increase in parameters can lead to additional computational overhead and even problems like overfitting. There are various ways to increase the number of parameters in AI models, but not all approaches yield the same improvements. For example, Google’s Switch Transformers scaled to trillions of parameters, but some of their smaller models outperformed them in certain use cases. Thus, other metrics should be considered when evaluating AI models. The exact relationship between parameter count and intelligence is still debated. John Blankenbaker, principal data scientist at SSA & Company, notes that larger models tend to replicate their training data more accurately, but the belief that more parameters inherently lead to greater intelligence is often wishful thinking. He points out that while these models may sound knowledgeable, they don’t actually possess true understanding. One challenge is the misunderstanding of what a parameter is. It’s not a word, feature, or unit of data but rather a component within the model‘s computation. Each parameter adjusts how the model processes inputs, much like turning a knob in a complex machine. In contrast to parameters in simpler models like linear regression, which have a clear interpretation, parameters in LLMs are opaque and offer no insight on their own. Christine Livingston, managing director at Protiviti, explains that parameters act as weights that allow flexibility in the model. However, more parameters can lead to overfitting, where the model performs well on training data but struggles with new information. Adnan Masood, chief AI architect at UST, highlights that parameters influence precision, accuracy, and data management needs. However, due to the size of LLMs, it’s impractical to focus on individual parameters. Instead, developers assess models based on their intended purpose, performance metrics, and ethical considerations. Understanding the data sources and pre-processing steps becomes critical in evaluating the model’s transparency. It’s important to differentiate between parameters, tokens, and words. A parameter is not a word; rather, it’s a value learned during training. Tokens are fragments of words, and LLMs are trained on these tokens, which are transformed into embeddings used by the model. The number of parameters influences a model’s complexity and capacity to learn. More parameters often lead to better performance, but they also increase computational demands. Larger models can be harder to train and operate, leading to slower response times and higher costs. In some cases, smaller models are preferred for domain-specific tasks because they generalize better and are easier to fine-tune. Transformer-based models like GPT-4 dwarf previous generations in parameter count. However, for edge-based applications where resources are limited, smaller models are preferred as they are more adaptable and efficient. Fine-tuning large models for specific domains remains a challenge, often requiring extensive oversight to avoid problems like overfitting. There is also growing recognition that parameter count alone is not the best way to measure a model’s performance. Alternatives like Stanford’s HELM and benchmarks such as GLUE and SuperGLUE assess models across multiple factors, including fairness, efficiency, and bias. Three trends are shaping how we think about parameters. First, AI developers are improving model performance without necessarily increasing parameters. A study of 231 models between 2012 and 2023 found that the computational power required for LLMs has halved every eight months, outpacing Moore’s Law. Second, new neural network approaches like Kolmogorov-Arnold Networks (KANs) show promise, achieving comparable results to traditional models with far fewer parameters. Lastly, agentic AI frameworks like Salesforce’s Agentforce offer a new architecture where domain-specific AI agents can outperform larger general-purpose models. As AI continues to evolve, it’s clear that while parameter count is an important consideration, it’s just one of many factors in evaluating a model’s overall capabilities. To stay on the cutting edge of artificial intelligence, contact Tectonic today. Like Related Posts Who is Salesforce? Who is Salesforce? Here is their story in their own words. From our inception, we’ve proudly embraced the identity of Read more Salesforce Marketing Cloud Transactional Emails Salesforce Marketing Cloud Transactional Emails are immediate, automated, non-promotional messages crucial to business operations and customer satisfaction, such as order Read more Salesforce Unites Einstein Analytics with Financial CRM Salesforce has unveiled a comprehensive analytics solution tailored for wealth managers, home office professionals, and retail bankers, merging its Financial Read more AI-Driven Propensity Scores AI plays a crucial role in propensity score estimation as it can discern underlying patterns between treatments and confounding variables Read more

Read More
October 18, 2024in Data, Generative AI, Google, Salesforce
Third Wave of AI at Salesforce
16Oct

Third Wave of AI at Salesforce

The Third Wave of AI at Salesforce: How Agentforce is Transforming the Landscape At Dreamforce 2024, Salesforce unveiled several exciting innovations, with Agentforce taking center stage. This insight explores the key changes and enhancements designed to improve efficiency and elevate customer interactions. Introducing Agentforce Agentforce is a customizable AI agent builder that empowers organizations to create and manage autonomous agents for various business tasks. But what exactly is an agent? An agent is akin to a chatbot but goes beyond traditional capabilities. While typical chatbots are restricted to scripted responses and predefined questions, Agentforce agents leverage large language models (LLMs) and generative AI to comprehend customer inquiries contextually. This enables them to make independent decisions, whether processing requests or resolving issues using real-time data from your company’s customer relationship management (CRM) system. The Role of Atlas At the heart of Agentforce’s functionality lies the Atlas reasoning engine, which acts as the operational brain. Unlike standard assistive tools, Atlas is an agentic system with the autonomy to act on behalf of the user. Atlas formulates a plan based on necessary actions and can adjust that plan based on evaluations or new information. When it’s time to engage, Atlas knows which business processes to activate and connects with customers or employees via their preferred channels. This sophisticated approach allows Agentforce to significantly enhance operational efficiency. By automating routine inquiries, it frees up your team to focus on more complex tasks, delivering a smoother experience for both staff and customers. Speed to Value One of Agentforce’s standout features is its emphasis on rapid implementation. Many AI projects can be resource-intensive and take months or even years to launch. However, Agentforce enables quick deployment by leveraging existing Salesforce infrastructure, allowing organizations to implement solutions rapidly and with greater control. Salesforce also offers pre-built Agentforce agents tailored to specific business needs—such as Service Agent, Sales Development Representative Agent, Sales Coach, Personal Shopper Agent, and Campaign Agent—all customizable with the Agent Builder. Agentforce for Service and Sales will be generally available starting October 25, 2024, with certain elements of the Atlas Reasoning Engine rolling out in February 2025. Pricing begins at $2 per conversation, with volume discounts available. Transforming Customer Insights with Data Cloud and Marketing Cloud Dreamforce also highlighted enhancements to Data Cloud, Salesforce’s backbone for all cloud products. The platform now supports processing unstructured data, which constitutes up to 90% of company data often overlooked by traditional reporting systems. With new capabilities for analyzing various unstructured formats—like video, audio, sales demos, customer service calls, and voicemails—businesses can derive valuable insights and make informed decisions across Customer 360. Furthermore, Data Cloud One enables organizations to connect siloed Salesforce instances effortlessly, promoting seamless data sharing through a no-code, point-and-click setup. The newly announced Marketing Cloud Advanced edition serves as the “big sister” to Marketing Cloud Growth, equipping larger marketing teams with enhanced features like Path Experiment, which tests different content strategies across channels, and Einstein Engagement Scoring for deeper insights into customer behavior. Together, these enhancements empower companies to engage customers more meaningfully and measurably across all touchpoints. Empowering the Workforce Through Education Salesforce is committed to making AI accessible for all. They recently announced free instructor-led courses and AI certifications available through 2025, aimed at equipping the Salesforce community with essential AI and data management skills. To support this initiative, Salesforce is establishing AI centers in major cities, starting with London, to provide hands-on training and resources, fostering AI expertise. They also launched a global Agentforce World Tour to promote understanding and adoption of the new capabilities introduced at Dreamforce, featuring repackaged sessions from the conference and opportunities for specialists to answer questions. The Bottom Line What does this mean for businesses? With the rollout of Agentforce, along with enhancements to Data Cloud and Marketing Cloud, organizations can operate more efficiently and connect with customers in more meaningful ways. Coupled with a focus on education through free courses and global outreach, getting on board has never been easier. If you’d like to discuss how we can help your business maximize its potential with Salesforce through data and AI, connect with us and schedule a meeting with our team. Legacy systems can create significant gaps between operations and employee needs, slowing lead processes and resulting in siloed, out-of-sync data that hampers business efficiency. Responding to inquiries within five minutes offers a 75% chance of converting leads into customers, emphasizing the need for rapid, effective marketing responses. Salesforce aims to help customers strengthen relationships, enhance productivity, and boost margins through its premier AI CRM for sales, service, marketing, and commerce, while also achieving these goals internally. Recognizing the complexity of its decade-old processes, including lead assignment across three systems and 2 million lines of custom code, Salesforce took on the role of “customer zero,” leveraging Data Cloud to create a unified view of customers known as the “Customer 360 Truth Profile.” This consolidation of disparate data laid the groundwork for enterprise-wide AI and automation, improving marketing automation and reducing lead time by 98%. As Michael Andrew, SVP of Marketing Decision Science at Salesforce, noted, this initiative enabled the company to provide high-quality leads to its sales team with enriched data and AI scoring while accelerating time to market and enhancing data quality. Embracing Customer Zero “Almost exactly a year ago, we set out with a beginner’s mind to transform our lead automation process with a solution that would send the best leads to the right sales teams within minutes of capturing their data and support us for the next decade,” said Andrew. The initial success metric was “speed to lead,” aiming to reduce the handoff time from 20 minutes to less than one minute. The focus was also on integrating customer and lead data to develop a more comprehensive 360-degree profile for each prospect, enhancing lead assignment and sales rep productivity. Another objective was to boost business agility by cutting the average time to implement assignment changes from four weeks to mere days. Accelerating Success with

Read More
October 16, 2024in Data, Enterprise, Generative AI, Google, Google Analytics, Marketing Automation, Salesforce
Zendesk Launches AI Agent Builder
15Oct

The State of AI

The State of AI: How We Got Here (and What’s Next) Artificial intelligence (AI) has evolved from the realm of science fiction into a transformative force reshaping industries and lives around the world. But how did AI develop into the technology we know today, and where is it headed next? At Dreamforce, two of Salesforce’s leading minds in AI—Chief Scientist Silvio Savarese and Chief Futurist Peter Schwartz—offered insights into AI’s past, present, and future. How We Got Here: The Evolution of AI AI’s roots trace back decades, and its journey has been defined by cycles of innovation and setbacks. Peter Schwartz, Salesforce’s Chief Futurist, shared a firsthand perspective on these developments. Having been involved in AI since the 1970s, Schwartz witnessed the first “AI winter,” a period of reduced funding and interest due to the immense challenges of understanding and replicating the human brain. In the 1990s and early 2000s, AI shifted from attempting to mimic human cognition to adopting data-driven models. This new direction opened up possibilities beyond the constraints of brain-inspired approaches. By the 2010s, neural networks re-emerged, revolutionizing AI by enabling systems to process raw data without extensive pre-processing. Savarese, who began his AI research during one of these challenging periods, emphasized the breakthroughs in neural networks and their successor, transformers. These advancements culminated in large language models (LLMs), which can now process massive datasets, generate natural language, and perform tasks ranging from creating content to developing action plans. Today, AI has progressed to a new frontier: large action models. These systems go beyond generating text, enabling AI to take actions, adapt through feedback, and refine performance autonomously. Where We Are Now: The Present State of AI The pace of AI innovation is staggering. Just a year ago, discussions centered on copilots—AI systems designed to assist humans. Now, the conversation has shifted to autonomous AI agents capable of performing complex tasks with minimal human oversight. Peter Schwartz highlighted the current uncertainties surrounding AI, particularly in regulated industries like banking and healthcare. Leaders are grappling with questions about deployment speed, regulatory hurdles, and the broader societal implications of AI. While many startups in the AI space will fail, some will emerge as the giants of the next generation. Salesforce’s own advancements, such as the Atlas Reasoning Engine, underscore the rapid progress. These technologies are shaping products like Agentforce, an AI-powered suite designed to revolutionize customer interactions and operational efficiency. What’s Next: The Future of AI According to Savarese, the future lies in autonomous AI systems, which include two categories: The Road Ahead As AI continues to evolve, it’s clear that its potential is boundless. However, the path forward will require careful navigation of ethical, regulatory, and practical challenges. The key to success lies in innovation, collaboration, and a commitment to creating systems that enhance human capabilities. For Salesforce, the journey has only just begun. With groundbreaking technologies and visionary leadership, the company is not just predicting the future of AI—it’s creating it. The State of AI. Like Related Posts Who is Salesforce? Who is Salesforce? Here is their story in their own words. From our inception, we’ve proudly embraced the identity of Read more Salesforce Marketing Cloud Transactional Emails Salesforce Marketing Cloud Transactional Emails are immediate, automated, non-promotional messages crucial to business operations and customer satisfaction, such as order Read more Salesforce Unites Einstein Analytics with Financial CRM Salesforce has unveiled a comprehensive analytics solution tailored for wealth managers, home office professionals, and retail bankers, merging its Financial Read more AI-Driven Propensity Scores AI plays a crucial role in propensity score estimation as it can discern underlying patterns between treatments and confounding variables Read more

Read More
October 15, 2024in Artificial Intelligence, Data, Salesforce, Technology
Generative AI Replaces Legacy Systems
15Oct

Generative AI Replaces Legacy Systems

Generative AI Will Overtake Legacy Stack Vendors With the rise of generative AI, legacy software vendors like Appian, IBM, Salesforce, SAP, Pegasystems, IFS, Oracle, Software AG, TIBCO, and UIPath are becoming increasingly obsolete. These vendors represent the old guard, clinging to outdated business process automation systems, while the future clearly belongs to AI-driven innovation. Back in the early 2010s, discussions around dynamic processes—self-assembling workflows created by artificial intelligence—were already gaining traction. The vision was to bypass the need for traditional process mapping or manually designing new interfaces. Instead, AI would dynamically generate processes in response to specific tasks, allowing for far greater flexibility and adaptability. However, business rules within BPMS (Business Process Management Systems) often imposed constraints that limited decision-making flexibility. Today, this vision is finally within reach. Many traditional stack vendors are scrambling to integrate generative AI into their offerings in a desperate bid to remain relevant. But the truth is, generative AI renders these vendors largely unnecessary. For instance, Pegasystems, like many others, now incorporates generative AI into its software, but users are still bound to old workflows and low-code development systems. The reliance on building processes, regardless of AI assistance, keeps them stuck in the past. Across the board—whether it’s ERP, CRM, or RPA—vendors such as Salesforce, SAP, and IFS remain tethered to their outdated systems, even though they possess all the necessary data, both structured and unstructured, to benefit from a simpler, AI-powered approach. All that’s needed is a generative AI layer on top to handle tasks like customer complaints. Consider a customer complaint scenario: traditionally, a complaint is processed through a defined workflow, often requiring the creation of expensive, custom SaaS solutions. But what if an LLM (Large Language Model) could handle this instead? The LLM could analyze the complaint, extract key information, assess urgency through sentiment analysis, and generate a custom workflow on the fly. It could even generate backend code in real-time to process refunds or update databases, all without relying on legacy front-end systems. The LLM’s ability to create and execute dynamic workflows eliminates the need for static business processes. The AI generates temporary code and UI elements to handle a specific interaction, then discards them once the task is complete. This shifts the focus away from traditional, bloated enterprise systems and towards dynamic, JIT (Just-In-Time) interactions that are tailored to each individual customer. The efficiency gains are not in cutting jobs but in eliminating the need for costly, antiquated software and lengthy digital transformation programs. Generative AI doesn’t require massive ERP or CRM implementations, and businesses can converse directly with customer data through AI, bypassing the need for complex system integrations. Master Data Management, which once consumed millions of dollars and years of effort, is now positioned to become a simple, AI-powered solution. Enterprises already have well-structured and clean data, and adding a generative AI layer could remove the need for integrating or syncing legacy systems. The era of major vendors selling AI-enhanced solutions built on top of decaying software stacks is coming to an end. The idea of using generative AI as the foundation for a new business operating system, without the need for bloated, legacy software, is increasingly appealing. With the global workflow automation market projected to grow to .4 billion by 2030, the future clearly belongs to AI-driven solutions. Like Related Posts Who is Salesforce? Who is Salesforce? Here is their story in their own words. From our inception, we’ve proudly embraced the identity of Read more Salesforce Marketing Cloud Transactional Emails Salesforce Marketing Cloud Transactional Emails are immediate, automated, non-promotional messages crucial to business operations and customer satisfaction, such as order Read more Salesforce Unites Einstein Analytics with Financial CRM Salesforce has unveiled a comprehensive analytics solution tailored for wealth managers, home office professionals, and retail bankers, merging its Financial Read more AI-Driven Propensity Scores AI plays a crucial role in propensity score estimation as it can discern underlying patterns between treatments and confounding variables Read more

Read More
October 15, 2024in Artificial Intelligence, Data, Enterprise, Generative AI, Salesforce
Benioff Excited About AI
14Oct

Benioff Excited About AI

Salesforce CEO Marc Benioff recently critiqued Microsoft for overhyping the capabilities of its Copilot AI tool, arguing that the tech giant has done a “tremendous disservice” to the industry. In a Rapid Response interview with Bob Safian, Benioff emphasized that Salesforce’s Agentforce is “what AI was meant to be.” More excited than ever, he sees Agentforce as a technology poised to transform industries in ways comparable to past cloud, mobile, and social revolutions. Benioff Excited About AI. Reflecting on Dreamforce 2024Benioff called this year’s Dreamforce the most significant yet. With 45,000 attendees and millions joining online, Agentforce took center stage, allowing users to build their own AI agents firsthand. This hands-on experience was vital, he said, to clear up misconceptions caused by overpromised AI products. Salesforce already handles trillions of AI transactions through its Einstein platform, but Benioff believes Agentforce represents a groundbreaking shift in enterprise AI. Agentforce vs. Copilot: A Clear DifferenceBenioff drew a sharp contrast between Agentforce and Microsoft’s Copilot, comparing the latter to the infamous Microsoft Clippy. According to Benioff, Copilot often fails to deliver meaningful results, creating confusion and dissatisfaction among customers. In contrast, Agentforce is set to deliver powerful outcomes by connecting customers, raising revenues, and augmenting employees. He anticipates that within a year, Salesforce will operate over a billion AI agents worldwide. Benioff Excited About AI. Benioff Excited About AI Agentforce’s Real-World ImpactSharing a story from the healthcare sector, Benioff illustrated how Agentforce has resolved over 90% of patient inquiries and scheduling needs for one large provider, enabling rapid and meaningful interactions. He foresees similar applications across media, finance, and travel, as Agentforce helps industries implement AI-driven agents with high success rates. Scheduled to go live on October 25, Agentforce is expected to be adopted by hundreds of thousands of companies. MIT IDE Annual Conference Insights: AI’s Potential and ChallengesWhile businesses explore AI’s possibilities, researchers at MIT’s Initiative on the Digital Economy (IDE) are investigating the complexities and ethical considerations of AI. At the 2024 MIT IDE Annual Conference, findings on AI’s influence on various domains were presented, with highlights including: These MIT findings highlight both the immense promise and the challenges AI presents, as companies like Salesforce aim to harness AI’s true potential while navigating ethical and practical concerns. Like Related Posts Who is Salesforce? Who is Salesforce? Here is their story in their own words. From our inception, we’ve proudly embraced the identity of Read more Salesforce Marketing Cloud Transactional Emails Salesforce Marketing Cloud Transactional Emails are immediate, automated, non-promotional messages crucial to business operations and customer satisfaction, such as order Read more Salesforce Unites Einstein Analytics with Financial CRM Salesforce has unveiled a comprehensive analytics solution tailored for wealth managers, home office professionals, and retail bankers, merging its Financial Read more AI-Driven Propensity Scores AI plays a crucial role in propensity score estimation as it can discern underlying patterns between treatments and confounding variables Read more

Read More
October 14, 2024in Agentforce Platform, Einstein 1 Platform, Generative AI, Salesforce
Impact of Generative AI
14Oct

Impact of Generative AI

Generative AI has emerged as the most dominant trend in data management and analytics, overshadowing all other technologies. This prominence began with the launch of ChatGPT by OpenAI in November 2022, which significantly advanced the capabilities of large language models (LLMs) and demonstrated the transformative potential of generative AI (GenAI) for enterprises. Generative AI’s impact is profound, particularly in making advanced business intelligence tools accessible to a broader range of employees, not just data scientists and analysts. Before the advent of GenAI, complex data management and analytics platforms required computer science skills, statistical expertise, and extensive data literacy. Generative AI has reduced these barriers, enabling more people to leverage data insights for decision-making. Another key advantage of generative AI is its ability to greatly enhance efficiency. It can automate time-consuming, repetitive tasks previously performed manually by data engineers and experts, acting as an independent agent in managing data processes. The landscape of generative AI has evolved rapidly. Following the launch of ChatGPT, a wave of competing LLMs has emerged. Initially, the transformative potential of these technologies was theoretical, but it is now becoming tangible. Companies like Google are developing tools to help customers build and deploy their own generative AI models and applications. Enterprises are increasingly moving from pilot testing to developing and implementing production models. Generative AI does not operate in isolation. Enterprises are also focusing on complementary aspects such as data quality and governance. Ensuring that the data feeding and training generative AI is reliable is crucial. Additionally, real-time data and automation are essential for making generative AI a proactive technology rather than a reactive one. Generative AI has highlighted the need for a robust data foundation. The main challenge now is ensuring that enterprise data is trusted, governed, and ready for AI applications. With the rise of multimodal data, enterprises require a unified approach to manage and govern diverse data types effectively. In addition to generative AI, other significant trends in data management and analytics include the focus on real-time data processing and automation. Integrating generative AI with real-time data streams and automated systems is expected to drive substantial business transformation. By enabling real-time insights and actions, businesses can achieve a level of operational efficiency previously unattainable. The convergence of these technologies is transforming business operations. Unified and simplified technology stacks, integrating foundational technologies, LLMs, and real-time data platforms, are essential for driving this transformation. The industry is making strides towards creating integrated solutions that support comprehensive data management and analytics. Like Related Posts Who is Salesforce? Who is Salesforce? Here is their story in their own words. From our inception, we’ve proudly embraced the identity of Read more Salesforce Unites Einstein Analytics with Financial CRM Salesforce has unveiled a comprehensive analytics solution tailored for wealth managers, home office professionals, and retail bankers, merging its Financial Read more AI-Driven Propensity Scores AI plays a crucial role in propensity score estimation as it can discern underlying patterns between treatments and confounding variables Read more Tectonic’s Successful Salesforce Track Record Salesforce Technology Services Integrator – Tectonic has successfully delivered Salesforce in a variety of industries including Public Sector, Hospitality, Manufacturing, Read more

Read More
October 14, 2024in Generative AI
Salesforce AI Introduces SFR-Judge
12Oct

Salesforce AI Introduces SFR-Judge

Salesforce AI Introduces SFR-Judge: A Family of Three Evaluation Models with 8B, 12B, and 70B Parameters, Powered by Meta Llama 3 and Mistral NeMO The rapid development of large language models (LLMs) has transformed natural language processing, making the need for accurate evaluation of these models more critical than ever. Traditional human evaluations, while effective, are time-consuming and impractical for the fast-paced evolution of AI models. Salesforce AI Introduces SFR-Judge. To address this, Salesforce AI Research has introduced SFR-Judge, a family of LLM-based judge models designed to revolutionize how AI outputs are evaluated. Built using Meta Llama 3 and Mistral NeMO, the SFR-Judge family includes models with 8 billion (8B), 12 billion (12B), and 70 billion (70B) parameters. These models are designed to handle evaluation tasks such as pairwise comparisons, single ratings, and binary classifications, streamlining the evaluation process for AI researchers. Overcoming Limitations in Traditional Judge Models Traditional LLMs used for evaluation often suffer from biases such as position bias (favoring responses based on their order) and length bias (preferring longer responses regardless of their accuracy). SFR-Judge addresses these issues by leveraging Direct Preference Optimization (DPO), a training method that enables the model to learn from both positive and negative examples, reducing bias and ensuring more consistent and accurate evaluations. Performance and Benchmarking SFR-Judge has been rigorously tested across 13 benchmarks covering three key evaluation tasks. It outperformed existing judge models, including proprietary models like GPT-4o, achieving top performance on 10 of the 13 benchmarks. Notably, on the RewardBench leaderboard, SFR-Judge achieved a 92.7% accuracy, marking a new high in LLM-based evaluation and demonstrating its potential not only as an evaluation tool but also as a reward model for reinforcement learning from human feedback (RLHF) scenarios. Innovative Training Approach The SFR-Judge models were trained using three distinct data formats: These diverse data formats allow SFR-Judge to generate well-rounded, accurate evaluations, making it a more reliable and robust tool for model assessment. Bias Mitigation and Robustness SFR-Judge was tested on EvalBiasBench, a benchmark designed to measure six types of bias. The results demonstrated significantly lower bias levels compared to competing models, along with high consistency in pairwise order comparisons. This robustness ensures that SFR-Judge’s evaluations remain stable, even when the order of responses is altered, making it a scalable and reliable alternative to human annotation. Key Takeaways: Conclusion Salesforce AI Research’s introduction of SFR-Judge represents a breakthrough in the automated evaluation of large language models. By incorporating Direct Preference Optimization and a diverse training approach, SFR-Judge sets a new standard for accuracy, bias reduction, and consistency. Its ability to provide detailed feedback and adapt to various evaluation tasks makes it a powerful tool for the AI community, streamlining the process of LLM assessment and setting the stage for future advancements in AI evaluation. Like Related Posts Who is Salesforce? Who is Salesforce? Here is their story in their own words. From our inception, we’ve proudly embraced the identity of Read more Salesforce Marketing Cloud Transactional Emails Salesforce Marketing Cloud Transactional Emails are immediate, automated, non-promotional messages crucial to business operations and customer satisfaction, such as order Read more Salesforce Unites Einstein Analytics with Financial CRM Salesforce has unveiled a comprehensive analytics solution tailored for wealth managers, home office professionals, and retail bankers, merging its Financial Read more AI-Driven Propensity Scores AI plays a crucial role in propensity score estimation as it can discern underlying patterns between treatments and confounding variables Read more

Read More
October 12, 2024in Data, Salesforce
gettectonic.com