Small Language Models Explained

September 30, 2024in Salesforce

Exploring Small Language Models (SLMs): Capabilities and Applications

Thank you for reading this post, don't forget to subscribe!

Large Language Models (LLMs) have been prominent in AI for some time, but Small Language Models (SLMs) are now enhancing our ability to work with natural and programming languages. While LLMs excel in general language understanding, certain applications require more accuracy and domain-specific knowledge than these models can provide. This has created a demand for custom SLMs that offer LLM-like performance while reducing runtime costs and providing a secure, manageable environment.

In this insight, we dig down into the world of SLMs, exploring their unique characteristics, benefits, and applications. We also discuss fine-tuning methods applied to Llama-2–13b, an SLM, to address specific challenges. The goal is to investigate how to make the fine-tuning process platform-independent. We selected Databricks for this purpose due to its compatibility with major cloud providers like Azure, Amazon Web Services (AWS), and Google Cloud Platform.

What Are Small Language Models?

In AI and natural language processing, SLMs are lightweight generative models with a focus on specific tasks. The term “small” refers to:

The size of the model’s neural network,
The number of parameters, and
The volume of training data.

SLMs like Google Gemini Nano, Microsoft’s Orca-2–7b, and Meta’s Llama-2–13b run efficiently on a single GPU and include over 5 billion parameters.

SLMs vs. LLMs

Size and Training: LLMs, such as ChatGPT, are larger and trained on extensive datasets, enabling them to handle complex natural language tasks with high accuracy. In contrast, SLMs are smaller and trained on more focused datasets, excelling in specific domains without the extensive scope of LLMs.
Natural Language Understanding: LLMs are adept at capturing intricate patterns in language, making them ideal for complex reasoning. SLMs, while more limited in their language scope, can be highly effective when used in appropriate contexts.
Resource Consumption: Training LLMs is resource-intensive, requiring significant computational power. SLMs are more cost-effective, needing less computational power and memory, making them suitable for on-premises and on-device deployments.
Bias and Efficiency: SLMs generally exhibit less bias due to their narrower training focus. They also offer faster inference times on local machines compared to LLMs, which may slow down with high user loads.

Applications of SLMs

SLMs are increasingly used across various sectors, including healthcare, technology, and beyond. Common applications include:

Text summarization
Text generation
Sentiment analysis
Chatbots
Named entity recognition
Spelling correction
Machine translation
Code generation

Fine-Tuning Small Language Models

Fine-tuning involves additional training of a pre-trained model to make it more domain-specific. This process updates the model’s parameters with new data to enhance its performance in targeted applications, such as text generation or question answering.

Hardware Requirements for Fine-Tuning

The hardware needs depend on the model size, project scale, and dataset. General recommendations include:

GPUs (potentially cloud-based)
Fast and reliable internet for data transfer
Powerful multi-core CPUs for data processing
Ample memory and storage

Data Preparation

Preparing data involves extracting text from PDFs, cleaning it, generating question-and-answer pairs, and then fine-tuning the model. Although GPT-3.5 was used for generating Q&A pairs, SLMs can also be utilized for this purpose based on the use case.

Fine-Tuning Process

You can use HuggingFace tools for fine-tuning Llama-2–13b-chat-hf. The dataset was converted into a HuggingFace-compatible format, and quantization techniques were applied to optimize performance. The fine-tuning lasted about 16 hours over 50 epochs, with the cost around $100/£83, excluding trial costs.

Results and Observations

The fine-tuned model demonstrated strong performance, with over 70% of answers being highly similar to those generated by GPT-3.5. The SLM achieved comparable results despite having fewer parameters. The process was successful on both AWS and Databricks platforms, showcasing the model’s adaptability.

SLMs have some limitations compared to LLMs, such as higher operational costs and restricted knowledge bases. However, they offer benefits in efficiency, versatility, and environmental impact. As SLMs continue to evolve, their relevance and popularity are likely to increase, especially with new models like Gemini Nano and Mixtral entering the market.