DDP

Large Language Models. How much do you know about them? Take the LLM Knowledge Test to find out. Question 1Do you need to have a vector store for all your text-based LLM use cases? A. Yes B. No Correct Answer: B ExplanationA vector store is used to store the vector representation of a word or sentence. These vector representations capture the semantic meaning of the words or sentences and are used in various NLP tasks. However, not all text-based LLM use cases require a vector store. Some tasks, such as summarization, sentiment analysis, and translation, do not need context augmentation. Here is why: Question 2Which technique helps mitigate bias in prompt-based learning? A. Fine-tuning B. Data augmentation C. Prompt calibration D. Gradient clipping Correct Answer: C ExplanationPrompt calibration involves adjusting prompts to minimize bias in the generated outputs. Fine-tuning modifies the model itself, while data augmentation expands the training data. Gradient clipping prevents exploding gradients during training. Question 3Which of the following is NOT a technique specifically used for aligning Large Language Models (LLMs) with human values and preferences? A. RLHF B. Direct Preference Optimization C. Data Augmentation Correct Answer: C ExplanationData Augmentation is a general machine learning technique that involves expanding the training data with variations or modifications of existing data. While it can indirectly impact LLM alignment by influencing the model’s learning patterns, it’s not specifically designed for human value alignment. Incorrect Options: A) Reinforcement Learning from Human Feedback (RLHF) is a technique where human feedback is used to refine the LLM’s reward function, guiding it towards generating outputs that align with human preferences. B) Direct Preference Optimization (DPO) is another technique that directly compares different LLM outputs based on human preferences to guide the learning process. Question 4In Reinforcement Learning from Human Feedback (RLHF), what describes “reward hacking”? A. Optimizes for desired behavior B. Exploits reward function Correct Answer: B ExplanationReward hacking refers to a situation in RLHF where the agent discovers unintended loopholes or biases in the reward function to achieve high rewards without actually following the desired behavior. The agent essentially “games the system” to maximize its reward metric. Why Option A is Incorrect:While optimizing for the desired behavior is the intended outcome of RLHF, it doesn’t represent reward hacking. Option A describes a successful training process. In reward hacking, the agent deviates from the desired behavior and finds an unintended way to maximize the reward. Question 5Fine-tuning GenAI model for a task (e.g., Creative writing), which factor significantly impacts the model’s ability to adapt to the target task? A. Size of fine-tuning dataset B. Pre-trained model architecture Correct Answer: B ExplanationThe architecture of the pre-trained model acts as the foundation for fine-tuning. A complex and versatile architecture like those used in large models (e.g., GPT-3) allows for greater adaptation to diverse tasks. The size of the fine-tuning dataset plays a role, but it’s secondary. A well-architected pre-trained model can learn from a relatively small dataset and generalize effectively to the target task. Why A is Incorrect:While the size of the fine-tuning dataset can enhance performance, it’s not the most crucial factor. Even a massive dataset cannot compensate for limitations in the pre-trained model’s architecture. A well-designed pre-trained model can extract relevant patterns from a smaller dataset and outperform a less sophisticated model with a larger dataset. Question 6What does the self-attention mechanism in transformer architecture allow the model to do? A. Weigh word importance B. Predict next word C. Automatic summarization Correct Answer: A ExplanationThe self-attention mechanism in transformers acts as a spotlight, illuminating the relative importance of words within a sentence. In essence, self-attention allows transformers to dynamically adjust the focus based on the current word being processed. Words with higher similarity scores contribute more significantly, leading to a richer understanding of word importance and sentence structure. This empowers transformers for various NLP tasks that heavily rely on context-aware analysis. Incorrect Options: Question 7What is one advantage of using subword algorithms like BPE or WordPiece in Large Language Models (LLMs)? A. Limit vocabulary size B. Reduce amount of training data C. Make computationally efficient Correct Answer: A ExplanationLLMs deal with massive amounts of text, leading to a very large vocabulary if you consider every single word. Subword algorithms like Byte Pair Encoding (BPE) and WordPiece break down words into smaller meaningful units (subwords) which are then used as the vocabulary. This significantly reduces the vocabulary size while still capturing the meaning of most words, making the model more efficient to train and use. Incorrect Answer Explanations: Question 8Compared to Softmax, how does Adaptive Softmax speed up large language models? A. Sparse word reps B. Zipf’s law exploit C. Pre-trained embedding Correct Answer: B ExplanationStandard Softmax struggles with vast vocabularies, requiring expensive calculations for every word. Imagine a large language model predicting the next word in a sentence. Softmax multiplies massive matrices for each word in the vocabulary, leading to billions of operations! Adaptive Softmax leverages Zipf’s law (common words are frequent, rare words are infrequent) to group words by frequency. Frequent words get precise calculations in smaller groups, while rare words are grouped together for more efficient computations. This significantly reduces the cost of training large language models. Incorrect Answer Explanations: Question 9Which configuration parameter for inference can be adjusted to either increase or decrease randomness within the model output layer? A. Max new tokens B. Top-k sampling C. Temperature Correct Answer: C ExplanationDuring text generation, large language models (LLMs) rely on a softmax layer to assign probabilities to potential next words. Temperature acts as a key parameter influencing the randomness of these probability distributions. Why other options are incorrect: Question 10What transformer model uses masking & bi-directional context for masked token prediction? A. Autoencoder B. Autoregressive C. Sequence-to-sequence Correct Answer: A ExplanationAutoencoder models are pre-trained using masked language modeling. They use randomly masked tokens in the input sequence, and the pretraining objective is to predict the masked tokens to reconstruct the original sentence. Question 11What technique allows you to scale model