Self-hosted LLMs (Llama, Mistral) in Prompt Engineering / GenAI - Practice Problems & Coding Challenges

Challenge - 5 Problems

🧠 Conceptual

intermediate

Understanding Model Size Impact on Self-hosted LLMs

Which of the following best explains how increasing the number of parameters in a self-hosted LLM like Llama or Mistral affects its performance and resource requirements?

A Increasing parameters reduces accuracy but speeds up inference and lowers memory use.

B Increasing parameters decreases both accuracy and resource requirements.

C Increasing parameters has no effect on accuracy but increases training time only.

D Increasing parameters improves model accuracy but requires more memory and slower inference.

💡 Hint

Think about how bigger models usually behave in terms of accuracy and hardware needs.

Larger models generally capture more complex patterns, improving accuracy, but they need more memory and compute, slowing down inference.

❓ Predict Output

intermediate

Output of Token Generation with Temperature in LLM

Given the following pseudocode for generating tokens from a self-hosted LLM with temperature=0.0, what is the expected behavior of the output tokens?

Prompt Engineering / GenAI

tokens = model.generate(input_ids, temperature=0.0, max_length=5)
print(tokens)

A The model outputs tokens only from a fixed vocabulary subset.

B The model outputs the most likely tokens deterministically.

C The model outputs random tokens with equal probability.

D The model outputs tokens with high randomness and diversity.

💡 Hint

Temperature controls randomness in token selection.

A temperature of 0.0 means the model picks the highest probability token every time, making output deterministic.

❓ Model Choice

advanced

Choosing a Self-hosted LLM for Low-latency Applications

You want to deploy a self-hosted LLM for a chatbot that requires fast responses on limited hardware. Which model choice is best?

A A smaller Mistral 7B model quantized to 4-bit precision.

B A large Llama 70B model with full precision weights.

C A large Mistral 30B model with no quantization.

D A medium Llama 13B model with float32 precision.

💡 Hint

Consider model size and quantization effects on speed and memory.

Smaller models with quantization reduce memory and speed up inference, suitable for low-latency on limited hardware.

❓ Metrics

advanced

Evaluating Self-hosted LLM Output Quality

Which metric is most appropriate to evaluate the quality of text generated by a self-hosted LLM like Llama or Mistral on a language generation task?

A Perplexity measuring how well the model predicts the next token.

B Accuracy measuring exact token matches with ground truth.

C Mean Squared Error between predicted and actual token embeddings.

D F1 score measuring classification correctness.

💡 Hint

Think about metrics used in language modeling tasks.

Perplexity measures how surprised the model is by the next token, indicating prediction quality in language generation.

🔧 Debug

expert

Debugging Memory Error in Self-hosted LLM Inference

You try to run inference on a Llama 13B model but get a CUDA out-of-memory error. Which action will most likely fix this issue?

A Add more layers to the model to distribute memory load.

B Increase learning rate to speed up training and reduce memory.

C Reduce batch size or use model quantization to lower memory use.

D Disable GPU and run inference on CPU only.

💡 Hint

Think about how to reduce GPU memory usage during inference.

Reducing batch size or quantizing the model reduces memory needed, preventing out-of-memory errors during inference.