This exercise started with a narrow goal: understand the different types of models and what changes when I use a generative model, an embedding model, or a task-specific head. I'm using small models so the mechanics are visible and experiments are easy to repeat.
Status: in progress · Code: AI scientist lab on GitHub
1. Generation: the same model, different interfaces
I started with Qwen3-0.6B, loading the model with Hugging Face's high-level pipeline and with AutoModelForCausalLM. Both use a causal language model; the difference is how much tokenization, generation configuration, and output processing I manage directly.
At this stage I wanted to understand the inference interface, not benchmark the model. Greedy decoding produced repetitive outputs in some examples; trying sampling made the difference between model weights and decoding policy concrete.
from transformers import AutoTokenizer, AutoModelForCausalLM
model_id = "Qwen/Qwen3-0.6B"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id)
inputs = tokenizer("Explain embeddings in one sentence.", return_tensors="pt")
outputs = model.generate(**inputs, max_new_tokens=80, do_sample=False)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))
Compare deterministic and sampled generation with the same prompt. Do not assume that a more fluent sampled response is more correct.
2. Embeddings: vectors instead of generated text
An embedding model maps input text into vectors. Unlike a causal LM, its typical output is a representation for downstream tasks, such as similarity, retrieval, or classification.
from sentence_transformers import SentenceTransformer
from sentence_transformers.util import cos_sim
encoder = SentenceTransformer("Qwen/Qwen3-Embedding-0.6B")
sentences = ["The patient has lung cancer.", "A lung tumor was diagnosed."]
embeddings = encoder.encode(sentences, convert_to_tensor=True)
similarities = cos_sim(embeddings, embeddings)
print(embeddings.shape)
print(similarities)
The diagonal of the cosine similarity matrix should be close to one for nonzero embeddings because each sentence is being compared to itself. Off-diagonal values capture similarity in the model's learned representation; they are not calibrated probabilities.
3. Embeddings plus a learned classification head
I then tried a small supervised head on top of fixed embeddings. This separates the representation learned by the embedding model from a downstream classifier that learns a decision boundary. A training loss curve can show whether the head is fitting the examples, but a low training loss on six examples is not evidence of generalization.
# Sketch of the downstream pattern:
# embeddings -> linear layer -> logits -> cross-entropy loss
# Plot loss by training step and evaluate on held-out examples.
What this clarified
A generative model predicts tokens. An embedding model produces representations intended for downstream comparison. A classifier head can turn those representations into task-specific predictions. The inference interface, model objective, and evaluation criterion each matter — and they shouldn't be conflated.
Next experiments: run held-out evaluation for the classifier, inspect embedding normalization, and compare model interfaces with consistent prompts and decoding settings.