Contrastive learning · 2026
How does an agent choose between possible actions? Explore CLM’s learned state–action matching, follow a scoring calculation, and understand a reproduced DeepSWE selection result.
Contrastive-LM project · Public documentation
START WITH THE IDEA
CLM learns to match a situation with possible actions. It encodes the state and candidates, applies separate learned projection heads, and scores how well they align. Explore the calculation with visible example scores, then see what the published agent-solution selection benchmark measures.
THE SIGNIFICANCE
A supplied action set makes the decision inspectable. CLM’s state and action representations can be computed separately, allowing repeated candidates to reuse embeddings. The encoder still does real work for fresh text.
Original architecture and caching ↗Hopper matched 31/38 (81.579%) on the published held-out DeepSWE tasks with the released tuned head, four-candidate budget and final-12-step scoring. Uniform random selection has 73.684% expected success; the available-candidate oracle reaches 89.474%. These are selection results on existing trajectories.
Hopper reproduction receipt and task selections ↗We used the unchanged evaluator’s CPU fallback and published embeddings. The documented evaluation URL was unavailable, so we used the older evaluation dataset explicitly linked by the authors. Its bytes could not be compared with the inaccessible canonical dataset. We did not regenerate solutions, retrain the head, measure GPU speed or establish MLX parity.
Dataset provenance and scope ↗Start with a duplicate-charge request and three possible actions. Edit their illustrative cosine scores, scale and temperature; inspect every resulting logit and probability.
Compare the approaches. Compare different temperatures and candidate sets on the same scores. This is an arithmetic experiment, not a model accuracy or latency comparison.
From the implementation repository after cloning:
node scripts/cli.mjs run clm --sampleSource & instructions ↗
Authored cosine similarities → scaled logits → probabilities and confidence margin. Runs entirely in your browser; no model, Ollama or API key. The recorded DeepSWE result is separate evidence.
CHOOSE THE RIGHT TOOL
Hopper’s application guidance. Compare on your own task before adopting a more complex approach.
Evaluate CLM on representative labeled cases with the matching encoder and head. Keep candidate generation and outcome verification visible.
The original repository supplies ranking and typed-question interfaces; its DeepSWE head is task-specific.
Use ordinary code, a lookup or a parser when it reliably expresses the decision.
Introduce learned scoring only when it improves your retained task evaluation.
A relative score cannot establish that the candidate set contains a suitable answer. Keep validation and action policy in application code.
Use the lab to see a confident winner even when all supplied similarities are poor.
from clm import CLMClient, Choice
# Requires the original CLM service and matching encoder.
client = CLMClient()
result = client.system_one(
state="My invoice was charged twice.",
questions={"team": Choice(
instructions="Which team should handle this?",
criteria={"billing": "Charges, invoices, refunds",
"technical": "Bugs and outages"})})
print(result.answers["team"].choice)
# Inspect and validate the response before acting.Free open-source code · GitHub ↗For actual inference, install the original contrastive-lm package and follow its encoder/server setup. Keep the encoder, pooling and head checkpoint compatible. The Python example below requires that separately running service; this website exercise runs without it.
The browser exercise starts from authored scores and teaches the arithmetic. Our recorded DeepSWE run uses a task-specific head and published embeddings; it does not establish general decision quality, MLX equivalence or GPU speed.
node scripts/cli.mjs run clm --sample
THE ORIGINAL SOURCE
Our explanation and runnable example are here. The original authors maintain the source and documentation linked below.
Contrastive Language Models · original source ↗Both expose bounded typed decisions. CLM publishes a scoring implementation; our Jev page explains its documented API. This is an editorial comparison, not a claim of shared model internals.
A candidate scorer can inform an agent’s next step; the harness still owns execution and checks. This is an architectural connection, not a documented dependency.
Primary architecture, training description, setup, authorship and usage examples.
README; inference source · Source documentationThe authors’ research narrative. This page does not mirror its contents or claim a paper publication.
Original research linked by the repository · Source documentationPublished model, split and benchmark result used in our reproduction.
verification.json; heldout_tasks.json; best_head.pt · Source documentationOlder evaluation source linked by the official training dataset; used after the canonical evaluation URL failed.
Evaluation snapshot e57c9483 · Source documentationDocuments the link to the older evaluation source. We evaluated with evaluation embeddings, not training data.
Dataset card · Source documentationDocumentation reviewed 2026-09-27. CLM v0.1; source bb42c6c5; reviewed September 27, 2026.
Original source ↗ · Documentation ↗
Explanations, diagrams and educational code by Hopper Labs. Code: MIT. Original publications and documentation remain with their authors. These are independent educational examples, not endorsements. Any recorded evaluation is described separately with its exact scope.