Capability Transfer Is a Multi-Objective Engineering Problem

A specialist model can be an attractive shortcut: if it already performs the behavior you need, why not move its learned differences into a stronger general model? The difficulty is that a checkpoint is not a neatly labeled collection of skills. Its weights encode useful behavior, regressions, and interactions together. Capability transfer is therefore less like installing a feature and more like choosing an operating point in a multi-objective system.

The Hunmin-CUA project makes that tension unusually visible. Its authors started from a large mixture-of-experts vision-language model and used a computer-use checkpoint as a transfer source. Instead of applying the entire weight difference, they selected module groups, truncated their deltas with singular value decomposition, and retained base weights elsewhere. Their released model improved agent results while remaining close to the base on the Korean evaluations they tracked. It did not fully match the specialist source on long-horizon execution.

Define preservation before optimizing

The most reusable decision happened before any particular merge recipe: preservation was treated as a constraint. Korean knowledge, GUI grounding, and agent execution were evaluated separately rather than collapsed into one headline score. That matters because specialization can shift capabilities that the target benchmark never exercises.

A practical project should write a capability contract before training. It can be short:

• one or two target metrics that should improve;
• several preservation metrics covering important existing behavior;
• tolerances for acceptable regression;
• deployment conditions shared by every candidate;
• a stopping rule chosen before noisy results arrive.

The contract turns checkpoint selection into an explicit trade-off. Without it, teams tend to keep the model with the largest target gain and discover collateral damage only after integration. Averages can also conceal unacceptable failures: language quality, visual localization, tool syntax, and task completion should remain separate when they represent different user needs.

Treat weight-space knobs as interventions

Low-rank delta transfer sounds precise, but rank is not a universal measure of capability. A retained fraction of weight-space energy says how much of a matrix difference remains under a mathematical norm. It does not say that the same fraction of a behavior was transferred. Different modules, ranks, and scaling choices alter the network jointly.

That suggests a disciplined experiment matrix. Start with the base, the full specialist, and the proposed transfer under one harness. Then vary only one transfer decision at a time: module inclusion, rank, or scale. For every candidate, record target and preservation metrics together. A smaller intervention that satisfies the capability contract can be preferable to a larger one with a slightly higher headline score.

Module exclusions also require careful language. Leaving embeddings or routers unchanged limits the intervention, but it does not prove those exclusions caused preservation. Hidden states entering an unchanged router can still change its decisions. Causal claims need ablations in which each exclusion is independently restored or removed. Until then, the honest result is an association: a particular transfer configuration and a particular evaluation outcome.

Make evaluation configuration part of the result

Computer-use evaluation is especially sensitive to its environment. Pixel budget, screenshot history, prompts, reasoning mode, action parsing, task partition, and turn limits can all change a score. Consequently, a number without its configuration is not a portable model property.

A useful comparison sheet should place the following beside every result:

1. exact task set and denominator;
2. image and context limits;
3. decoding and reasoning settings;
4. step cap and malformed-action policy;
5. number of independent runs;
6. dispersion for repeated measurements.

Single-run agent results deserve particular restraint. They can identify a promising direction, but small checkpoint differences may be indistinguishable from run-to-run variation. Fixed selection rules reduce the temptation to choose a checkpoint after seeing a noisy sequence. Repeated grounding or language evaluations are valuable, but their precision should not be projected onto a different benchmark that was run once.

Verify the artifact, not just the recipe

Training success and deployment success are separate claims. Quantization, adapter merging, serialization, and serving paths can change or obscure a small update. A merge command completing and the model loading only establish artifact integrity; they do not establish behavioral preservation.

The release checklist should therefore rerun a compact behavioral suite on the exact artifact and serving path intended for use. Compare the runtime adapter, merged checkpoint, and quantized form when those are plausible deployment options. If their outputs differ, investigate before attributing the change to a single mechanism: quantization effects, task-level offsets, and evaluation noise can produce similar aggregate symptoms.

A better definition of success

The strongest lesson from selective capability transfer is methodological. Success is not copying the maximum amount of a specialist checkpoint. It is finding the smallest well-supported intervention that moves the target behavior while respecting an explicit preservation budget.

That framing improves both engineering and reporting. It encourages controlled comparisons, prevents unrelated benchmarks from being mixed, and makes limitations legible. Most importantly, it replaces the vague promise of “adding a skill” with a testable claim: under a documented configuration, this artifact improves these tasks, stays within these regression limits, and stops here for reasons chosen in advance.