Reasoning as Value-Guided Latent-Space Optimization

Suppose a language model answers a math problem incorrectly. We could ask it again, sample a hundred solutions, grow a search tree, or update its weights. All of these responses spend more computation, but none of them questions where the thinking happens: they search harder in the same discrete space, or they change the model itself.

There is another option: leave the weights untouched and optimize the internal states of this one problem. The model does not try another sentence. It adjusts the continuous vectors from which a sentence will be generated, a gradient step at a time, guided by a value signal—correctness, model confidence, image quality—until it stops improving. When the optimization ends, the states are discarded; the next problem starts from the original frozen model.

The methods that do this—collectively, test-time latent reasoning—differ in detail, but they share a philosophical hypothesis large enough to organize this entire article:

A great latent space is one where following the gradient of value is enough to solve complex problems.

The sentence makes a precise claim about where the difficulty of reasoning should reside. The optimizers in this literature are deliberately simple—REINFORCE-style estimators, a handful of update steps, in some cases zeroth-order perturbations—so the burden of performance falls on two components. The first is the latent space: the representation in which optimization takes place. It determines what a single update can express, how far credit can propagate from a sequence-level outcome back to the states that produced it, and whether better solutions lie within reach of a short optimization trajectory. The second is value guidance: the signal—a verifier, a self-reward, a confidence estimate—that defines the objective and orients each step. When such minimal machinery improves reasoning, the improvement must be attributed to these two components rather than to the sophistication of the search itself. The research program, accordingly, is not the design of stronger optimizers; it is the identification of latent spaces, and of value functions over them, under which simple optimization suffices.

Table of Contents

• Token space is a poor place for gradients
• One objective, three ingredients
• LatentSeek: value guidance at the boundary of language
• GradCuit: carrying value deeper into the network
• The emerging family: a search for the right space and the right value
• Conclusion
• Citation

Token space is a poor place for gradients

Chain-of-thought turned language into a computational scratchpad (Wei et al., 2022). As an interface it is remarkably useful: reasoning steps can be inspected, edited, verified, and fed back to the model. But consider it as a space for value-guided optimization. Token space is discrete: there are no directions, only alternatives, so a value can rank candidates but never point. At every step, a high-dimensional hidden state must pass through a vocabulary-sized distribution and collapse to a single token; once the token is sampled, most information in that distribution is gone. To improve in this space is to resample and vote, as self-consistency does over complete chains (Wang et al., 2023), or to score and branch over partial texts, as Tree of Thoughts does (Yao et al., 2023). This is guidance without gradients: every step costs a rollout, and the value’s advice arrives only after the fact.

The loss matters most when a model is genuinely uncertain among several useful continuations. A latent vector can, in principle, carry aspects of many alternatives at once; a token must commit to one. Natural language also spends capacity on grammar, connective tissue, and exposition—valuable for communication, incidental to computation. If the hypothesis is right—if the right space lets a simple, value-guided step do the work—then the natural project is to relocate reasoning into a space that is continuous, information-rich, and reachable by gradients. The literature has approached that relocation in stages.

• Learn a continuous scratchpad. Implicit-CoT methods gradually hide textual steps during training (Deng et al., 2024). Coconut feeds the last hidden state back as the next input embedding instead of decoding it to a token (Hao et al., 2025). Latent Thought Models go a step further and infer per-instance latent vectors at inference time through variational updates (Kong et al., 2025)—latent computation, but driven by likelihood rather than by a value placed on the answer. All of these build the space by changing how the model is trained.

• Relax the vocabulary. Soft Thinking passes a probability-weighted mixture of token embeddings forward, preserving uncertainty across several concepts without additional training (Zhang et al., 2025). The space becomes continuous but remains tied to the vocabulary simplex.

• Optimize activations directly. PPLM established an early precedent by differentiating an attribute objective into a frozen language model’s activations (Dathathri et al., 2020). Soft and prefix tuning optimize continuous prompts, though ordinarily as reusable artifacts learned from a dataset (Li & Liang, 2021).

Test-time latent reasoning takes the final step: an ephemeral state, optimized for one input, under an explicit value available at inference time. “Latent reasoning” is used loosely in the literature—for learned recurrent thoughts, soft token mixtures, ordinary implicit computation in hidden states, or explicitly optimized activations. Here we mean the last: hidden states as decision variables, value as altitude.

One objective, three ingredients

Let $c$ be a problem, $x=(x_1,\ldots,x_T)$ a generated reasoning trajectory, and $\pi_\theta$ a frozen autoregressive model. Ordinary generation samples

$$ \pi_\theta(x\mid c)=\prod_{t=1}^{T}\pi_\theta(x_t\mid x_{< t},c). $$

Test-time reasoning introduces a value $R(x,c)$ and spends an inference budget searching for a high-scoring trajectory (Snell et al., 2025). Best-of-$N$ searches by drawing $N$ leaves. Tree search explores prefixes. Test-time latent reasoning introduces continuous, instance-specific variables $z$ and instead solves

$$ z^* = \arg\max_z\; \mathbb{E}_{x\sim\pi_\theta(\cdot\mid z,c)}[R(x,c)]. $$

This is the thesis in symbols: the only search direction the family allows itself is the policy gradient of value with respect to $z$.

The parameters $\theta$ never change, and the update rule—as we will see—stays almost embarrassingly simple across the whole family. That is the point. The objective divides the labor among three ingredients: an optimizer, kept deliberately plain; a value, which supplies direction; and a space, which decides whether that direction can be followed. The interesting questions all concern the second and third:

1. Where in the network does $z$ live?
2. How does it affect the generated trajectory?
3. How does a sequence-level value assign credit back to it?
4. What prevents optimization from leaving the model’s familiar activation manifold?

The methods below are best read as successive answers—each choosing a different space, each guided by a different value, and each discovering how much those two choices alone decide.

LatentSeek: value guidance at the boundary of language

For each position $t$, a Transformer produces a final hidden state $z_t$ immediately before the language-model head. This is the outermost latent space available—one linear map away from the vocabulary. LatentSeek (Li et al., 2025) cuts the computational graph at exactly this boundary and treats a prefix of final hidden states as independent, optimizable variables.

The search starts from a sensible initialization: an ordinary CoT rollout. If that rollout contains $T$ tokens, the method keeps roughly the first $N=\rho T$ hidden states, where $\rho$ is a fractional optimization ratio. These states are decoded independently through the frozen LM head:

$$ \pi(x\mid z,c) = \underbrace{\prod_{t=1}^{N}\pi_{\text{head}}(x_t\mid z_t)}_{\text{decode optimized latents}} \underbrace{\prod_{t=N+1}^{T}\pi_\theta(x_t\mid x_{< t},c)}_{\text{continue autoregressively}}. $$

After the latent prefix is converted into tokens, the model finishes the response in the usual way. The value is the model’s own judgment: a self-reward prompt asks the same model to score the solution. The update is as plain as promised—a REINFORCE-style step (Williams, 1992):

$$ z_t \leftarrow z_t + \eta\, \mathbb{E}_{x\sim\pi(\cdot\mid z,c)} \left[R(x,c)\nabla_{z_t}\log \pi_{\text{head}}(x_t\mid z_t)\right]. $$

Note what the simplicity buys: the search variable is continuous, but the value can be arbitrary. It need not be differentiable. Value-weighted log-probability gradients tell each $z_t$ how to make its sampled token more or less likely.

But every space has a geometry, and this one’s limits how far guidance can travel. Treating the latents as independent prevents the first latent from monopolizing the update through the autoregressive chain and expands the effective search surface. The price is that the gradient reaching $z_t$ describes only its influence on the token decoded at the same position. Its influence on later reasoning must pass through a discrete sample. A terminal value can say that the trajectory was good; it cannot say which early latent altered which later inference. At the boundary of language, guidance keeps being interrupted by tokens.

Across GSM8K, MATH-500, and AIME 2024, LatentSeek reports consistent gains over CoT and several sampling or reflection baselines, with improvements that continue as the allowed number of latent updates grows. One result dissects the thesis especially cleanly: with a perfect answer verifier in place of self-reward, the average gain over CoT becomes much larger. Same space, same optimizer, better value—much better outcome. The space contains better solutions than the practical value can reliably identify; the terrain outruns the guide.

The qualitative cases probe from another side. Some optimized prefixes become grammatically broken or semantically opaque while still leading to the correct answer. This may indicate that the model has useful computational paths that do not resemble explanations humans would write. It may also indicate value exploitation, accidental shortcutting, or off-manifold activation. Correctness alone cannot distinguish these stories.

QUESTION: Two trains leave San Rafael at the same time [...] What’s the distance covered by each train in the two days?
ANSWER: 230
CoT: To solve this problem, we need to break it down into steps. [...] So, the distance covered by each train in the two days is 197.5 miles. \boxed{197.5}
LatentSeek: Let find this„ let’ll more understand it down step two andLet 1: BothThe the trains are same route west both first time on80ward) on same and and can consider they the travel travel same distance of So’s denote the common xd’ they both 80 miles on x have write up an following: [...] Since both trains travel the same distance in each direction, the distance covered by each train is 230 miles. The final answer is: \boxed{230}

That is the productive ambiguity in the phrase “seek in the dark”: outside language the search is least constrained—and language was our main window into what the model was doing.

GradCuit: carrying value deeper into the network

If final hidden states form a flawed space because guidance cannot travel through it, the remedy is to choose a space where it can. GradCuit (Yu et al., 2026) relocates the latent variables to the network’s interior. Choose an intermediate layer $\ell$ in an $M$-layer decoder. Run the prompt and previously generated tokens through layers $1{:}\ell$, insert $N$ optimizable latent states, and pass the concatenated sequence through layers $\ell{+}1{:}M$:

$$ \pi(x_t\mid x_{< t},z^{(\ell)},c) = \operatorname{LMHead}\!\left( \operatorname{Transformer}_{\ell+1:M} \left[h_c^{(\ell)},z^{(\ell)},h_{x_{< t}}^{(\ell)}\right] \right). $$

Because the inserted states precede the continuation, causal self-attention lets every later token attend to every latent. The full trajectory now factorizes as

$$ \pi(x\mid z^{(\ell)},c) =\prod_{t=1}^{T}\pi(x_t\mid x_{< t},z^{(\ell)},c), $$

and the gradient for latent $z_i^{(\ell)}$ aggregates direct contributions from the entire continuation:

$$ \nabla_{z_i^{(\ell)}}J = \sum_{t=1}^{T} \mathbb{E}\left[ R(x,c)\, \nabla_{z_i^{(\ell)}} \log\pi(x_t\mid x_{< t},z^{(\ell)},c) \right]. $$

The value is still sequence-level. The estimator is still policy-gradient-like. Nothing about the optimizer got smarter. What changed is how far guidance can reach: a later token can now assign credit directly to an earlier latent through the remaining attention blocks, with no latent-to-token-to-latent handoff before the continuation can use the optimized state. This is why “circuit” is more than branding—the same self-attention graph serves as a forward computational route and a backward credit-assignment route. The space was chosen so that value could flow.

The numbers behave the way the hypothesis predicts. Across five instruction-tuned backbones, three benchmarks, and two answer formats, GradCuit reports 64.5% average accuracy: 6.6 percentage points above standard CoT and 2.4 points above the strongest enhanced-reasoning baseline in the study. Across seven learning rates, it reduces the standard deviation of accuracy from 1.53 for LatentSeek to 0.82. A space that transmits guidance well is also more forgiving to optimize in.

One ablation reads almost as a controlled experiment on the thesis itself. Replace the value gradient with a Gaussian random direction, and this unguided variant remains competitive with LatentSeek; restore the value, and guided optimization adds another 2.4 points on average. Taken as a decomposition of the thesis’s two nouns: the space does much of the work before any guidance arrives, and value guidance completes it. The ablation also keeps us honest—we cannot attribute every gain to precise credit assignment when some of it comes from changing the space itself.

Where does the guidance land? Measuring, for each continuation token, the norm of its gradient with respect to all optimized latents, GradCuit finds that “because,” “therefore,” “then,” and similar reasoning connectors receive the strongest gradients across GPQA-Diamond, GSM8K, and MATH-500. One interpretation is that optimized latents primarily steer how the model moves between reasoning steps rather than rewriting all content uniformly. This is a first-order sensitivity result, not a complete mechanistic explanation, but it sketches where value enters the trace.

A note on the Jacobian Lens

Why should the middle of the network be the great space? Interpretability offers an independent—and concurrent—answer. The Jacobian Lens (Gurnee et al., 2026), developed in parallel with GradCuit, refines the classic logit lens by asking which directions of an intermediate residual stream are disposed, once transformed by all later layers, to surface as output tokens; Anthropic calls this subspace J-space and finds it behaves like a limited-capacity global workspace—silent intermediate results, planned words, causally swappable concepts—concentrated in a middle band of layers. The convergence is striking for work that emerged in parallel: both judge a hidden state by its downstream influence rather than its decodability in place, both rely on Jacobian structure through the later blocks (corpus-averaged for the lens, instance-specific for GradCuit), and both point to the middle of the network, where a representation is formed enough to matter yet still has layers left to act through. The instruments differ—the lens is a reusable, token-indexed readout, GradCuit a per-instance optimizer whose updates need not remain verbalizable—but that is the division of labor: one reads the workspace; the other steers it.

The emerging family: a search for the right space and the right value

Seen through this hypothesis, the branching family of methods is a systematic exploration of spaces and values—differing in where the state is inserted and what signal guides it.

Method	Optimized object	Feedback	Key distinction
LatentSeek	Final-layer states for an initial output prefix	Self-reward or verifier	Decodes optimized states before continuation
LTPO	Input-side latent thought vectors	Model confidence	Uses Gaussian perturbations and zeroth-order policy gradients (Ye et al., 2026)
GradCuit	Prefix states at a selected intermediate layer	Self-reward	Assigns gradients from the full continuation through attention
MILR	Joint text and image output-side states	Image-quality critic	Extends latent search to unified multimodal generation (Mi et al., 2026)
DMLR	Latent think tokens plus selected visual features	Confidence	Interleaves latent optimization with dynamic visual retrieval (Liu et al., 2026)

The table’s first message is that “latent” is not one location—and that locations are not equally great. Input embeddings are easy to inject but far from the output objective. Final-layer states are close to the vocabulary but have little downstream network left to transform them. Intermediate states retain both contextual meaning and computational runway. GradCuit’s empirical preference for early-to-middle layers—roughly 25% to 50% depth, with task dependence—is therefore plausible: the representation is formed enough to optimize, yet enough layers remain to absorb and refine the intervention.

The second message sits in the feedback column: self-reward, confidence, a learned critic. Every practical value is cheaper—and weaker—than a true verifier. The family is exploring both nouns of the thesis at once, and its progress will be bounded by whichever lags.

Interpretability sees the same trade-off over depth. Early states may not yet expose the relevant concept; late states may only encode the answer already chosen. The middle is where a thought can be both abstract and consequential—which may be exactly what makes it steerable by simple gradients.

Conclusion

Return, finally, to the sentence this article has been circling: a great latent space is one where following the gradient of value is enough to solve complex problems. Its deepest implication is not that hidden states are mysterious thoughts. It is that inference need not be a one-way execution of fixed weights: when a frozen model gets one problem wrong, there is somewhere to stand and something to follow—states to revise, and a value to say which revision is better.

The evidence assembled here reads as two tests of that sentence. LatentSeek shows the headroom is real: even at the boundary of language, final hidden states hold better solutions than the model’s first pass—more, indeed, than its self-supplied value can yet cash in. GradCuit shows the headroom grows when the space is chosen well: moved to the network’s interior, where attention carries value back to every latent, the same simple optimizer becomes both stronger and more stable. Two instruments, one reading: intermediate representations are not transient by-products. They are the terrain on which value-guided search succeeds or fails.

Much remains before the sentence hardens from philosophical hypothesis into law: reliable values, compute-matched scaling curves, safeguards against off-manifold exploitation, evidence beyond compact verifiable tasks—and, ideally, systems that combine the expressive freedom of continuous optimization with the auditability of a readable workspace.

But the direction of travel is set. We do not need cleverer search so much as better places to search, and better guides to search with. Where such spaces exist, the field’s task is to find them; where they do not, to learn to shape them—until following the gradient of value is enough.

Citation

@misc{li2026ttlr,
  author = {Li, Hengli and Zheng, Zilong and Zhang, Chi and Zhu, Song-Chun and Wu, Ying Nian},
  title = {Reasoning as Value-Guided Latent-Space Optimization},
  year = {2026},
  url = {https://latentreasoning.github.io/test-time-latent-reasoning}
}

References

1. Dathathri, S., Madotto, A., Lan, J., Hung, J., Frank, E., Molino, P., Yosinski, J., & Liu, R. (2020). Plug and Play Language Models: A Simple Approach to Controlled Text Generation. International Conference on Learning Representations. https://arxiv.org/abs/1912.02164
2. Deng, Y., Choi, Y., & Shieber, S. (2024). From Explicit CoT to Implicit CoT: Learning to Internalize CoT Step by Step. arXiv preprint arXiv:2405.14838. https://arxiv.org/abs/2405.14838
3. Gurnee, W., Sofroniew, N., Pearce, A., Piotrowski, M., Kauvar, I., Chen, R., Soligo, A., Bogdan, P., Ong, E., Wang, R., Thompson, B., Abrahams, D., Kantamneni, S., Ameisen, E., Batson, J., & Lindsey, J. (2026). Verbalizable Representations Form a Global Workspace in Language Models. arXiv preprint arXiv:2607.15495. https://transformer-circuits.pub/2026/workspace/index.html
4. Hao, S., Sukhbaatar, S., Su, D., Li, X., Hu, Z., Weston, J., & Tian, Y. (2025). Training Large Language Models to Reason in a Continuous Latent Space. Conference on Language Modeling. https://arxiv.org/abs/2412.06769
5. Kong, D., Zhao, M., Xu, D., Pang, B., Wang, S., Honig, E., Si, Z., Li, C., Xie, J., Xie, S., & Wu, Y. N. (2025). Latent Thought Models with Variational Bayes Inference-Time Computation. International Conference on Machine Learning. https://arxiv.org/abs/2502.01567
6. Li, X. L. & Liang, P. (2021). Prefix-Tuning: Optimizing Continuous Prompts for Generation. Proceedings of ACL-IJCNLP. https://arxiv.org/abs/2101.00190
7. Li, H., Li, C., Wu, T., Zhu, X., Wang, Y., Yu, Z., Jiang, E. H., Zhu, S. C., Jia, Z., Wu, Y. N., & Zheng, Z. (2025). Seek in the Dark: Reasoning via Test-Time Instance-Level Policy Gradient in Latent Space. arXiv preprint arXiv:2505.13308. https://arxiv.org/abs/2505.13308
8. Liu, C., Yang, Y., Fan, Y., Wei, Q., Liu, S., & Wang, X. E. (2026). Reasoning Within the Mind: Dynamic Multimodal Interleaving in Latent Space. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. https://arxiv.org/abs/2512.12623
9. Mi, Y., Li, H., Zhao, Y., Li, C., Wu, H., Ma, X., Zhu, S. C., Wu, Y. N., & Li, Q. (2026). MILR: Improving Multimodal Image Generation via Test-Time Latent Reasoning. International Conference on Learning Representations. https://arxiv.org/abs/2509.22761
10. Snell, C., Lee, J., Xu, K., & Kumar, A. (2025). Scaling LLM Test-Time Compute Optimally Can Be More Effective Than Scaling Model Parameters. International Conference on Learning Representations. https://arxiv.org/abs/2408.03314
11. Wang, X., Wei, J., Schuurmans, D., Le, Q. V., Chi, E. H., Narang, S., Chowdhery, A., & Zhou, D. (2023). Self-Consistency Improves Chain of Thought Reasoning in Language Models. International Conference on Learning Representations. https://arxiv.org/abs/2203.11171
12. Wei, J., Wang, X., Schuurmans, D., Bosma, M., Ichter, B., Xia, F., Chi, E. H., Le, Q. V., & Zhou, D. (2022). Chain-of-Thought Prompting Elicits Reasoning in Large Language Models. Advances in Neural Information Processing Systems, 35.
13. Williams, R. J. (1992). Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning. Machine Learning, 8, 229–256.
14. Yao, S., Yu, D., Zhao, J., Shafran, I., Griffiths, T. L., Cao, Y., & Narasimhan, K. (2023). Tree of Thoughts: Deliberate Problem Solving with Large Language Models. Advances in Neural Information Processing Systems, 36. https://arxiv.org/abs/2305.10601
15. Ye, W., Liang, Y., & Shan, L. (2026). Thinking on the Fly: Test-Time Reasoning Enhancement via Latent Thought Policy Optimization. International Conference on Learning Representations. https://arxiv.org/abs/2510.04182
16. Yu, Z., Shen, Q., Li, H., Zhang, Z., Zhu, S. C., Zhang, C., & Zheng, Z. (2026). GradCuit: Credit-Assigned Gradient Flow Enables Robust and Interpretable Test-Time Latent Reasoning. arXiv preprint arXiv:2608.02585. https://arxiv.org/abs/2608.02585
17. Zhang, Z., He, X., Yan, W., Shen, A., Zhao, C., Wang, S., Shen, Y., & Wang, X. E. (2025). Soft Thinking: Unlocking the Reasoning Potential of LLMs in Continuous Concept Space. arXiv preprint arXiv:2505.15778. https://arxiv.org/abs/2505.15778