arXiv:2512.05117v2 Announce Type: replace-cross Abstract: We show that deep neural networks trained across diverse tasks exhibit remarkably similar low-dimensional parametric subspaces. We provide the first large-scale empirical evidence that demonstrates that neural networks systematically converge to shared spectral subspaces regardless of initialization, task, or domain. Through mode-wise spectral analysis of over 1100 models – including […]
Dropout Prompt Learning: Towards Robust and Adaptive Vision-Language Models
arXiv:2512.07234v1 Announce Type: cross Abstract: Dropout is a widely used regularization technique which improves the generalization ability of a model by randomly dropping neurons. In light of this, we propose Dropout Prompt Learning, which aims for applying dropout to improve the robustness of the vision-language models. Different from the vanilla dropout, we apply dropout on […]
AutoICE: Automatically Synthesizing Verifiable C Code via LLM-driven Evolution
arXiv:2512.07501v1 Announce Type: cross Abstract: Automatically synthesizing verifiable code from natural language requirements ensures software correctness and reliability while significantly lowering the barrier to adopting the techniques of formal methods. With the rise of large language models (LLMs), long-standing efforts at autoformalization have gained new momentum. However, existing approaches suffer from severe syntactic and semantic […]
Ghost in the Transformer: Detecting Model Reuse with Invariant Spectral Signatures
arXiv:2511.06390v3 Announce Type: replace-cross Abstract: Large Language Models (LLMs) are widely adopted, but their high training cost leads many developers to fine-tune existing open-source models. While most adhere to open-source licenses, some falsely claim original training despite clear derivation from public models, raising pressing concerns about intellectual property protection and the need to verify model […]
Incorporating Structure and Chord Constraints in Symbolic Transformer-based Melodic Harmonization
arXiv:2512.07627v1 Announce Type: cross Abstract: Transformer architectures offer significant advantages regarding the generation of symbolic music; their capabilities for incorporating user preferences toward what they generate is being studied under many aspects. This paper studies the inclusion of predefined chord constraints in melodic harmonization, i.e., where a desired chord at a specific location is provided […]
Towards Robust Protective Perturbation against DeepFake Face Swapping
arXiv:2512.07228v1 Announce Type: cross Abstract: DeepFake face swapping enables highly realistic identity forgeries, posing serious privacy and security risks. A common defence embeds invisible perturbations into images, but these are fragile and often destroyed by basic transformations such as compression or resizing. In this paper, we first conduct a systematic analysis of 30 transformations across […]
Provable Long-Range Benefits of Next-Token Prediction
arXiv:2512.07818v1 Announce Type: cross Abstract: Why do modern language models, trained to do well on next-word prediction, appear to generate coherent documents and capture long-range structure? Here we show that next-token prediction is provably powerful for learning longer-range structure, even with common neural network architectures. Specifically, we prove that optimizing next-token prediction over a Recurrent […]
Mind The Gap: Quantifying Mechanistic Gaps in Algorithmic Reasoning via Neural Compilation
arXiv:2505.18623v2 Announce Type: replace Abstract: This paper aims to understand how neural networks learn algorithmic reasoning by addressing two questions: How faithful are learned algorithms when they are effective, and why do neural networks fail to learn effective algorithms otherwise? To answer these questions, we use neural compilation, a technique that directly encodes a source […]
General Exploratory Bonus for Optimistic Exploration in RLHF
arXiv:2510.03269v3 Announce Type: replace-cross Abstract: Optimistic exploration is central to improving sample efficiency in reinforcement learning with human feedback, yet existing exploratory bonus methods to incentivize exploration often fail to realize optimism. We provide a theoretical analysis showing that current formulations, under KL or $alpha$-divergence regularization, unintentionally bias exploration toward high-probability regions of the reference […]
Kimi-Dev: Agentless Training as Skill Prior for SWE-Agents
arXiv:2509.23045v3 Announce Type: replace Abstract: Large Language Models (LLMs) are increasingly applied to software engineering (SWE), with SWE-bench as a key benchmark. Solutions are split into SWE-Agent frameworks with multi-turn interactions and workflow-based Agentless methods with single-turn verifiable steps. We argue these paradigms are not mutually exclusive: reasoning-intensive Agentless training induces skill priors, including localization, […]