Science & Mathematics

Contrastive vision-language models such as CLIP typically rely on independently parameterized transformer blocks across depth. We investigate the extent to which these blocks can instead be reused recurrently while preserving the transferable capabilities of CLIP. We study recursive transformer backbones in which a small set of $N$ transformer blocks is applied $M$ times per forward pass. By holding $N \cdot M$ fixed to the depth of a standard ViT-B, our models preserve computational depth while progressively increasing parameter sharing. We consider two recursive CLIP model families: rCLIP, where only the vision encoder is recursive, and r$^2$CLIP, where recurrence is also introduced in the text encoder. We study these models under both direct contrastive training and through knowledge distillation from a strong ViT-B/16 teacher pretrained on DFN-2B, using the substantially smaller CC3M and CC12M datasets. We evaluate the resulting models on zero-shot classification and cross-modal retrieval, and further study their adaptability through few-shot learning, visual grounding, and concept unlearning. Our results show that CLIP can tolerate substantial recurrent block reuse, although its effect depends on the encoder and downstream task. Recurrence also provides parameter efficiency by construction, since fewer distinct transformer blocks need to be stored and adapted. Our results establish recurrent block reuse as a practical design space for vision-language models and highlight its implications for knowledge transfer, downstream adaptation, and parameter efficiency.

Most action-conditioned video world models remain limited to a single player, while existing multiplayer models support only small, fixed groups of agents. We introduce FlockWorld, an artificial-life environment built on Reynolds's Boids algorithm for studying population-scale multi-agent video world models. Specifically, FlockWorld renders synchronized egocentric crops, assigns distinct colors to preserve camera-agent identity across views, exposes rule-derived accelerations as actions, and lets the number of camera agents be configured independently of the size of the surrounding population. We further introduce FlockDiT, an action-conditioned latent diffusion transformer trained with flow matching on FlockWorld data to jointly generate synchronized egocentric videos for multiple agents. Our results suggest that joint generation benefits agent detection at density-matched settings and that diffusion forcing improves rollout fidelity. Together, FlockWorld and FlockDiT provide a controlled testbed for multi-agent video generation.

Accurately capturing the Mullins effect across varying material compositions remains challenging, as constitutive models must unify flexible representations of stress softening with physical consistency and robust parameter calibration. In this work, we investigate parameter-dependent physics-augmented neural network constitutive models within the pseudo-elastic modeling framework, focusing on structural constraints that ensure physically admissible neural softening functions. This paper has three key contributions: (1) We formulate different neural softening functions within the pseudo-elastic framework, including a multiplicative decomposition of the softening function that enables parameter-dependent ansatz functions satisfying common mechanical constitutive conditions by construction. (2) We derive a sufficient condition for ellipticity of pseudo-elastic constitutive models, which can be enforced in the constitutive models by construction. (3) We systematically investigate how flexibility and structural constraints influence the performance of neural softening functions in terms of prediction accuracy and calibration robustness. For this, we consider multiaxial experimental data of carbon black-filled EPDM rubber synthesized with varying amounts of curatives. The findings of our work suggest that neural softening functions are a promising extension of the classical pseudo-elastic framework and, when subject to suitable constraints, can improve predictive accuracy for highly nonlinear, parameter-dependent materials.

Binary-weight language models store one bit per weight, but training them keeps a full-precision latent copy of each weight plus Adam state, about 96 extra bits per weight, so a deployed binary model cannot be adapted on the device that runs it. We replace the latent weight with a 2-8-bit residual buffer quantized by stochastic rounding, which cuts the training state per weight from 97 bits to 3-9, up to 19x less. The resulting error-diffusion (ED) optimizer moves the residual-carrying loop of image halftoning onto the weight-update path. Each buffer accumulates the unrealized part of every update and flips the weight when the accumulated value crosses a threshold, and it works only when its quantization grid has a level exactly at that threshold. Under this threshold-alignment rule, 4-bit buffers match 8-bit buffers on a binary multilayer perceptron benchmark, and the same rule lifts a different latent-free optimizer, Bop, from a 10-point deficit at 4 bits to full-precision accuracy. On character-level transformers with 97% of parameters binary, ED trains within 5-11% of the latent-weight reference at a 2,000-step budget and within 1.7% at convergence. On scaling ladders over enwik8 and wikitext-103, up to 19M parameters on enwik8 (70M with the wikitext-103 embeddings), the latent-free loss stays within -2.3% to +2.9% of the reference at all six sizes once a per-step cap on the buffer update removes a launch instability; without the cap it trails by 10% at the largest tokenized size. A deployed binary model continues training under ED at one tenth of the latent training state, matching latent-weight continuation, and a fused CPU kernel completes the adaptation within a 275 MB training-memory budget, which latent-weight training cannot. A 250-step probe sets the flip learning rate at any new scale.

The largest open German text datasets are built from broad web crawls, which provide scale but limited coverage of specific domains. We introduce RefinedGAW, a processed and annotated resource derived from 26 semi-annual crawls of the German Academic Web, totaling 122 million documents and 314 billion tokens, with full crawl and capture history of every text. Documents labeled as academic research account for 84% of German RefinedGAW tokens across HTML and PDF. At 87.6 billion tokens, this exceeds the 64.9 billion tokens in the German subsets of FineWeb2 and FinePDFs, despite their almost five-times-larger combined size. RefinedGAW also contains substantial amounts of educational, math-heavy, and expert-oriented content with minimal commercial bias. Across HTML and PDF, 88% of German RefinedGAW documents have no near-duplicate candidate match in the three evaluated reference datasets, supporting complementary coverage. We release a queryable metadata index for fine-grained data selection and a rehydration tool for retrieving and processing source documents.

AI has made significant contributions to scientific discovery. Autonomous agents can now independently analyse complex problems, design experiments, write and execute code, analyse findings, and derive scientific conclusions. Nevertheless, errors in AI-generated code, experimental procedures, or evidence interpretation can lead to invalid research conclusions, undermining the reliability and reproducibility of automated scientific discovery. In this vision paper, we propose Software Engineering for AI-Driven Science (SE4AI4Sci) as an emerging research area for addressing this challenge. As an organising framework, we introduce the Scientific Assurance Chain, connecting intent, experimental design, implementation, execution, evidence, and claim. Through six documented failures in recent AI-for-science systems, we show where this chain can break, where existing assurance mechanisms fall short, and what new research challenges follow. We also outline a research agenda for adapting and extending software engineering techniques across the chain.

Autonomous underwater vehicles (AUVs) navigate without satellite positioning and rely on Doppler velocity log (DVL) and inertial sensors for motion estimation. Deep learning methods for processing these sensors require training data covering the motion the AUV encounters, commonly obtained through costly sea trials. To overcome this limitation, we present a conditional diffusion framework that generates synthetic DVL and inertial signals to replace real recordings in training. The generator employs an Elucidated diffusion model conditioned on three per-channel motion descriptors. Three configurations are evaluated: DVL beams alone, inertial channels alone, and all channels jointly from one model. Across thirteen trajectories of the real dataset, the generated signals followed the requested conditions with Pearson correlations between 0.72 and 0.997, transferred to trajectories withheld from generator training, and did not replicate any training window. A BeamsNet velocity estimator trained on generated signals alone, with no real recording retained, matched the velocity error obtained with the full real training set. Replacing up to all real windows with generated ones caused no measurable loss. Thus, this approach reduces the dependence of learned AUV navigation on sea-trial data, enables training when recordings are scarce, and facilitates developing and evaluating deep learning methods for underwater navigation.

The sampling procedures of diffusion and flow-matching models involve hundreds of denoising steps incurring substantial inference time, which has motivated a growing body of work on training schemes for few-step and one-step generation. Existing approaches primarily seek to align the generation trajectory of few-step and base models, yet their training objectives only involve discrepancies at a single point in the trajectory, encouraging only local \emph{point-level} alignment. While recent methods such as \emph{Shortcut-CSL} introduce cumulative objective that enforce alignment over the entire trajectory, this is suboptimal as they do not directly enforce alignment between the distributions induced by the two models throughout the generation process. We propose \emph{Cumulative Marginal Matching} (CMM), a principled objective for few-step generation: by viewing flow-matching as a sequence of intermediate marginal distributions connecting the prior and data distributions, CMM directly optimizes the cumulative discrepancy between the base and few-step marginal distributions over the entire sequence. Specifically, we formulate this discrepancy using the Wasserstein-2 distance. By exploiting the pushforward representation of flow-matching, we transform an intractable cumulative Wasserstein objective into a tractable surrogate that can be optimized using samples from the intermediate marginal, which are readily generated through the conditional interpolation process of flow-matching. Furthermore, we show that the existing \emph{Shortcut-CSL} objective arises as a first-order approximation of that of the CMM, establishing a theoretical connection between the two objectives. Experiments on standard benchmarks demonstrate improvement in generation quality over \emph{Shortcut-CSL} and other state-of-the-art one- and few-step baselines under identical training budgets.

Agent harnesses enable large language models to act through tools, observe the results, and iteratively work toward a task goal. Verification checks and revises agents' work to improve output quality. Yet there is limited guidance on how harnesses should organize this effort. We study verification as an axis of test-time scaling by examining when to intervene, who checks the work, and how checking leads to revision. Our framework spans four layers of the agent lifecycle: initial instructions, mid-turn reminders, completion gates, and independent review. We compare 17 designs across seven frontier coding models on Terminal-Bench (TB) and SWE-bench Verified (SWE), and introduce VisualBench for expert evaluation of visual and interactive artifacts. Initial instructions and execution-time reminders yield modest, model-dependent average gains on the coding benchmarks, without substantially increasing average output token use. Completion gates can yield larger gains but may substantially increase cost. Independent review by a critic offers a more favorable accuracy-cost tradeoff, improving average accuracy by 5.9-9.5 percentage points on TB and 3.9-6.6 on SWE at 1.13-1.25 and 1.00-1.04 times baseline cost, respectively. On VisualBench, independent review achieves net preferences of 13.2-23.8 points over the baseline across its four designs, exceeding the evaluated L0-L2 designs. Additional review rounds or independent critics further improve average TB accuracy by up to 3.5 percentage points. Together, these results provide empirical guidance for verification design in agent harnesses and support independent review as a practical axis of test-time scaling.

Verifying uniform observability of a linear time-varying system generally requires its state-transition matrix. This paper develops necessary and sufficient conditions that can be checked directly from the coefficient matrices for two classes of systems. For constant dynamics with real spectrum and a measurable, essentially bounded output matrix, uniform observability is equivalent to persistent excitation of the output matrix on each eigenspace. An equivalent condition requires persistent excitation of the Kalman observability stack. The second result concerns cascades with a constant observable measured subsystem coupled to a hidden state. Under uniform regularity assumptions, linear combinations of output derivatives eliminate the measured state and yield an equivalent observability test for the hidden dynamics. When those dynamics have real spectrum, the test reduces to excitation on each eigenspace intersected with the kernel of the direct measurements. An application to arbitrary-order integrator chains with measurements of ranges to fixed anchors gives an exact excitation condition for the associated lifted LTV system. A second application treats direct hidden measurements and compares the condition with existing sufficient tests.

We describe HobbyLM, a family of small sparse Mixture-of-Experts (MoE) models trained from scratch with a few hundred to a few thousand dollars of rented GPU time. The core is a decoder-only transformer with fine-grained experts, sigmoid routing with auxiliary-loss-free load balancing, grouped-query attention with per-head QK-normalization, and aMuon/AdamW optimizer split. Each of these choices was selected with a single-variable ablation suite at 130M parameters. On this architecture we train a 500M-parameter model (169M active) on 40B FineWeb tokens and post-train it into a chat model, a masked-diffusion language model, a function-calling and computer-use agent, and a single "omni" checkpoint that reads text, images, video, sound and speech through frozen encoders and small projectors. We run these models on a laptop CPU with a from-scratch Rust inference engine. Two of the authors then adopted the architecture without architectural changes to build HobbyLM-1B (1.04B total, 305M active parameters), trained on a 100B-token curated mixture with a high-quality anneal. HobbyLM-1B reaches a 47.5 average on seven zero-shot benchmarks, and its builders also release an instruction-tuned model and an on-device macOS computer-use adapter. Finally, we describe HobbyLM-Image, a 333M-parameter flow-matching diffusion transformer that generates 1024x1024 images in a 32x-compressed latent space, together with a dependency-light CPU implementation of the full text-to-image pipeline. We report what worked, what did not, and where the small-scale ceilings lie. We also state for each component which author designed or built it.

A robot searching for the source of an odor must eventually answer a deceptively hard question: have I arrived? Near the source, concentration gradients flatten, turbulence makes the signal intermittent, and a small robot often lacks the positional accuracy to map the plume. Here, we describe a lightweight remedy inspired by a World War II statistics conundrum: the German tank problem. To approximate enemy armament production, Allied analysts developed an estimator based on serial numbers from captured tanks. Analogously, we use the same procedure over sampled chemical sensor readings from a robot to create estimates for the peak concentration of an odor plume, and therefore the odor's source. Treating each sensor reading as an observation, our unbiased estimator needs only a running maximum and a sample counter. An exact confidence bound turns both numbers into a principled rule that the robot or UAV can use to determine whether to keep searching for the plume source or land. We derive the estimator and its bound, demonstrate its sample efficiency, and examine its robustness even when its assumptions fail. Finally, we summarize the estimator's role as the termination criterion for olfactory navigation.

Theoretical modelling of diamond growth should ideally provide estimates of the growth rate depending on experimental conditions. To that end, a description of all possible reactions related to the competing adsorption, desorption and migration mechanisms is required. Here, we provide precise calculations of the reaction rate coefficient of methyl-related reactions on hydrogen-terminated (100) diamond surface, as $\mathrm{CH_3}$ is known to be the main growth precursor in chemical vapor deposition (CVD) growth of diamond. To achieve this goal, we successfully combined Density Functional Theory (DFT) and variational transition state theory (VTST), allowing to extend our analysis to barrierless reaction. Our rigorous treatment of the quasi-free rotation of $\mathrm{CH_3}$ adsorbates on the surface allows us to overcome numerical errors for the finite difference calculations of the vibrational spectra for evaluating the partition functions of the system. Our results suggest that the mechanism that deplete the $\mathrm{CH_3}$ concentration the most is the absorption on a surface radical site. We also evidence the anisotropy of $\mathrm{CH_3}$ migration along the dimer of the (100)-(2x1):H surface. In a more general perspective, the calculated reaction rates coefficient for methyl reactions and migration presented in this work, together with the data for hydrogen reaction and other main gas surface reactions, bridge the gap between the atomic-scale description and the large-scale modelling of diamond growth.

Large language models (LLMs) are neural networks trained on huge text collections to generate text by predicting what comes next. Engineers using ChatGPT and other generative artificial intelligence (AI) tools need a clear picture of what training changes and prompts add. The article starts with the Transformer, which connects words in context, and pre-training, where models learn to predict text. It then explains how fine-tuning adapts models to requests, with reinforcement learning from human feedback using human answer rankings. Next, retrieval-augmented generation (RAG) supplies documents, while chain-of-thought prompting encourages intermediate steps and agents use tools. FlashAttention lowers training and running costs by reducing data movement during attention; quantized low-rank adaptation (QLoRA) saves fine-tuning memory. Readers get a life cycle figure and tables comparing methods and their limits. A short checklist for judging LLM evaluation claims flags test material in training (data contamination) and model-based grading (LLM-as-a-judge). It also covers very long inputs (long context) and hallucination, meaning false or unsupported claims. Check factual accuracy separately from whether an answer sounds helpful.

When electrons move through non-trivial spin textures in magnetic metals, conduction electrons couple to magnetic order through both their charge and their spin, acquiring non-trivial equations of motion and a dynamic spin polarization providing the foundation of spintronics. These mechanisms are, however, almost invariably reciprocal and the resistance a current encounters does not depend on its flow direction. Here we show that a chiral crystal lattice and an independently reversible chiral magnetic order together produce a diode whose forward direction is set by the magnetism rather than by the structure. In the triangular-lattice antiferromagnet Co$_{1/3}$TaS$_2$, whose chiral lattice hosts a chiral magnetic order of its own, independent handedness, we observe a pronounced nonreciprocal resistance. Unlike conventional electrical magnetochiral anisotropy, which is driven by an external magnetic field, our spontaneous magnetic diode effect arises from an internal noncoplanar spin texture with vanishing net magnetization. Symmetry analysis and model calculations show that the effect originates from the interplay of noncoplanar spin order and spin-orbit coupling, which generates a chiral, time-reversal-symmetry-breaking electronic response. Crucially, reversing the magnetic handedness reverses the sign of the nonreciprocal resistance, demonstrating magnetic control of the diode polarity.