Minseok Jeong

Random Network Distillation (RND) is a scalable novelty signal for RL based on a fixed random target network and a trained predictor. However, it can overgeneralize: the predictor extrapolates the random target off the data manifold, causing novelty scores to collapse on unfamiliar inputs. This paper analyzes this failure through a kernel-theoretic lens. Under a GP target model and an NTK-regime predictor, we show that the RND novelty score is a cross-kernel residual variance: it is governed jointly by the target covariance kernel and the predictor NTK, rather than by either one alone. This identifies overgeneralization as spectral under-excitation. Motivated by this view, we formulate spectral target design as a principle for shaping RND novelty geometry. As a concrete instance, we replace the implicit random-network target with a bandwidth-controlled Random Fourier Feature (RFF) target, redistributing residual signal toward modes less attenuated by the predictor. Experiments on offline D4RL, online Atari, and Fetch manipulation benchmarks show improved novelty discrimination and downstream RL performance.

Koopman operator theory provides a linear representation of nonlinear dynamics, offering strong potential for long-horizon prediction. However, most existing methods rely on snapshot-based one-step fitting and often suffer from error accumulation over extended rollouts. To address this issue, prior efforts have explored extending the training horizon or developing model-based approaches that leverage physics-informed formulations—such as infinitesimal generator fitting—to better exploit the underlying system dynamics. In this paper, we show that local transition fitting itself fundamentally limits long-horizon stability, whereas generator-based learning provides a more principled alternative. Despite their reliable advantages, generator-based learning schemes remain computationally demanding, as training dense MLP-based dictionaries with Jacobian computations is often intractable in practice. Building upon this line of work, we introduce a Random Fourier Feature–lifted Koopman formulation (RFF-PIKN) that employs shift-invariant RKHS representations for scalable and stable approximation, reducing computational cost under repeated resampling. We theoretically show that RFF-PIKN converges to a local minimizer of the true population generator risk, enabling accurate and stable long-horizon prediction. These results highlight the promise of RKHS-based physics-informed Koopman learning in advancing model-based prediction frameworks.

Recently, physics-informed neural networks (PINNs) have gained attention in the scientific community for their potential to solve partial differential equations (PDEs). However, they face challenges related to resource efficiency and slow convergence. Adaptive sampling methods, which prioritize collocation points with high residuals, improve both efficiency and accuracy. However, these methods often neglect points with medium or low residuals, which can affect stability as the complexity of the model increases. In this paper, we investigate this limitation and show that high residual-based approaches require stricter learning rate bounds to ensure stability. To address this, we propose a Langevin dynamics-based Adaptive Sampling (LAS) framework that is robust to various learning rates and model complexities. Our experiments demonstrate that the proposed method outperforms existing approaches in terms of relative error, and stability across a range of enviroments, including high-dimensional PDEs where Monte Carlo integration-based methods typically suffer from instability.