related work of Facing Off World Model Backbones: RNNs, Transformers, and S4

“Facing Off World Model Backbones: RNNs, Transformers, and S4” (Deng, Park, and Ahn, 2023) studies which sequence-model backbone is most effective for latent world models in model-based reinforcement learning. It introduces S4WM, applying Structured State Space Sequence (S4) layers to high-dimensional visual latent imagination.

Related work

• Latent world models / model-based RL. The closest foundations are latent-dynamics agents such as PlaNet, Dreamer, DreamerV2, and DreamerV3, which learn compact stochastic state representations and predict latent future trajectories for policy optimization. The paper positions recurrent RSSM-style models—especially Dreamer—as the dominant baseline family.

• Recurrent neural networks. RNN, LSTM, and GRU dynamics models provide online recurrent state updates and have been central to visual world modeling. Their main limitation is sequential training over time, which reduces temporal parallelism and can make modeling very long contexts harder.

• Transformer world models. Transformer-based dynamics models, including TransDreamer, replace recurrence with self-attention, enabling parallel training and direct access to long histories. Their standard quadratic attention cost in sequence length makes long imagined rollouts and long-context visual modeling computationally expensive.

• Structured state-space models. S4 is a structured state-space sequence model designed for long-range dependencies, combining a recurrent state-space interpretation with a convolutional form that permits parallel sequence training. The paper argues that this makes S4 a promising middle ground: linear-time recurrent inference and parallelizable training.

• S4WM’s distinction. Rather than using S4 for conventional sequence benchmarks, S4WM incorporates it into a latent visual world model, with image encoding/decoding and stochastic latent dynamics, then evaluates it against RNN and Transformer backbones under a common world-modeling framework. The stated contribution is the first S4-based world model for generating high-dimensional image sequences via latent imagination.

A concise positioning statement for a literature review:

Prior world models have primarily used recurrent latent dynamics or Transformers. RNN-based models are efficient at inference but train sequentially, while Transformer world models parallelize training but have quadratic temporal cost. S4WM explores structured state-space layers as an alternative backbone intended to retain parallel training while supporting efficient long-horizon dynamics modeling.