IDEAS, EXPLAINED

Blog Posts

Notes on generative intelligence, visual systems and the ideas behind them.

2024 — 2026

Writing archive

Newest first

Video generation

The Forcing Family: Turning Bidirectional Video DiTs into Streaming Generators

Streaming video generation is a change in the model’s information, state distribution, denoising schedule, and memory. This article develops a unified interpretation of the forcing family through four mismatches, explaining why causal distillation, self-generated histories, rolling denoising, and persistent context must be designed together.

Persistent context frames feed a causal film strip, with progressively noisy future chunks and an open-ended timeline.
CAUSAL HISTORYCONTINUOUS FRAMES
Video generation

Controllable Video Generation

A technical guide to controllable video generation, covering keyframe, structural, trajectory, reference-video, and camera controls. The article unifies these methods through condition organization, spatiotemporal encoding, feature injection, and the distinction between soft guidance and hard constraints.

Three video frames carry joint-level pose controls, spatial boxes and an explicit motion path on a shared time axis.
SPACE & TIMECONTROLLED MOTION
Generative models

Controllable Generation in Diffusion and Flow-Based Models

Controllable generation is no longer a collection of isolated tricks built around text prompts or guidance scales. In modern diffusion and flow-based models, control can be imposed by reshaping the sampling dynamics, injecting external conditions, optimizing intermediate states, adapting concept carriers, or training native multimodal interfaces. This article develops a unified dynamical view of guided and controlled generation: from classifier guidance and classifier-free guidance to inversion-based editing, measurement guidance, ControlNet-style structural control, DreamBooth-style personalization, training-free identity injection, and instruction-following image editing. Across these methods, the central question remains the same: how can we steer a generative trajectory toward human-specified intent while preserving realism, diversity, editability, and consistency?

A condition adds a guidance vector to a generative path, steering noise through an energy landscape toward a specified target.
DIRECTION FIELDHUMAN INTENT
Generative models

Embedding Space Design For Diffusion and Flow Architecture

This article presents a unified perspective on embedding space design for diffusion, flow matching, and flow-map generative models, systematically analyzing how spatial, temporal, semantic, and multimodal conditions are encoded and injected into modern generative backbones, with emphasis on controllability, scalability, and multi-resolution generalization.

A spatial coordinate lattice, a time waveform and rotary position geometry feed condition injection points in a generative backbone.
ENCODE CONDITIONSINJECT FEATURES
Alignment

Preference Alignment of Diffusion and Flow-Matching Models

Preference alignment for diffusion and flow-matching models is not merely a collection of post-training tricks, but a problem of reshaping generative trajectories under human-defined utility. This article builds a unified trajectory-policy view that connects reinforcement learning, denoising dynamics, flow-based generation, and preference optimization into a common mathematical framework.

Denoising trajectories branch into ranked image outcomes; a preferred route receives the strongest emphasis and a selection mark.
TRAJECTORY POLICYPREFERRED OUTCOME
Generative models

From Trajectories to Operators: A Unified Flow Map Perspective on Generative Modeling

In this post, we reframe continuous-time generative modeling from integrating trajectories to learning two-time operators (flow maps). This operator view unifies diffusion, flow matching, and consistency models, and suggests a practical diagnostic — semigroup-consistent jumps yield both step-robust generation and low compositional drift. We derive Eulerian/Lagrangian distillation objectives and use inpainting experiments to show why semigroup-consistent jumps can be both step-robust and composition-stable.

Three time planes connect through two composed flow-map arcs and a single direct operator arc between the same endpoints.
TWO-TIME OPERATORSCOMPOSED JUMPS
Multimodal

Discrete Visual Tokenizers for Multimodal LLMs: Bridging Vision and Language

How do we convert continuous images into discrete tokens that large language models can understand? This survey traces the complete evolution of discrete image tokenization—from the groundbreaking VQ-VAE in 2017 to today’s FSQ, LFQ, and TiTok designed for multimodal LLMs. We reveal a compelling narrative of three paradigm shifts: the initial quest for better generative models, the existential challenge from diffusion models, and the remarkable renaissance driven by vision-language unification. Whether you’re building multimodal AI systems or seeking to understand the technical foundations behind models like Chameleon and Emu3, this comprehensive survey provides the complete picture.

A continuous image is divided into patches, quantized through a highlighted codebook entry, and translated into discrete visual token IDs.
CONTINUOUS IMAGEDISCRETE TOKENS
Multimodal

Visual Representation Learning

This post provides a systematic, mechanics-first tour of visual representation learning in the foundation-model era—framing representation as the internal currency that enables native multimodality and the unification of understanding and generation. It connects the major paradigms—contrastive learning, negative-free self-distillation, masked modeling (pixel/token/feature targets), generative representations (VAE/AR/Diffusion/GAN), multimodal learning (alignment vs fusion), and JEPA-style joint-embedding prediction—into one coherent map, emphasizing objectives, anti-collapse mechanisms, and what each family’s representations are actually good for.

Augmented views, masked patches, self-distillation and latent prediction surround a shared visual representation manifold.
VISUAL EXPERIENCEUSEFUL REPRESENTATIONS
Generative models

Inverse Problems with Generative Priors

Inverse problems, which aim to recover a signal of interest from indirect and often corrupted measurements, are a cornerstone of computational science and engineering. These problems are typically ill-posed, necessitating the use of prior knowledge to regularize the solution space and ensure a unique and stable reconstruction. This post provides a structured exposition of the evolution of priors in solving inverse problems, from classical formulations to the modern paradigm of deep generative models. This work aims to bridge the conceptual gap between classical and modern techniques, offering a unified view of the role of priors in solving generative inverse problems.

A partially observed image is reconstructed by intersecting measurement evidence with a generative prior manifold.
OBSERVATIONS + PRIORRECONSTRUCTION
Generative models

Accelerating Diffusion Sampling: From Multi-Step to Single-step Generation

This article takes a deep dive into the evolution of diffusion model sampling techniques, tracing the progression from early score-based models with Langevin Dynamics, through discrete and non-Markov diffusion processes, to continuous-time SDE/ODE formulations, specialized numerical solvers, and cutting-edge methods such as consistency models, distillation, and flow matching. Our goal is to provide both a historical perspective and a unified theoretical framework to help readers understand not only how these methods work but why they were developed.

A long denoising arc with many evaluation points is contrasted with a direct learned leap from noise to a clean endpoint.
MANY EVALUATIONSONE LEARNED LEAP
Generative models

Fast Generation with Flow Matching

Fast sampling has become a central goal in generative modeling, enabling the transition from high-fidelity but computationally intensive diffusion models to real-time generation systems. While diffusion models rely on tailored numerical solvers to mitigate the stiffness of their probability flow ODEs, flow matching defines dynamics through smooth interpolation paths, fundamentally altering the challenges of acceleration. This article provides a comprehensive overview of fast sampling in flow matching, with emphasis on path linearization strategies (e.g., Rectified Flow, ReFlow, SlimFlow, InstaFlow), the integration of consistency models, and emerging approaches such as flow generators.

Three distinct path families straighten from reflow to shortcut jumps and mean-velocity transport for fewer-step generation.
RECTIFY THE PATHREDUCE THE STEPS
Generative models

From Diffusion to Flow: A New Genrative Paradigm

In this post, we uncovered the foundations of Flow Matching: the limitations of diffusion models, the constraints of continuous flows, and the transformative idea of directly learning the path between distributions. From the intuition of Rectified Flow to the unifying lens of Stochastic Interpolants, Flow Matching emerged as more than a method — it is a paradigm that reframes generation as learning currents of transformation. With this conceptual map in hand, we are now ready to move from theory to practice.

A source point cloud is transported through a learned vector field into a structured two-branch target distribution.
PROBABILITY PATHSVELOCITY FIELDS
Generative models

High-Order PF-ODE Solver in Diffusion Models

Diffusion sampling can be cast as integrating the probability flow ODE (PF-ODE), but dropping it into a generic ODE toolbox rarely delivers the best speed–quality trade-off. This post first revisits core numerical-analysis ideas. It then explains why vanilla integrators underperform on the semi-linear, sometimes stiff PF-ODE in low-NFE regimes, and surveys families that exploit diffusion-specific structure: pseudo-numerical samplers (PLMS/PNDM) and semi-analytic/high-order solvers (DEIS, DPM-Solver/++/UniPC). The goal is a practical, unified view of when and why these PF-ODE samplers work beyond “just use RK4.”

A smooth probability-flow trajectory, coarse numerical approximations and a magnified local-error region illustrate high-order diffusion solvers.
NUMERICAL STRUCTUREACCURATE UPDATES
Generative models

Consistency Theory: From Discrete Constraints to Continuous Flows

Consistency models (CMs) have recently emerged as a powerful paradigm for accelerating diffusion sampling by directly learning mappings that preserve consistency across noisy representations of the same data. This paper provides a comprehensive study of the Consistency Family, tracing its evolution from discrete consistency models to continuous-time formulations and trajectory-based extensions. We begin by revisiting the foundational motivation behind CMs and systematically derive their discrete and continuous objectives under both consistency distillation and consistency training paradigms.

Several noisy states along one trajectory map to the same clean target, illustrating the consistency condition across time.
DIFFERENT NOISE STATESONE CLEAN TARGET
Generative models

Diffusion Architectures Part I: Stability-Oriented Designs

This article explores how network architectures shape the stability of diffusion model training. We contrast U-Net and Transformer-based (DiT) backbones, analyzing how skip connections, residual scaling, and normalization influence gradient propagation across noise levels. By surveying stability-oriented innovations such as AdaGN, AdaLN-Zero, and skip pathway regulation, we reveal why architectural choices can determine whether training converges smoothly or collapses. The discussion provides both theoretical insights and practical design rules for building robust diffusion models.

A U-Net with gated skip bridges and a transformer with scaled residual routes illustrate stable gradient propagation.
GATED SKIPSSTABLE GRADIENTS
Generative models

Analysis of the Stability and Efficiency of Diffusion Model Training

while diffusion models have revolutionized generative AI, their training challenges stem from a combination of resource intensity, optimization intricacies, and deployment hurdles. A stable training process ensures that the model produces good quality samples and converges efficiently over time without suffering from numerical instabilities.

Conceptual signal, noise and velocity information curves share an SNR axis above sampling, weighting, prediction-target and scale controls; no measured data is plotted.
SNR & OBJECTIVESBALANCED TRAINING
Generative models

Unifying Discrete and Continuous Perspectives in Diffusion Models

Diffusion models have been shown to be a highly promising approach in the field of image generation. They treat image generation as two independent processes: the forward process, which transforms a complex data distribution into a known prior distribution (typically a standard normal distribution) by gradually injecting noise; and the reverse process, which transforms the prior distribution back into the complex data distribution by gradually removing the noise.

A discrete forward-noising chain is linked to a continuous reverse-time flow as the step size tends to zero.
DISCRETE TIMECONTINUOUS DYNAMICS