Benchmark vs ComfyUI

ComfyUI and Nox run the same generation jobs on the same machine, so the timings are directly comparable. Each pipeline is generated twice: a Cold run in a fresh process (including model and pipeline load) and a Warm run immediately afterwards. Speedup is calculated as ComfyUI time / Nox time; above 1x means Nox was faster. Both engines load the exact same checkpoints: INT8 ConvRot for every pipeline except SDXL, which runs in FP16 on both.

Warm timings

SSIM compares the saved Nox and ComfyUI Warm outputs (seed 43). Image scores use all RGB pixels; video scores average nine evenly spaced decoded frames. Higher SSIM means closer outputs.

Cold timings and peak VRAM

ComfyUI total benchmark time: 490.53s Nox total benchmark time: 290.29s

Peak VRAM is reported using each benchmark’s native measurement method: actual device usage for ComfyUI and tracked allocation peak for Nox. Nox keeps models loaded in VRAM until the memory is needed for something else, instead of freeing it after every generation. On GPUs with more memory its peak therefore rises toward the available VRAM; this avoids reloading models and does not reflect the minimum a pipeline needs to run.

What is measured

Every pipeline runs its built-in generation preset with the same fixed prompts and a deterministic seed (42 for Cold, 43 for Warm). Image pipelines render a single 1024x1024 frame. Video pipelines render at 24 fps: LTX-2.3 and LTX-2.5 render 121 frames at 512x768, and MiniMax-H3 renders 56 frames at 512x768.

Environment

• GPU: NVIDIA GeForce RTX 4080 (SM 8.9), FP8 matmul supported, INT8 matmul supported, NVFP4 matmul unsupported
• ComfyUI version: 0.43.0
• ComfyUI revision: 426f5eea52a7ece1dd599fff4cb6cd1f352ddd24
• ComfyUI PyTorch version: 2.9.1+cu128