Benchmark vs ComfyUI

ComfyUI and Nox run the same generation jobs on the same machine, so the timings are directly comparable. Each pipeline is generated twice: a Cold run in a fresh process (including model and pipeline load) and a Warm run immediately afterwards. Speedup is calculated as ComfyUI time / Nox time; above 1x means Nox was faster. Both engines load the exact same checkpoints: INT8 ConvRot for every pipeline except SDXL, which runs in FP16 on both.

Warm timings

Cold timings and peak VRAM

ComfyUI total benchmark time: 600.52s Nox total benchmark time: 350.34s

Peak VRAM is reported using each benchmark’s native measurement method: actual device usage for ComfyUI and tracked allocation peak for Nox. Nox keeps models loaded in VRAM until the memory is needed for something else, instead of freeing it after every generation. On GPUs with more memory its peak therefore rises toward the available VRAM; this avoids reloading models and does not reflect the minimum a pipeline needs to run.

What is measured

Every pipeline runs its built-in generation preset with the same fixed prompts and a deterministic seed (42 for Cold, 43 for Warm). Image pipelines render a single 1024x1024 frame. Video pipelines render at 24 fps: LTX-2.3 and LTX-2.5 render 121 frames at 512x768, and MiniMax-H3 renders 56 frames at 512x768.

Environment

• GPU: NVIDIA GeForce RTX 3090 (SM 8.6), FP8 matmul unsupported, INT8 matmul supported, NVFP4 matmul unsupported
• ComfyUI version: 0.37.0
• ComfyUI PyTorch version: 2.9.1+cu128