InputLightweight
Diffusion image in the loop
Generated images used inside the scene.
Images generated on demand and used inside the scene: a texture from a prompt, a background that matches the 3D objects, a live feed repainted a few frames per second. Needs a GPU server and costs money per image.
- Breaks when
- cost per frameneeds a GPU server
- Also called
- img2img, realtime diffusion, StreamDiffusion
Prompts
Generated texture on a 3D object
A prompt field calls a serverless endpoint that requests one 1024² image from an image model and returns a URL; load it as a Three.js texture with anisotropy 8 and apply it to a product mesh with a 600 ms crossfade from the previous texture (two materials, a uniform mix in onBeforeCompile). Show progress with a shimmer, cache results by prompt, and cap the request size.
Frame-to-frame img2img loop
Design the pipeline for a live img2img loop: capture the Three.js canvas at 512², send it with a fixed prompt and low denoise strength (0.35) to a streaming diffusion endpoint via WebSocket, receive the result, and display it with a 2-frame buffer; explain how the denoise strength trades stability against change and what the achievable fps is.
Depth-conditioned background
Render the scene’s depth buffer to an image, send it as a control image with a prompt to a depth-conditioned model, and use the result as a matching background plane so the generated world lines up with the 3D objects. Show the depth image beside the result for review.
Related
Resources
None yet. Find one and add it with npm run add -- <url>.