Stable Diffusion WebGPU: AI-Driven Image Generation in Your Browser
Introducing Stable Diffusion WebGPU, an innovative web-based application that harnesses AI to create captivating images right from your browser. Developed within the create-react-app framework, this tool empowers users to produce visually stunning results.
System Requirements:
To embark on your creative journey, ensure you have JavaScript enabled and are using the latest Chrome browser version. A couple of experimental flags, namely “Experimental WebAssembly” and “Experimental WebAssembly JavaScript Promise Integration (JSPI),” should also be activated.
The Creative Process:
The application operates by executing a series of inference steps, each taking approximately 1 minute, plus an additional 10 seconds for the VAE decoder to craft your image. Note that keeping DevTools open may affect performance, potentially doubling the processing time.
For optimal results, we recommend a minimum of 20 steps, although 3 steps suffice for demonstration purposes. Fear not, as model files are cached to eliminate the need for redundant downloads.
User-Friendly Interface:
Stable Diffusion WebGPU boasts a user-friendly interface, allowing you to effortlessly load models, initiate image generation, and instantly view the outcome. In case you encounter any challenges, our FAQ section provides comprehensive troubleshooting guidance.
Performance Considerations:
While our application operates on a GPU, it’s essential to understand that the webgpu implementation in onnxruntime is in its early stages. Some operations may be incomplete, resulting in continuous data transfers between the CPU and GPU, which can impact performance.
At present, multi-threading is not supported, and WebAssembly limitations restrict the creation of 64-bit memory with SharedArrayBuffer. Our developer is actively working to address these issues through proposed spec changes and engine patches.
Open Source and Future Enhancements:
For those interested in diving deeper, the source code for Stable Diffusion WebGPU is available on GitHub, enabling users to run it locally. Additionally, we provide a patched version of onnxruntime, enabling compatibility with large language models through transformers.js, although its reliability in all scenarios is not guaranteed.
Our developer is committed to further improving the tool and plans to submit a pull request to the onnxruntime repository. Stay tuned for exciting enhancements!
Unlock the world of AI-driven image generation with Stable Diffusion WebGPU, and let your creativity soar.