Riffusion App

What is Riffusion ?

Riffusion is an app for real-time music generation with stable diffusion.

It is built with Next.js, React, Typescript, three.js, Tailwind, and Vercel.

• Web app

• Inference server

• Model checkpoint

• Google Colab notebookOpen In Colab

• Gradio Web Demo

Install Riffusion App

This is a Next.js project bootstrapped with create-next-app .

First, make sure you have Node v18 or greater installed using node --version .

Install packages:

npm install

Run the development server:

npm run dev

# or

yarn dev

Open http://localhost:3000 with your browser to see the app.

The app home is at pages/index.js . The page auto-updates as you edit the file. The about page is at pages/about.tsx .

The pages/api directory is mapped to /api/* . Files in this directory are treated as API routes instead of React pages.

Inference Server

To actually generate model outputs, we need a model backend that responds to inference requests via API. If you have a large GPU that can run stable diffusion in under five seconds, clone and run the instructions in the inference server to run the Flask app.

You will need to add a .env.local file in the root of this repository specifying the URL of the inference server:

RIFFUSION_FLASK_URL=http://127.0.0.1:3013/run_inference/