Sonify

Sonify is an AI-powered music generator that allows users to create music by providing lyrics or selecting a style of music. The project leverages cutting-edge AI models for generating music, lyrics, and even custom thumbnails for each song. The system is inspired by projects like Suno and Riffusion.

Table of Contents

• Overview
• Screenshot
• Tech Stack
• Architecture
• Features
• Setup and Installation
• Usage
• Contributing
• License

Overview

Sonify allows users to input lyrics or select a desired musical style, and in return, it generates a complete song. The system integrates several AI technologies:

• YuE for generating the music
• Ollama for generating lyrics
• Stable Diffusion for generating an eye-catching song thumbnail

This integration provides a seamless experience where musical creativity is just a few inputs away.

Screenshot

Tech Stack

• Frontend: Vite + Vue.js
• Backend: Flask (Python)

Architecture

The project is structured as a single repository with both frontend and backend components:

• Frontend:
  The user interface built with Vue.js using Vite for fast development and optimized builds. It serves as the primary interaction platform where users can enter lyrics or pick a style.

• Backend:
  A Python Flask API that handles the core functionalities:

  • Music generation using the YuE model.
  • Lyrics generation with ChatGPT.
  • Thumbnail generation with Stable Diffusion.

The components communicate seamlessly to deliver a smooth end-to-end experience.

Features

• AI Music Generation:
  Generate custom music tracks based on user inputs.

• Custom Lyrics Generation:
  Use ChatGPT to create lyrics or complement user-provided lyrics.

• Thumbnail Generation:
  Automatically generate attractive thumbnails for your song using Stable Diffusion.

• User Inputs:
  Support for both lyric-based and style-based music generation.

• Modern UI:
  A sleek and user-friendly interface built with Vue.js for a smooth interaction experience.

Setup and Installation

Prerequisites

• Node.js 22 or later and npm/yarn (for the frontend)
• Python 3.8+ (for the backend)
• Ensure you have the necessary libraries and environment setups for AI model integrations (check documentation from YuE, ChatGPT API, and Stable Diffusion).

Clone the Repository

git clone https://github.com/ChocoMeow/sonify.git
cd sonify

Frontend Setup

1. Navigate to the frontend directory.

2. Install dependencies:

   npm install
   # or
   yarn install

3. Run the development server:

   npm run dev
   # or
   yarn dev

Backend Setup

1. Navigate to the backend directory.

2. Create a virtual environment and activate it (optional but recommended):

   python -m venv venv
   source venv/bin/activate  # On Windows, use: venv\Scripts\activate

3. Install the required Python packages:

   pip install -r requirements.txt

4. Rename settings Example.json to settings.json and customize your settings.

5. Run the Flask application:

   flask run

Usage

1. Open your browser and navigate to http://localhost:5173 (or your configured port).
2. Use the interface to either:
   • Enter your lyrics
   • Select a musical style
3. Click the Generate button to create your music. Your song will be generated using the backend APIs integrating YuE, ChatGPT, and Stable Diffusion.
4. Enjoy your custom AI-generated track and thumbnail!

Contributing

Contributions are welcome! If you have suggestions for improvements or features, feel free to submit a pull request.

License

This project is licensed under the MIT License.