README.md

memegraph

Memegraph is a search engine and recommendation system for memes using OpenAI CLIP.

Search memes via keywords, explore memes with the same template, and discover memes that are visually similar.

Don't know the name of a meme? Just describe the meme!

Discover memes about your favorite topics

Looking for THAT obscure meme? Look it up!

Discover similar memes

New way to browse memes

Development

Important

This app uses MLX to run CLIP. To generate embeddings, a device with an Apple Silicon processor is needed. For non-Apple Silicon devices, the app can still run, provided there are already images and embeddings in the database, but it will not be able to search for memes by image similarity.

Install dependencies

pip install -r requirements.txt

Download OpenAI CLIP model (clip-vit-base-patch32) from Hugging Face and convert to Apple's MLX

python clip/convert.py

The model is dowloaded to clip/mlx_model/ directory unless argument --model_dir is provided.

Modify main.py to point to the correct model. This contains the functions to generate the embeddings.

Modify script.py to point to your images directory, blob storage, and Postgres database.

Generate image embeddings, store images in an blob storage bucket, and store embeddings in a database

python script.py

Run the app

python app.py

Data

All memes were retrieved from the top 1000 posts of all time on Reddit's meme subreddits using PRAW on the Reddit's API free tier.

Notes from the author

Python is not my primary language and I'm new to machine learning, so I greatly appreciate and welcome constructive feedback!