LLM-Aided FPGA Design for Sobel Filter Implementations with HLS: Memory and Latency Optimization

This repository contains three different implementations of the Sobel filter algorithm using High-Level Synthesis (HLS) for FPGA deployment. The implementations showcase the evolution of the design from a memory-intensive approach to a highly optimized version in terms of both memory usage and latency, leveraging LLM-based optimizations.

Overview

The Sobel filter is an edge detection algorithm widely used in image processing. This repository provides a comparative study of different HLS implementations, highlighting the impact of design choices on resource utilization and performance.

Design Evolution

The repository is structured into three main folders, each representing a different stage in the design's evolution:

1. sobelFilter: This folder contains the original design, which loads the entire input image into FPGA local memory. While straightforward, this approach leads to significant memory overhead, especially for high-resolution images.

2. sobelFilter_opt1: This folder contains a memory-optimized design generated by Gemini 2.0 Flash LLM. The LLM was prompted to reduce memory usage by processing the image row by row and using line buffers. This significantly reduces the memory footprint compared to the original design.

3. sobelFilter_opt2: This folder builds upon the memory-optimized design by further improving latency. The LLM was prompted to define a 3x3 shift register for a sliding window of input pixels and use two memory buffers to store the previous two lines of image pixels, ensuring that Sobel filter operations are performed once per clock cycle.

Design Concept

The following diagram illustrates the design concept and optimization strategies employed in this project:

Folder Structure

• sobelFilter:

  • sobel.h: Header file for the original Sobel filter implementation.
  • sobel.c: C source code for the original Sobel filter implementation.
  • tb.c: Testbench for the original Sobel filter implementation.

• sobelFilter_opt1:

  • sobelFilter.hpp: Header file for the memory-optimized Sobel filter implementation.
  • sobelFilter.cpp: Memory-optimized C++ source code for the Sobel filter, generated by Gemini 2.0 Flash LLM.
  • sobelFilter_tb.cpp: Testbench for the memory-optimized Sobel filter implementation.
  • png2txt.py: Python script to convert a PNG image to a text file for the testbench.
  • txt2png.py: Python script to convert the output text file back to a PNG image.
  • run_hls.tcl: Tcl script for running the Vitis HLS flow.
  • pic_in.txt: Input image data in text format.
  • pic_out.txt: Output image data in text format.
  • test.png: Example input image.
  • prompt.md: Prompt used to optimize HLS C++ design.

• sobelFilter_opt2:

  • sobelFilter.hpp: Header file for the latency-optimized Sobel filter implementation.
  • sobelFilter.cpp: C++ source code for the latency-optimized Sobel filter implementation.
  • sobelFilter_tb.cpp: Testbench for the latency-optimized Sobel filter implementation.
  • png2txt.py: Python script to convert a PNG image to a text file for the testbench.
  • txt2png.py: Python script to convert the output text file back to a PNG image.
  • run_hls.tcl: Tcl script for running the Vitis HLS flow.
  • pic_in.txt: Input image data in text format.
  • pic_out.txt: Output image data in text format.
  • test.png: Example input image.
  • prompt.md: Prompt used to optimize HLS C++ design.
  • readme.md: This file, providing an overview of the project.

Requirements

• Xilinx Vitis HLS
• Python 3
• PIL (Pillow) library for Python (pip install Pillow)

Usage

Each folder contains a complete HLS project that can be synthesized and implemented on an FPGA. Follow the steps below to run each implementation:

1. Generate Input Stimulus:

   • Place the input image (e.g., test.png in sobelFilter_opt2) in the same directory as png2txt.py.

   • Run the png2txt.py script to convert the PNG image to a text file (pic_in.txt):

     python png2txt.py

   • Modify the input and output paths, and resolution in the script as needed.

2. Run Vitis HLS:

   • Open Vitis HLS and navigate to the project directory.

   • Execute the run_hls.tcl script:

     vitis_hls -f run_hls.tcl

   • This script performs C simulation, C synthesis, and co-simulation. You can modify the script to enable or disable specific steps.

3. Verify the Output:

   • After the HLS flow completes, the output pixel data will be in pic_out.txt.

   • Run the txt2png.py script to convert the text file back to a PNG image (output.png):

     python txt2png.py

   • Modify the input and output paths, and resolution in the script as needed.

   • View the generated output.png to verify the Sobel filter's output.

LLM-Based Optimizations

The key optimizations in sobelFilter_opt1 and sobelFilter_opt2 were achieved by prompting the Gemini 2.0 Flash LLM with specific design goals. The prompt.md files in sobelFilter_opt1 and sobelFilter_opt2 contain the prompt used to guide the LLM in optimizing the HLS C++ design, emphasizing reduced memory usage and minimized latency.

Notes

• Ensure that the input image resolution matches the WIDTH and HEIGHT parameters defined in the header files (e.g., sobelFilter.hpp).
• The performance and resource utilization of each implementation will vary depending on the target FPGA device and HLS settings.