[Bug]: 413 Request Entity Too Large: The data value transmitted exceeds the capacity limit.

[alpilotx]

Is there an existing issue for the same bug?

• I have checked the existing issues.

RAGFlow workspace code commit ID

8939206

RAGFlow image version

v0.15.0 full

Other environment information

VMWare Hardware: 32 Cores, 256 GB RAM
OS: Ubuntu 24.04.1 LTS
Docker Image: RAGFLOW_IMAGE=infiniflow/ragflow:v0.15.0

Actual behavior

In the "File Management" I try to upload 1176 TXT files, having a total of < 5 MByte (essentially they are all C# source code files).
I can select the directory, where it recognizes all the files.
When I click "OK" (to start the Upload), I get the message:

• "<RequestEntityTooLarge '413: Request Entity Too Large'>"

I want to highlight, that according to the "Web developer" output, the upload volume content-length is 4258188! So, thats about 4,06 MBytes.

I have already checked this Bug report:

• [Bug]: show the network error alert when upload large file failed. #1782

And tried to set all referenced parameters to some ridiculous sizes (1 GB!)

• docker/.env
  • MAX_CONTENT_LENGTH=1073741824
• docker/nginx/nginx.conf
  • worker_connections 4096;
  • client_max_body_size 1024M;
• docker/docker-compose.yml
  • MAX_CONTENT_LENGTH=1073741824

But I still get the same error.

When I reduce my upload to a subset, which is 212 Files weighing only 1,02 MBytes ... then Upload works.

Expected behavior

No response

Steps to reproduce

Like describe in "Actual behavior"
Effectively you only need a lot (in my case 1176) TXT Files which all together have as overall size of 4-5 MBytes. Then try to upload this to the "File Management"

Additional information

No response

[Snify89]

Does it behave the same when using HTTP/Python API?

[alpilotx]

Does it behave the same when using HTTP/Python API?

How can this be tested or configured? (while working with Docker-Images!)

[Snify89]

Does it behave the same when using HTTP/Python API?

How can this be tested or configured? (while working with Docker-Images!)

You could use curl with your Ragflow API key and upload them.

[alpilotx]

Ok, I already figured out how to set up Python with the API calls ( I could make it create a new dataset ... so obviously the API communication works) , but I don't really know how I could quickly / from code make it upload the same 1176 files (although I am senior IT guy, I am not a python coder)

[Snify89]

Ok, I already figured out how to set up Python with the API calls ( I could make it create a new dataset ... so obviously the API communication works) , but I don't really know how I could quickly / from code make it upload the same 1176 files (although I am senior IT guy, I am not a python coder)

Try it with a bash script and curl like this:

!/bin/bash


dataset_id="yourdatasetid"
api_key="yourragflowapikey"
host="yourragflow ip/host" 
directory_path="/path/to/documents" 

url="http://$host/api/v1/datasets/$dataset_id/documents"


auth_header="Authorization: Bearer $api_key"

find "$directory_path" -type f -name "*" | while read file_path; do

    file_name=$(basename "$file_path")

    curl --request POST \
         --url "$url" \
         --header "Content-Type: multipart/form-data" \
         --header "$auth_header" \
         --form "file=@$file_path" \
         --form "file_name=$file_name"

    if [ $? -eq 0 ]; then
        echo "File $file_name successfully uploaded."
    else
        echo "Error uploading file $file_name."
    fi
done

[alpilotx]

ugh .... well I managed to get it running via WSL (as my main workstation is Windows ... and the files reside there).
And it seems, this one is working and all the 1176 files all show up in the workspace.

My suspicion is this: your test script is uploading the files one-by-one, while the "File Management" is doing it in one big JSON object (one single upload). This is what web develper hints at when it shows one upload with content-length is 4258188!

Well, I hope you can sort it out now without my input, as I will soon away for a season related time-out.

[Snify89]

It seems to be a frontend issue then, I suppose?

[cike8899]

Are you using nginx to forward traffic on the host outside the docker container? If so, you need to configure client_max_body_size on the host's nginx.

[alpilotx]

Well, as written in detail in my original post, I already set these parameters too:

docker/nginx/nginx.conf
worker_connections 4096;
client_max_body_size 1024M

... to no avail

[cike8899]

Please make sure that nginx is running on your host.
If nginx on the host is forwarding, then the host's nginx needs to be configured with client_max_body_size, not the one in the docker container

[alpilotx]

Please make sure that nginx is running on your host. If nginx on the host is forwarding, then the host's nginx needs to be configured with client_max_body_size, not the one in the docker container

Hmm @cike8899 I am not 100% sure what / how exactly you mean it.
Because to my knowledge I only have the one NGINX instance within the ragflow-server container and no the nginx being active anywhere else.

(and yes, the entire ragflow installation is the official docke compose one running in a docker env on a large linux VM)